Participatory Sense-making as Consensual Validation of Phenomenal Data

INTRODUCTION

In this article, I present a method of consensually validating phenomenal data. In qualitative research, consensual validation refers to the process of checking with our co-researchers whether the categories induced during the analysis of raw data correspond to their subjective reports[1]. I propose to make use of participatory sense-making to establish intersubjective vocabulary to describe specific aspects of our co-researchers’1 experience. I understand phenomenal data to be consensually validated when a group of co-researchers possesses a vocabulary with which they can describe their experience. This method of consensual validation constitutes a fusion of the prismatic approach and participatory sense-making, supported by theoretical discussions. The current iteration of the method started with the prismatic approach, a method for the study of embodied social dynamics. Based on my co-researchers’ feedback, the method of consensual validation subsequently underwent a number of modifications. The aim of this article is to offer a method of validating phenomenal data as the term is understood by Varela and Shear: subjective reports on lived experiences rather than philosophical intuitions about experience2[2]. While consensual validation has been claimed in various studies[3, 4], a detailed protocol has thus far not been described. Considering scientific transparency[5], in particular when it comes to qualitative research[6], explicit protocols for consensual validation are necessary. The article is structured as follows: In the first section, I draw a comparison between phenomenology and natural history as exemplars of descriptive sciences, arguing that nominalization (i.e., naming the objects of inquiry) in both constitutes intersubjectively accessible data. This discussion works to support the position that only such data can be considered valid. In the second section, I argue that any vocabulary with which we might refer to shared experience needs to be autonomously constructed. In the third section, I present how the method was developed. In the fourth section, I present my guidelines for consensual validation of phenomenal data. In the fifth section, I discuss the epistemic status of phenomenal data validated with our approach. A final introductory note: the method of validating phenomenal data proposed in this article is agnostic regarding the method that was used to acquire the data in the first place. It operates under a specific theory of knowledge that may or may not be admissible by some methodological frameworks in first-person research. As such, this method of consensual validation is not meant to be universal. It amounts to merely a proposal. Researchers should determine on a case-by-case basis whether the proposed method of consensual validation suits their research question and epistemological commitments.

CONSENSUAL VALIDATION AS INTERSUBJECTIVE ACCESSIBILITY: THE CASE OF PHENOMENOLOGY AND NATURAL HISTORY

In this section, I discuss the nature of intersubjectively valid data, particularly in the context of descriptive sciences. Descriptive sciences, broadly construed, are the sciences that outline the properties of a given phenomenon, rather than offer the conditions that give rise to said phenomenon or the mechanisms of how the phenomenon operates. What for many researchers precludes the scientific study of experience is the claim that phenomenal data are subjective, i.e., that experiential reports are inaccessible to a community of researchers, and thereby cannot constitute objects of scientific inquiry[7]. Namely, two researchers cannot observe phenomenal data as it is only accessible in the first person. I argue, however, that through nominalization2 phenomenal data can asymptotically approach the point where the overlap between experiences of different individuals is considerable enough to denote an intersubjectively accessible phenomenon, provided that the method of acquiring the original phenomenal data is systematic. Intersubjective accessibility would make phenomenal data accessible to a community of researchers (i.e., they would be able to agree regarding the nature of experience under investigation). This approach is valid within natural history. By analogy, I believe that this holds in descriptive sciences across the board, including phenomenology. I will use the example of natural history to show that descriptive sciences primarily rely on nominalization, i.e., naming of phenomena under investigation, to establish intersubjective consensus about what they are studying. A common argument leveled against the scientific study of experience is that such a project can be merely descriptive[8] or that it can provide only lists of contents of consciousness[9]. The derision of descriptive science has been argued to be more of a prejudice[10, 11]. Biology and astronomy were both descriptive sciences[12, 13]; today, calls have been made to return descriptive work into the purview of biological sciences in general[14, 15], and descriptive approaches have become the gold standard in genetics[16]. A broader defense of descriptive sciences is beyond the scope of this article. I merely wish to suggest that many scientific disciplines rely heavily on a variety of descriptive frameworks. I will now argue for descriptive sciences using nominalization as a method of rendering their objects of investigation intersubjectively accessible. An example of a descriptive science is biology, specifically when it assumed the form of natural history. Natural history refers to the Modern Era scientific program that aimed at establishing a taxonomy of the natural world. As Michel Foucault[17; p.114] writes in The Order of Things, “natural history is nothing more than the nomination of the visible.” In explanation, 18th century naturalists relied on visual epistemology to investigate the natural world; i.e., the idea that depictions in and of themselves carry epistemic value. Practically, this means that scientists were trained in how to observe, represent, and describe the natural world. That is, the descriptions and depictions provided by experts, trained in scientific observation of the world, were ascribed a truth value. Expert descriptions of plant-life necessitated a philological approach towards the construction of knowledge, i.e., the comparison of different reports had to be conducted both at the level of observation (either of actual preserved specimens or detailed illustration) and the careful study of botanical texts. One could argue that natural history is no longer an accepted scientific program. Note, however, that genetics still constitutes a descriptive science[16]. I argue that there is significant similarity between the methods of natural history and phenomenology3. Much like how observation, description, comparison, and classification constituted the method of natural history (e.g., the Linnaean taxonomy of organisms[18]), phenomenology aims at classifying experience. The centrality of classification in phenomenology is summarized by Jean-Paul Sartre[19; p.5] in his account of imagination: “[P]roduce images in ourselves, reflect on these images, describe them, which is to say, try to determine and classify their distinctive characteristics.” The taxonomic streak in Sartre is self-evident, however, an important difference between phenomenology and natural history lies in the epistemic value of philology and visual epistemology. The central method of philology, i.e., the exegesis of texts, is criticized heavily by phenomenologist Paul Ricoeur[20; p.91]: All interpretation places the interpreter in medias res and never at the beginning nor at the end. We happen upon a conversation which has already begun and in which we try to orient ourselves in order to make our own contribution to it. But the ideal of an intuitive foundation is that of an interpretation which, at a certain moment, would become a total vision. In other words, the philological method relies solely on the interpretation of texts for the construction of knowledge, while Husserlian phenomenology relies on an intuitive understanding of the subject at hand as well (although, see also[21]). Similarly, visual epistemology is somewhat problematic in phenomenology. Edmund Husserl’s schematized depictions of time consciousness[22], for instance, were criticized and later on amended both by Maurice Merleau-Ponty[23] and Francisco Varela[24] (for a more recent attempt at a visual presentation of phenomenal data, see also[25]). I argue that this controversial status of visual depictions of experience stems from loss of intuitive information about the subjective dimension of the mind when reducing phenomenal data to merely the visual modality. Now, to move from the divergences between the epistemologies of natural history and phenomenology back to their similarities. The biggest commonality between phenomenology and natural history is that they rely on a specific context of observation to achieve an understanding of their respective objects of inquiry. While knowledge construction is indeed supplemented by texts, the principal way in which naturalists obtained knowledge was through the eidetic (i.e., knowledge-giving) intuition of vision[26]. In Visible Empire, a monograph on 18th century botany, Daniela Bleichmar[27; p.47] writes that naturalists posited two ways of observing the world: the spectacular gaze and the scientific gaze, where [t]he word “spectacle” does not connote superficial entertainment: while amusing and pleasurable, nature is always instructive. […] [T]he word “spectacle” refers to a mode of seeing predicated on notions of transparency and immediacy, a way of looking that was open to everyone, regardless of background, and required no specialized training. By contrast, scientific observation consisted of uncovering the underlying order of nature. Its “goal was not simple, immediate looking but rather expert observation, going beyond superficial traits to focus on the significant (ibid.).” The importance of specialized observation is the biggest difference between natural history and philology proper. Specifically, natural history never relied on texts as the sole source of knowledge[28]. As Bleichmar[27; p.46] writes: “Eighteenth-century natural history publications repeatedly proclaim that vision constitutes the best method for investigating nature and that images provide the preferred means of transmitting this knowledge.” It is the eidetic intuition of vision that is the true bearer of knowledge in natural history rather than the texts themselves. Phenomenology relies on a specialized form of observation as well. It observes experience while bracketing [An. Gr. epoché; Ger. Einklammerung] the natural attitude. The idea of the natural attitude is that before reflecting on it, we exist immersed in a world that quite simply appears to be there and we are uncovering it with our senses. Our understanding of the world, of the entities that inhabit it, and of our own consciousness, is laden with assumptions and theories. As Edmund Husserl[29; p.2] writes in Thing and Space: In the natural attitude of spirit, an existing world stands before our eyes, a world that extends infinitely in space that now is, previously was, and in the future will be. This world consists of an inexhaustible abundance of things, which now endure and now change, combine with one another and then again separate, exercise effects on one another and then undergo them. However, if we are to form a theory of experience that takes into account the properties of experience, rather than imposing upon them a framework of natural sciences[30-34], we must learn how to separate ourselves from the natural attitude. We can do this by performing the act of bracketing. Bracketing refers to suspending the assumptions and theories we hold about the mind, experience, the world and our existence in the world, including scientific theories about them[35]. It refers to an attitude of wonder before the world[36]. We must note that the spectacular gaze and the scientific gaze do not map onto the natural attitude and the act of bracketing, respectively. Instead, both the spectacular gaze and the scientific gaze exist within the natural attitude [for an empirical account, consider[37]. Indeed, in his preface to the French translation of Husserl’s[38; p.xx] Ideas Pertaining to a Pure Phenomenology and Phenomenological Philosophy, Ricouer places the whole of natural history within the domain of the natural attitude: I am at first lost and forgotten in the world, lost among things, in ideas, among plants and beasts, among others […] We understand Naturalism as the lowest degree of the natural attitude and at this level it conveys its own collapse; because if I am lost in the world, I am already ready to treat myself as a thing in the world.4 However, the approach of natural history may guide us in constructing consensually validated phenomenal data. Specifically, if naturalism represents a way of nominalizing the visual world as seen by expert observation, phenomenology may be the way of nominalizing the experiential world. By adopting the assumptions of natural history, I hold that when nominalization of the experiential is done in a systematic way, it is rendered intersubjectively accessible. The question, however, is how can we establish a vocabulary that would refer to specific aspects of experience?

PARTICIPATORY SENSE-MAKING AND AUTONOMOUS NEGOTIATION OF MEANING

The idea of establishing a shared vocabulary with which to refer to the same aspects of a given experience is not new. For example, during Heinrich Klüver’s research into the structure of hallucinations, he discovered that his participants were prone to giving mystical descriptions of their experience. To circumvent this problem, he trained his participants beforehand by providing them with concepts with which to describe their experience[39]. Further, the idea of naming a specific aspect of one’s experience is part and parcel of the qualitative approach to research, where the basic analytical tool is coding. We ascribe abstract descriptions to qualitative reports (typically rendered in the form of text). At the highest levels of abstraction, we may induce categories that are nothing more than words or phrases[40]. However, it is typically the researchers who name the categories under investigation and present them to their participants for them to validate the coding process5. From the point of view of phenomenology, this approach is problematic as what is established is not a vocabulary that could then be organically used to describe aspects of experience, not to mention that empirical phenomenological studies commonly do not even engage in the process of data validation with their participants. Research has shown that if we wish to create a lexicon within which the meaning is grounded in the outside world, the lexemes must be autonomously constructed and negotiated by different agents in the interacting community[41]. Luc Steels attempted to solve the symbol grounding problem – how symbolic structures relate to the outside world[42] – by constructing artificial agents which at any time could assume control over one of two cameras placed in front of a board of colored shapes. When two artificial agents took control over the cameras, they could point to the same shape and name it. If one of the agents already had a name for the shape and the other one did not, the latter simply adopted the name. If both agents already had a name for it, they negotiated a new name for the shape. Eventually, the community of artificial agents negotiated a shared vocabulary for all the shapes on board[41]. Steels argues that an autonomous negotiating of meaning is a necessary condition for relating a symbolic structure to an object in an outside word (i.e., for grounding a symbol). By analogy with the symbol grounding problem, the question for empirical phenomenology is how to create a vocabulary to describe the experienced (rather than outside) world. How can we then go about solving the problem of the construction of a vocabulary with which we would be able to refer to those aspects of our experiential world that are brought to the fore by a specific research situation? One possible approach was suggested by Hanne de Jaegher[43], and Elizaveta Solomonova and Sha Xin Wei[44]: participatory sense-making. Participatory sense-making refers to an enactivist theory of social cognition and language. The idea behind participatory sense-making is that within social interaction, cognizers coordinate their behaviors such that the social interaction itself becomes autonomous. By constraining each other, the cognizers modify their behavior (either bodily or linguistically) so as not to bring the interaction to an end[45][46]. In the development of our method, the position of participatory sense-making seriously was taken seriously. It was incorporated both as a means of validating data (as will be explained below, the co-researchers have to agree on the framework of nominalization), as well as a criterion for validated data (i.e., the vocabulary needs to be conducive to establishing an autonomous interaction). These theoretical discussions lead to an important methodological consideration: we do not present our co-researchers with already formed categories (although we may have provisionally constructed them during the analysis of phenomenal data). Instead, we provide them with ethnographic descriptions of experience, constructed from several reports (see below). During the validation sessions, the community of co-researchers then collectively ascribes names to these descriptions, or – if the descriptions do not correspond to their experience – either divide them further or group them together. In doing so, we further equalize the participant-researcher power dynamics: We invite our co-researchers to be part of the data analysis process rather than merely the data acquisition process. This autonomous, collective construction of a vocabulary with which to refer to the co-researchers’ experience constitutes the central innovation of this approach. Further, it represents the central criterion for whether we have managed to establish a vocabulary with which to describe our experience: if, by the end of the validation session, the co-researchers are able to discuss their experience naturally with our vocabulary without their interaction breaking down, the phenomenal data can be considered consensually validated. This principle is illustrated in Figure 1.

Figure 1. A schematic depiction of using participatory sense-making for consensually validating phenomenal data.

CONSTRUCTING A METHOD OF CONSENSUAL VALIDATION

In this section, I present how the method of intersubjectively validating phenomenal data was developed. I first present the starting point: the so-called prismatic approach. Then, I present how this approach was augmented using participatory sense-making and philosophical phenomenology. Finally, I discuss how the debriefings on individuals’ experience of validating phenomenal data were used to further improve this method of consensual validation. The method was developed alongside a longitudinal empirical phenomenological project that investigated the experience of solving a visual span task[47]. Throughout the remainder of the text, the reader should keep in mind that the validation sessions were conducted after the processes of data acquisition and analysis. As such, they do not serve as a means of gathering data. Rather, they are aimed at testing and improving the findings of the qualitative analysis. For context, I will now briefly outline the research design of the study.

RESEARCH DESIGN

The visual span task is a psychological task used to measure how many objects, presented in the visual modality, and individual can maintain in her working memory. The visual span task consisted of a presentation of a grid, in which certain cells were filled in. The grid was presented for 2 500 milliseconds. Immediately after the grid disappeared, an empty grid appeared, and our co-researchers had to fill in the black cells to match the first grid. If the co-researchers successfully reconstructed the grid, they received positive feedback, and the difficulty of the task increased. Conversely, if the co-researchers were unsuccessful in reconstructing the grid, they received negative feedback. Upon making two subsequent mistakes, the task stopped, and the co-researchers received a number denoting the span of their visual working memory. Figure 2 depicts an example of a to-be remembered grid. The co-researchers were stopped after a random trial[48], and a phenomenological interview followed. The interview explored the time interval spanning the presentation of the stimulus, the delay, and the reconstruction of the stimulus. The interview procedure was structured based on a broadly qualitative approach. In the first part, the interview was open-ended, gathering phenomenal data based on the co-researchers’ experience. Afterwards, theoretical sampling[40] was used to address specific hypotheses about the experience of a visual working memory task. All the interviews were conducted by the author of this article. After conducting phenomenological interviews on the experience of solving the visual span task, a closed-form debriefing, based on extant discussions on the validity of phenomenal data, was used to ascertain the quality of the interview. Four interviews were conducted with each co-researcher. During each interview, only one sample of experience was gathered. The data were analyzed according to the principles of constructivist grounded theory[40]. The study focused on two aspects of experience: the strategies used to solve a visual working memory task, and the attitudes individuals take towards the psychological task. In the analysis process, experiential categories were constructed according to these two research interests.

Figure 2. Example of a stimulus in the visual span task.

VALIDATING THE DATA IN THE VISUAL-SPAN STUDY

Once the phenomenal data were analyzed, consensual validation workshops informed by the prismatic approach were conducted. Consensual validation was performed in order to a) validate the analysis protocol; and b) establish an intersubjective vocabulary with which to refer to different aspects of experience of solving a visual span task. The prismatic approach to gathering data on embodied social dynamics was developed by Barbara Pieper and Daniel Clénin[49, 50]. According to this approach, the participants of a given study attend a workshop whose goal is to investigate a specific aspect of their experience. At this workshop, the participants engage in the specific embodied social dynamic that is being investigated by jointly performing a given practice and observing how the related social dynamic unfolds. In particular, they are interested in various ways this social dynamic may or may not manifest itself. From this workshop model suggested by the prismatic approach, the following elements were adopted: 1. The joint investigation of experience; 2. Reacquainting participants with the aspect of experience under investigation; and 3. Observing different ways in which a given aspect of experience may appear to individuals. In line with the ideas of reflective cognitive science[51], in particular when it comes to the scientific investigation of experience[48; Ch.21], debriefings on the experience of validating the data were conducted . As several critical points entered the co-researchers’ awareness, the approach to consensual validation was iteratively refined. In total, 11 co-researchers participated in these workshops (up to four per session, spread out across four sessions). Each co-researcher (except for the principal investigator) attended only one workshop. Each workshop began with a brief lecture delivered by the principal investigator outlining the findings of the study (e.g., that we can decompose the attitude individuals take towards a visual working memory task into three dimensions), as well as the purpose of the workshop. During this initial lecture, it was emphasized that the goal is not to confirm the way the principal investigator ordered the phenomenal data during the qualitative analysis, but to challenge it. The goal of the lecture is to attempt to create as equal relationship as possible between the co-researchers. While pragmatically there is still a principal investigator guiding and organizing the study, her authoritative role should be reduced. A sense of equality among co-researchers allows for a discursive space to be opened where the co-researchers feel comfortable challenging and critiquing the current understanding of the phenomenal data. During the validation workshops, each participating co-researcher assumed both the role of the person observing her experience and the role of interviewing the person observing her experience. Co-researchers sat together in the same room. The co-researchers took turns in solving the visual-span task, while observing their experience. All the other co-researchers guided the individual who performed the visual-span task through an empirical phenomenological interview. The interviews were done jointly by the whole group: that is, one person performed the visual span task while observing her experience, and all the other co-researchers interviewed her together. This process was repeated until every participating co-researcher was in the role of observing her experience. The structure of the interview used by the group of co-researchers took the same form as during the process of data acquisition in the visual-span study: it began by a broad outline of experience, followed by pointed questions, aimed at describing each individual aspect of experience. Further, the iterative process of observing and reporting on one’s experience, and interviewing the reporting co-researchers, was repeated for every aspect of experience under investigation (i.e., one round to gather phenomenal data on strategies used to solve a visual span task, and one round to gather phenomenal data on attitudes taken towards the task). Afterwards, the validation workshop took the form of a group discussion where we compared the way the principal investigator analyzed the data, and how the co-researchers’ lived experience compares. The co-researchers were able to point out where the current state of analysis corresponds to their experience and where it deviates from it in a group setting. The last stage of the workshop was to attempt to jointly construct mutually agreed upon names for the aspects of experience under investigation. After the validation session, the co-researchers were given the following semi-structured questionnaire to report on their experience of the validation session itself: 1. Do you believe we succeeded in establishing a common language to describe the experience of solving a visual working memory task? 2. Do you believe that the group setting changed your experience or your reporting on the experience? 3. Do you believe that aspects of experience that you reported on during the group discussion represent your experience during the performance of the visual working memory task? 4. Do you believe that having to report on your experience in language changed your experience? 5. What do you think could be changed in our approach to consensual validation? Based on the responses to these questions, the structure of the workshop was iteratively modified: the subsequent workshop took into account the feedback from the previous one and was adapted accordingly. In the next section, the guidelines towards consensually validating phenomenal data by using participatory sense-making are presented.

PARTICIPATORY SENSE-MAKING AS A METHOD OF VALIDATING PHENOMENAL DATA: OUR GUIDELINES

In the present section, the method of validating phenomenal data is presented. Crucially, the validation workshops are conducted after the process of data acquisition is complete and on the same sample of co-researchers. Based on the theoretical principles outlined in the first two sections, and the feedback gained from our workshops, the following guidelines towards using participatory sense-making as a method of consensually validating phenomenal data were reached. 1. The starting point are not categories constructed during the qualitative analysis of the data, but composite ethnographic descriptions; 2. (If possible) the experiences under investigation are provoked during the validation session; 3. Co-researchers are asked to observe whether it is necessary that their experience corresponds to the composite descriptions or whether it can deviate; 4. Co-researchers collectively construct the vocabulary with which to refer to their experience; 5. The co-researchers once again observe their experience (if that is possible), and describe it with the newly-established vocabulary. In this section, we will take a closer look at each of these steps, providing principled and empirical support for them.

COMPOSITE DESCRIPTIONS OF EXPERIENCE

The starting point is phenomenal data that has been subjected to some form of analysis. At present, this method of consensual validation is not committed to adhering either to approaches to data analysis that are tailor-made for experiential reports[52][53] or qualitative data in general[40][54]. Rather, in light of qualitative research’s commitment to methodological pluralism[55], I consider various approaches to data analysis equally valid. Importantly, we do not set out to validate named categories as established by the qualitative analysis of the phenomenal data. Rather, we present our co-researchers with composite ethnographic descriptions. Composite descriptions refer to a style of presentation whereby we do not present concrete descriptions of the phenomenon under investigation, but we combine a number of descriptions and abstract them away into a typical and telling example[56]. The principal investigator presents these composite descriptions as well as a detailed research design during an introductory lecture. The purpose of this lecture is to inform the co-researchers of all the aspects of the study (which may have been unclear to them during the initial interviews), as well as to explicitly let them know that there is nothing that is being hidden from themThis position of empowerment is crucial so as to minimize the influence of demand characteristics[57] on their reports. Specifically, we do not wish for them to attempt to guess what the goal of the research is[58]. Rather, we explicitly inform them about the research goal, and then challenge them to attempt to challenge our categories. This challenge is both stated during the introductory lecture and made a part of the process of going through experiential categories later on6. It is important that the overall arc of the workshop is clear to the co-researchers. As one co-researcher reports in her feedback on the workshop: Because we were mostly talking, I feel that everything was left hanging in the air. We never reached any clear conclusions. We could write down different ways in which others experienced things and then through conversation see which experiences overlap or are related to each other. (Co-researcher 14, feedback on the workshop)

The feedback that the discussions were floating in the middle of nowhere was shared by many co-researchers, in particular those who were involved in longer sessions. I suggest that ahead of the validation session, the co-researchers be given all the composite descriptions with room allocated for notes and eventual names. This allows them to make notes on aspects of experience not currently under investigation, as well as give them a sense of completion.

REACQUANTING THE CO-RESEARCHERS WITH THE EXPERIENCE UNDER INVESTIGATION

Once we make the aim of the study known, we invite the co-researchers to reacquaint themselves with the experience under investigation. If this experience can be easily induced (e.g., by means of a psychological task), each co-researcher is given the possibility of privacy to observe their experience once again (i.e., we provoke it)[3,52, 53]. If, on the other hand, the experience cannot be reproduced, the co-researchers are given time to consult their journals. Importantly, reacquainting with the experience should be done in a setting that reflects the social context of the original investigation, as was reported by one of our co-researchers:

We should not do the task together during the group session (with people watching what you are doing) because this aspect changed my experience quite a lot and was pretty distracting. I felt less secure about my report than when I did the task on my own during the [original sessions], and my experience was quite a bit different back then. (Co-researcher 11, feedback on the workshop)

JOINT INVESTIGATION OF EXPERIENCE

Then, the co-researchers interview a single individual on her experience. We continue with this process until every co-researcher has been in the role of the reporting on their experience. In other words, individuals who had previously simply participated in the study (by observing and reporting on their experience) now help guide each other towards observing and describing their experience (i.e., they become interviewers). During the process of mutual interviewing, the co-researchers are invited to again observe their experience or to read up on their journal notes. They are prompted to observe specific qualities of their experience: 1. is the experience under investigation necessarily such as it was observed during the qualitative analysis, or can it be modulated with specific mental gestures? 2. can you observe specific differences between two closely related experiential categories? This line of questioning (i.e., observing whether a specific aspect of experience is necessarily structured in such a way as observed during the acquisition of phenomenal data or if it can differ in some respect) may be problematic within schools of thought in empirical phenomenology that emphasize open-beginninged attitude towards gathering data[3,48]. As will be explained in Section 5, the presented method for validation of phenomenal data adheres to process-oriented constructivism. Namely, we are trying to establish a framework that is contingent (i.e., there are potentially infinite possible ways of dividing the categories), and simultaneously internally consistent. In this case, an example of an internal inconsistency would be two categories that would describe the same aspect of one’s experience. A similar approach was adopted by Husserl[35; p.220] (emphasis in the original): Since every negatum and affirmatum is itself an Object posited as existent, it can, like everything intended to as having a mode of being, become affirmed or denied. In consequence of the constitution of something as existent effected anew at every step, an ideally infinite chain of reiterated modifications therefore results. Or, on Andrea Staiti’s[59; p.815] paraphrase: “Higher-order affirmation or negation occurs when we set out to revisit a foregoing simple judgment in order to confirm or disconfirm the veridicality of its proposed state of affairs.” In explanation, we check whether the structure of a specific aspect of experience – as observed in the acquisition of phenomenal data (whether it be through philosophical intuition or second-person methods) – reflects the lived experience of our individuals, we attempt to provoke experiences that either conform to or deviate from it. During the joint phenomenological interview, our goal is to contrast different experiences of the same phenomenon, trying to ascertain which aspects of experience necessarily remain the same across individual co-researchers, and which aspects may vary. These variations in experience need not be explicit. As the following co-researcher reports, just witnessing how other people experience the same experimental setting, may prompt them to observe their own experience differently: My experience changed in that I started to pay attention to new aspects of experience that were described by the others but that I was not aware of before. I tried to check if I had a similar experience as them, that I just had not realized before, or if these aspects were completely absent to me. But actually, I do not think that my experience changed by that but rather the awareness of my experiences. (Co-researcher 11, feedback on the workshop) I will now demonstrate two examples of the attempt at challenging the induced experiential categories. To reiterate: these categories were first induced by the principal investigator during the qualitative analysis stage of the study. In the validation workshops, the co-researchers are presented with a composite description of the experiential category. They attempt to discover whether this experiential category is something that they can observe in their experience or if it deviates from it. The first example of the validation interview during the workshops represents a case that led to an elimination of the category experimental orientation – how it feels to be a participant in a psychological experiment. Importantly, during the initial process of data acquisition, several reports were gathered on how one’s experience changes when participating in an experiment: Co-researcher 7: Certainly, [being in an experiment] feels different. It cannot be compared to anything experienced outside of this setting. So, trying to think of the things I do inside of this setting, of the skills you need to memorize it, whether it’s committing a poem to memory, it’s different, because you are not trying to compare yourself to others. And then also, sort of the complete setup, of somebody watching you. It was not that I was thinking I have to prove myself, I have to solve 75 of them, but it certainly alters how you engage with it. And also what you perceive and notice. (Co-researcher 7, validation interview) Co-researcher 9: I just noticed another really important dimension about how it feels to be me in an experiment. It’s this dimension of how much you are trying to understand the task itself behind everything. So, going through one pattern, and another pattern, it is one thing. But I often simultaneously try to understand how the task itself works. I mentioned that in our Interview. I noticed that it always starts with four squares, and every time you get two of them correct, the number increases by one. And in this way, you can predict how many black squares in total will appear in the next square, and I was using this knowledge to check if I have all of them. I counted them and I knew how many of them there had to be. (Co-researcher 9, validation interview) The consensual validation revealed that we may break this experience down into its various social dimensions, as well as the attitude of being goal oriented. While these two aspects of experience are prevalent when somebody is undergoing an experimental situation, they are not unique to it. Consider the following example: Co-researcher 7: It sometimes feels in a museum that I have this divide-and-conquer feeling. How I look at the image and parts of the image, and I try to understand what it shows, what it means. That can be very prominent as a strategy that unfolds itself. So, sometimes going to the museum can be quite exhausting for me and sometimes it does not feel like I am going there to enjoy the paintings but that I am going there to look and understand. That sort of creates this task-mentality. (Co-researcher 7, validation interview) Prior to the validation study, a category experimental orientation was induced during qualitative analysis. Experimental orientation was an experiential category that described the particular atmosphere of experience that is present when one is interacting with a psychological task in a research setting. Experimental orientation satisfied many of the standard criteria in qualitative research for it to be induced as a specific category: 1. it was reported by a number of different co-researchers; 2. it was clearly distinguishable from other categories, and we observed a number of limiting cases (i.e., clear instances of phenomenal data that did not conform to the category of experimental orientation). However, during the validation session it became apparent that while the experience of solving a visual-spatial working memory task was a unique experience within the context of this study, the experience of participating in a study was not. This led me to abandon it as an experiential category, as it amounted to an interpretation on the part of the principal investigator, rather than a faithful reflection of the lived experience of the co-researchers. For the second example, let us look at how we constructed a new category during validation sessions. During the qualitative analysis of the data, two categories related to association were induced. The first was an experience in which a to-be-remembered stimulus is accompanied by a visual feeling of something that it resembles. This visual feeling appears as a mental image that is either projected in the outside world, or exists in an internal mental space. The second was an experience in which a to-be-remembered stimulus is accompanied by a clear idea of what it resembles. This idea may be explicitly articulated in an inner voice. This potential category may be demonstrated with a picturesque example observed in the study. A co-researcher was reminded of the swastika by the to-be-remembered stimulus. As I will demonstrate the differences between the established categories visually, I will replace the swastika with the symbol of the fictional state of Tomania from the film The Great Dictator[60]. In the co-researcher’s experience, this association was accompanied by mental imagery. Ultimately, this aspect of experience was named “visual image.” Another co-researcher experienced a sense of the to-be-remembered stimulus being like something that might be used as a symbol in a totalitarian regime. As he noticed that this configuration of black-and-white squares was fairly common, he ascribed the meaning of totalitarian iconography to it. This aspect of experience was ultimately named “symbolized description.” These two aspects of experience are depicted in Figure 3 as a) and b). During the validation session, however, we concluded that visual image corresponds to two aspects of experience: one that is comprised by the experience of imagery, and the other comprised of various existential feelings[61]. Thus, we introduced a new category: atmospheric image. In Figure 3, the new categories are depicted under headings c) and d). Such a refinement and removal of categories is the central goal of the joint interview: we wish to find aspects of experience that are stable and understandable across co-researchers.

Figure 3. Categories associated with association.

NOMINALIZING EXPERIENTIAL WORLDS

After all the co-researchers were both in the role of observing and reporting on their experience, and guiding others through a second-person phenomenological interview, we collectively decide on how to name a specific aspect of experience. For example, consider the following exchange between four co-researchers: Co-researcher 11: The only thing that feels forced to me is to put competitiveness between me and other people, and against myself on the same axis. I do not know. Principal investigator: Would you say that these are two different things? Co-researcher 11: Yeah, it’s a very different feeling. [...] Co-researcher 10: Exactly! There’s an element of jealousy when it’s competitiveness with other people. Co-researcher 11: Yes! Yes! Co-researcher 10: There’s this element that you want to be better than other people Co-researcher 9: Yeah, competitiveness with me is something that I never experienced as negative. It was more encouraging. So, yeah, I would call it ambition. (validation workshop) It is important that each individual is comfortable with the names, and any concerns regarding the vocabulary should be addressed. The final step in the process of validation is for co-researchers to once more be exposed to the experience under investigation, either by provoking it or by consulting their journals. In this step, the co-researchers must be able to discuss their experience with one another with our vocabulary (as per participatory sense-making, the interaction must not break down). Being able to organically use the vocabulary constitutes the most important criterion for an intersubjectively accessible vocabulary on experience. In the study, this was achieved during the third validation workshop, where co-researchers were able to discuss their experience of solving a visual-span task in great detail using the jointly agreed upon vocabulary.

THE EPISTEMIC STATUS OF CONSENSUALLY VALIDATED PHENOMENAL DATA

In the previous section, I laid out the proposed method of consensually validating phenomenal data. Importantly, my approach hinges on a group of co-researchers observing their experience, comparing said experience to composite descriptions of similar experience, and then jointly constructing a vocabulary with which to describe them. This method of consensual validation operates under the constructivist epistemology. Constructivism claims that knowledge is not uncovered from an observer-independent world, but rather that it is constructed by the researchers[62,63]. In relation to consensual validation, this means that we as co-researchers do not agree on what objectively exists in our experience, but rather that we collectively create a contingent body of knowledge about our experience. It is not knowledge about experience as is, but experiential reports as constructed in a given study[34]. This epistemology can be specified further: the proposed method of consensual validation follows constructivist epistemology augmented by process-oriented ontology[64], developed by Alfred North Whitehead. In Process and Reality, Whitehead[65; p.67] writes: Actual entities atomize the extensive continuum. This continuum is in itself merely the potentiality for division; an actual entity effects this division. The objectification of the contemporary world merely expresses that world in terms of its potentiality for subdivision and in terms of the mutual perspectives which any such subdivision will bring into real effectiveness. In explanation, the processes that constitute the world can be divided in any number of ways. How we divide the world, however, is entirely open-ended. The potentiality of the world may therefore contain contradictions, but once we make it concrete and discrete, the world is divided into an internally coherent system. In Whitehead’s process-oriented philosophy, entities refer to parts of the world that are spatially and temporally located. However, as will be seen below, this framework has been productively used to analyze experiential worlds as well[64]. Potentiality for division could consistently be observed in the visual-span study as the frameworks of experiential categories that were constructed did not apply in their entirety for all of the co-researchers. What applied to an individual co-researchers was some subset of this framework. Consider the following report on the validation session: We found multiple common languages. Or rather, I feel that we reached two or three common ways of experiencing the task, and mostly each person could identify themselves with (at least) one way. (Co-researcher 14, feedback on the workshops) The result of this approach is not the construction of the language for describing experience, but the construction of a language. In the following quote from a co-researcher, we can see that this language is approximately precise, which means that it is both adequate for the description of our experience, while it simultaneously remains possible to articulate it in a slightly different way: Sometimes I felt just a tiny little bit forced to fit my experiences into given categories and dimensions because I thought it would reduce them a bit too much. On the other hand, they did fit into the suggested categories overall, and I imagine that some sort of reduction might be necessary to find common patterns or reach common grounds. It’s a bit as if you would measure many people’s heights and some would be 1,745 m and some would be 1,748 m and you put them both into the category of being 1,74 m. It’s some sort of reduction of data, but I do not think that these 0,003 m make that much of a difference in our world. (Co-researcher 11, feedback on the workshops)

Figure 4. Schematic depiction of fitting our vocabulary to experience.

Figure 5. Mary’s experience before (left) and after (right) consensual validation.

CONCLUSION

I have offered a method of consensually validating phenomenal data. I constructed the method based on similarities between phenomenology and other descriptive sciences, exemplified by 18th century natural history. While phenomenology and natural history both constitute descriptive sciences, whereby their task is to observe, describe, and classify phenomena under investigation, there are some major differences between them. The foremost difference between the two disciplines is that natural history relies on visual epistemology and exegesis of texts. Neither of these approaches is tenable within phenomenology. An important commonality between the disciplines, however, is the importance of intuitive, expert observation. As natural history divides the ways of observing the world into the spectacular and the scientific view, so does phenomenology divide its observation into the natural attitude and bracketing thereof. While these two reductions do not map one onto the other, they still attest to the intuitive way knowledge is given to observers. Based on this kinship, I argued that much like the nominalization of the visual world led to an intersubjectively observable body of scientific knowledge in natural history, nominalization of experiential world may lead to intersubjectively accessible phenomenal data. I suggested that the nominalization of the experiential world should be done by an autonomous group of co-researchers, rather than being imposed by the principal investigator and checked against the experience of the co-researchers. I derived the importance of autonomous construction of vocabulary both from discussions on the symbol grounding problem, and participatory sense-making. To implement this solution, I iteratively modified the prismatic approach, a method of gathering data on social dynamics. Based on reports on experience of consensual validation, I modified the method. The final iteration of my approach consists of a group of co-researchers being exposed to composite descriptions of experience. They are invited to once again observe their experience, comparing it to the composite descriptions. They are asked to attempt to falsify the composite descriptions. Finally, co-researchers construct a possible vocabulary for description of experience under investigation. This vocabulary is contingent on a particular research setting, but nonetheless offers a way of operationalizing aspects of experience.

REMARKS

1 In line with empirical phenomenology, we use the term co-researcher instead of the traditional term participant[4468,69].

2 The discussion about the subjective nature of data on experience is broad and beyond the scope of this article. For example, Hurlburt[48] claims that subjective reports gathered with the descriptive experience sampling technique are radically non-subjective, in the sense that there is a correct way of describing what an individual was experiencing in a given moment. Here, I wish to address what Froese and colleagues[70] would refer to as the theory-experience gap: the difference between what a researcher might theorize individuals will experience in a given context and what the individuals end up experiencing in that context. We may introduce a third type of experience: naive subjective reports, which are the beliefs individuals may have about their experience. Among these three types, only systematically acquired data on lived experience (the second type) may be considered to be intersubjectively accessible.

3 Nominalism is a philosophically burdened term. As Ian Hacking[70; p.81] writes in The Social Construction of What?, nominalists hold that “[t]he world is so autonomous, so much to itself, that it does not even have what we call structure in itself. We make our puny representations of this world, but all the structure of which we can conceive lies within our representations.” Hacking writes on: “The nominalist retorts that we have a good deal to do with organizing what we call a fact. The world of nature does not just come with a totality of facts: rather it is we who organize the world into facts[70; p.174].” In other words, nominalism does not claim that by naming the world we “cut it at the joint,” but rather that we impose structure upon it. As will be seen in section 5, we are sympathetic to this view, i.e., when we name experience, we do not name it according to the underlying structure of experience. Rather, by naming experience, we impose structure upon it.

4 I use the term phenomenology in a somewhat monolithic sense, disregarding the many different approaches within phenomenology, such as empirical phenomenology[3, 48, 69, 72], philosophical phenomenology[22, 29, 35] and phenomenological psychopathology[73] to name just a few. While these different schools of thought have different epistemological commitments, I believe that our method may be useful in bringing descriptions of experience to the fore of a community of researchers, regardless of whether the original phenomenal data was obtained through philosophical intuition or various second-person methods.

5 I thank Clémence Compain for help with the French translation.

6 I do not wish to suggest that this one-directional nature of qualitative research has never been addressed. Indeed, one of the cornerstones of the constructivist grounded theory approach to qualitative analysis is to account the ways in which the researcher constructs rather than discovers knowledge[35, 73, 74]

7 A principled criticism of this explication of phenomena under investigation is that we may induce the experience through suggestion effect. Pete Lush and colleagues[67] have observed that in the general population the rate of suggestibility of individuals is normally distributed; i.e., among highly suggestible individuals, instructions in a research setting may indeed cause them to have the experience presupposed by the research design. While this effect is problematic for empirical research in mind sciences for a number of reasons, we believe that as long as it indeed provokes an experience in our co-researchers rather than merely lead them to say they experienced something, this does not reduce the epistemic value of our phenomenal data.