#1234 Uljana Feest - Operationism in Psychology: An Epistemology of Exploration

00:00 Ricardo Lopes: Hello, everyone. Welcome to a new episode of the Dissenter. I'm your host, as always, Ricardo Lops, and today I'm joined by Doctor Juliana Feist. She is a professor at the Institute of Philosophy at the University of Hanover, and today we're talking about her book, Operationism in Psychology and Epistemology of Exploration. So, Doctor Feist, welcome to the show. It's a pleasure to everyone.

00:26 Uljana Feest: Thank you for having me.

00:28 Ricardo Lopes: So what motivated you to write such a book? What are its premises?

00:35 Uljana Feest: Um, WHAT motivated me, this book actually took me a really long time to write. Uh, SO I guess it's good to be asked what actually started it, um, because then throughout working on the book, I think more and more different questions and motivations kind of appeared. But the earliest motivation really was that I had a background in psychology before I got into history and philosophy of science. So I went to graduate school in history and philosophy of Science, but I had already, I had a prior degree in psychology. And, uh, and had always had an interest in philosophy, uh, especially philosophy of language at the time. And I was curious to see that in philosophy of language, whenever people talked about operational definitions, uh, and you noticed that I have the term operationism in the title of my book. Uh, SO when people in philosophy talked about operational definitions, it seemed to be kind of understood that That that's a bad thing, and that, you know, that's a very outmoded way of talking about the meaning of concepts. And coming out of psychology, I felt that that wasn't quite capturing what psychologists have in mind when they say that they're operationally defining their concepts. So, really, I think at the bottom of the start of this, this project really was this kind of interest in what is it that operational definitions do in psychology, what does operationism or operationalism, uh, actually, the, the two terms mean the same as far as I'm concerned. So, so what, what, um, what function does this position have in actual research?

02:11 Ricardo Lopes: Mhm. Uh, AND what is the philosophy of experimentation, which is basically the area you're dealing with in your book?

02:22 Uljana Feest: Right, great. Uh, THE philosophy experimentation, um, is something I got interested in as I was thinking about this, uh, this project. Because increasing, so, I guess the reason why I thought that there's more to operational definitions than what is commonly assumed in philosophy of language was because I saw psychologists when they're thinking about how to design their experiments, they're thinking about, well, how should I operationally define my concepts. That's kind of the first choice you have to make. Um, AND we'll get back to in what sense they mean operationally defined. But, um, So, so then that drew my attention to the fact that there is an area of philosophy of science, um, called philosophy of experimentation, which, uh, in its kind of most recent installment, um, Started in the 1980s, I would say. So Ian Hacking is a big name, but other names in the 1980s are Alan Franklin, Deborah Mayo. Uh, Friedrich Steinle and, and, and kind of more on the side of history of science. So it kind of starts in both philosophy of science and history of science. And it questions some, um, some of the basic assumptions of traditional philosophy of science, which typically starts with theories, right? So, so, and that's like in, in Ian Hacking's book, Representing and intervening, he has that very nicely in the book. Traditional, I mean, obviously, this is all very you know, uh, idealized and doesn't probably do justice to every individual working in philosophy of science, but he has this, this, this general line of saying, in philosophy of science, people talk about theories as if they were already there, uh, kind of coming out of nowhere. And then the question is, how do we justify them by means of evidence, right? Uh, AND then in, in that picture, experiments only have one role to play, namely to Uh, to pro produce data in support of, uh, of a theory. And one thing that hacking notes at the time is they never talk about, um, other functions of experiments that other functions of experiments might play other than, um, test theories, and they also don't talk about how data are produced. And so that's really the, the bit that really kind of grabbed my attention because I think that is crucial, uh, and probably in all of experimental science, but especially in, in the one that I'm familiar with, kind of thinking about what kind of question am I pursuing and how do I go about producing data that will speak to that question. So, uh, so I guess I kind of did, uh, you were asking what is philosophy of experimentation, um, Yeah, and I guess I answered the follow-up question already, which is, what is it that is that I find interesting about it?

05:09 Ricardo Lopes: Mhm. So, in the book, you go through an example to introduce the kinds of questions you're interested in, in, uh, tell us about the example of the study of implicit memory and the kinds of questions it raises.

05:27 Uljana Feest: So one of the things that I highlight in the book is that the um that the epistemic situation of researchers is characterized by um By kind of lack kind of conceptual openness and um with respect, and with respect to the, what the object that they're studying, it is characterized by uh what I say, what I call epistemic blurriness. So they, they maybe start out by thinking, here's, here's something I'm interested in, but I'm not really quite sure, you know, what it is, and that's why I study it. Uh, AND you were just mentioning implicit memory, which is one of the examples I use, um, and so So then that kind of raises the very fundamental question of, well, if, if this object that I'm interested in, well, first of all, why do I think that there is an object, right? Um, WHAT kind of, what gives rise to this idea that there's an object that I would, that, that I, that I ought to be interested in? Uh, AND secondly, given that, given this kind of blurriness of the object, how do I get an empirical handle on, um, on how to study it? Uh, NOW remind me what was your question. Oh yeah, so the example was like the, the example I, so I use implicit memory kind of to illustrate something about both of these questions. Um, SO implicit memory is a concept that emerges in philosophy in, in psychology, excuse me, in the early 1970s, and, um. And it gets introduced kind of alongside a new kind of memory test. Uh, SO people up to then kind of the standard way of testing for memory is, you know, you have people memorize something and then you present them with a test, and that can be either, uh, recall or recognition. So you ask people, uh, do you recall, you know, seeing this item before? Or, or you say, uh, do you, or do you recall what the items that we presented you, that's recall and recognition is here I'm presenting you with a list, which of those have you seen before in, in, in the previous study period. So those are what we would now call explicit tests because they ask subjects to explicitly report on what they remember. And then this new type, type of test became uh Or was introduced, um, Namely priming tests, and priming tests have a different, uh have they use a different um strategy for getting at people's memories, namely by um Presenting them with so-called primes. So, I might in the study period have presented you with a bunch of words, and then in the, in the test period, I might um Uh, oh no, sorry, sorry. I might do a study for period that give you, yeah, give you some, some things to remember in the, and then before testing whether you remember them, I prime you by, for example, giving you words that are semantically related to the ones that I want you to remember. So, you might not remember, um, The previous ones that you studied. Like, so on an explicit test, you might not remember what, what, that you, whatever, uh Were presented with, uh, that, that, that you previously learned the word um Butter. Um, BUT, but if I give you the word bread as a, as a prime, um, and ask you to, to freely associate something that comes to your mind, you will, you will produce the word butter. And so, then that is taken as an indicator that something of the previous study period, um, kind of stuck. Um, uh, AND, and so, and even if you don't remember that you learned this word, um, some, there's The prime triggers, the, the recollection of it, and because this is not, uh, and this kind of result is um dissociated from the results of experimental tests, meaning that people who uh can, can do well on the one while not doing well on the other with the same kinds of items. So this kind of functional dissociation gave rise to the idea that there might be two types of memory. Uh, I'm gonna say memory systems, but really it's often used much more vaguely that there are two types of memory. Um, AND with that idea that, oh, we're testing, do we have a functional dissociation between the results of two different types of memory tests. That gives rise to the idea that we have a novel object of research. We have an object of research that is separate. It's a type of memory, but it's different from the type of memory from what we previously thought memory was. And so for the time being, we're just gonna try and investigate this further and we're gonna use priming tests as a way to access this um supposed or this, um, yeah, this, this type of memory that we now think might exist. Mhm.

10:49 Ricardo Lopes: So, how the, how are conceptual work and experimental work intertwined?

10:57 Uljana Feest: Um, Yeah, good question. And I think it directly follows up on, on what I was just talking about. My, my, um, so this is basically a very, very basic premise of the entire book is that conceptual and experimental work is intertwined, or rather that if you do experimental work in psychology, it's intertwined with conceptual work. Um, AND I, and I consciously kind of contrast conceptual and theoretical because I don't want to start with the idea that I already have a theory. And now I'm just testing it, but rather, I have a vague conception of an object, and that vague conception of an object, basically that, that concept is what gets further articulated. In the course of doing experimental work. And so experimental and conceptual work are intertwined in the following way. I need at least a rough concept to do an experiment on the object in the first place, but ideally, the results of the experiments I do are going to inform the concept. Um, AND, um, and thus, you know, improve the concept. So, in case, um, listeners are familiar with Hasso Chang's work on, on iterative, the iterative process of scientific investigation, this is a similar idea that we kind of have a back and forth between kind of the concept informing the types of experiments we produce, we, we set up, and the kinds of data we produce, which in turn, um, Mm And, well, if things go well, inform the concept or kind of feedback into the concept.

12:34 Ricardo Lopes: Mhm. So, uh, tell us then about how experiments are designed not only to test theories but also to perform other epistemic functions as well, such as detecting entities or calibrating instruments.

12:54 Uljana Feest: So I think what, what you're referring to here is, uh, is the Well, some of the, the crucial insights of early work in philosophy experimentation were, were scholars like Hacking or, or Steinle, um, said this is a very idealized, abstracted, uh what, you know, the, the notion that experiments only test theories is very idealized and doesn't really do justice to kind of the work that's being done on the ground. Um, And then they, they Draw attention, especially hacking and also Alan Franklin, draw attention to the ways in which experiments can be used to, uh, for example, performing multiple experiments can, can be a way of, um, arguing for the reality of a given entity. Um, HACKING has, um, has different kinds of, of arguments, but, but one is about, um, if you, if you, if you're detecting. Uh, SORRY, I was gonna say electron, but if you're, if you're detecting a particular entity in a microscope, um, How do you know that what you're detecting is, is, um, is really there, as opposed to being an, an artifact of the, of the instrument. So, so then his, one of the, one of the kind of um ideas he introduced into the literature is, well, if you can detect a similar thing by means of multiple different instruments, you are kind of simultaneously uh providing Um, Providing more and more evidence, independent evidence of the reality of that thing that, that you're seeing, but you're also kind of cross validating the instrument. So if multiple instruments validate the same, uh uh uh detect the same thing, that also strengthens, uh, not only our belief in the reality of the thing that is being detected, but also in the reliability of the, of the instruments. So that's kind of one way in, in which Um, experiments can play a role other than theory testing. Um, AND for, again, for, for hacking, it has this kind of dual function. He's, he's an entity realist, so he wants to say, rather than simply testing whether theories are true or maybe more importantly than testing whether theories are true, we try to test whether particular entities really exist. So that's kind of um in the tradition um that, that was started by people like Hack in the 1980s. In my own work, I kind of want to go beyond that, or at least I feel that in psychology, that's not quite the right question to ask because we're not dealing with, you know, tiny entities that we see under microscopes. We're dealing with very complex. Objects that are constituted of multiple phenomena. And so, um, so in my work and maybe we can talk about this in, in, in more detail in a moment, um, I'm really more interested in how our experiments used to explore objects of psychological research.

16:12 Ricardo Lopes: Right. So tell us then about those objects. What are the objects of research in experimental psychology? Are they phenomena? Are they mechanisms or something else?

16:27 Uljana Feest: So they can be all of the above, but the way that I, that I think of objects of res of psychological research specifically. Um, IS, is that there are kind of clusters of phenomena, but maybe let me take a step back first and, uh, and just talk about the notion of an object of research because I mean it's maybe intuitively clear an object of research is the object that we're interested in, but I also, I use it in a somewhat um. A slightly different sense, or at least I'd like to capture the, the, the fact that when you're interested in a particular object, uh, you don't know whether it's going to be a legitimate. Uh, OBJECT in the long run, like you're, like, like, let's take implicit memory, you know, you have these results on priming effects as associated from effects on, on explicit memory. You're kind of hypothetically thinking there, there might be something there that's worth investigating. I'm gonna call it implicit memory. Now, that's an object of research, but it might not end up being a stable object. It might not end up Uh, be, you know, playing, um, an important theoretical role, you know, in the, in the future of psychology, but at the point in time when, when uh, when the psychologist comes up with this concept, they don't know this, right? So an object of research has inherently, it's, it's on the one, it's the object that we're directing our attention to, but it also inherently has that uncertainty about. Uh, WE don't know a lot about it, and we don't even know whether it will turn out to be a kind of a, a stable kind of psychological kind that in the long run we're gonna believe to be, you know, real or important. So, so an object of research in my mind has, uh, so by saying object of research, I'm not committing to a realism about that object. Uh, HOWEVER, I am saying that psychologists, in order to do research on, on an object of, on such an object, they have to have a, a somewhat realistic attitude to, toward it, you know, you can't, you can't explore something unless you think that there's something to explore. So. OK, so that's kind of about the, the general notion of an object of research, um, of, of which I say that it's quote unquote, epistemically blurry, which is really the summary of everything I just said, you know, there's something there, and I'm not entirely sure what it is because it's blurry, uh, and it might turn out, turn out not to be as interesting as I thought, but, but precisely because of the blurriness, um, I'm gonna investigate it because it seems like an interesting object right now. Now, what are objects of research in psychology and how are they different from phenomena? Now, of course, it's, it's perfectly, you, you could say, well, it's an, it's, it's an, you know, this is an interesting phenomenon that we have here, you know, I mean, have these priming effects, um, that's an interesting phenomenon. Um, I'm gonna call the phenomenon uh implicit memory. Um. In my book, I try to, or I, I draw a kind of terminological distinction here, and between an object of research, which is the object that we're interested, like, which is a kind of a more, a broader category. So, when I say I'm interested in, in this priming effect, that That I would say is a phenomenon, uh, in the sense of the, the philosophers, uh, Jim Bogan and Jim Woodward have a paper, Saving the Phenomena, uh, from 1988. And in that paper, they say phenomena are, or in that paper are one of the fallout papers, they say phenomena are stable regularities in the world. Um, SO I would say kind of following that definition, a phenomenon, uh, like a priming effect is a phenomenon insofar as it's a stable regularity in the world which one can debate, but let's just assume that it is a stable regularity, that's a phenomenon. Um, NOW, my claim is that objects of psychological research are comprised of multiple phenomena. So, you might, you might get interested in implicit memory because of the priming effect, which you take to be indicative of this object that you are now interested in, but the two are not the same. I think the, the, the notion of an object of research which you then give a name. Um, YOU know, implicit memory, it's closely connected with the priming effect, but it's not the same. So that, that's how I draw the distinction. Um, YOU were also asking about mechanisms, um, so mechanisms, for example, cognitive mechanisms or neural mechanisms. I would say are, are, are phenomenon because they're repeatable regularities. Uh, AND so I would argue that an object of psychological research. Um WHICH consists of multiple phenomena, some of those phenomena can be mechanisms, right? But not all of them have to be mechanisms. They can also be behavioral regularities or they can be certain regularities in, in kind of types of experiences people have. Um, OR, you know, kinds of, uh, kinds of, um, cognitive processes that, that, that are associated with an object. So, so yeah, the, the short answer to your question is the, the, the term object of research, uh, in, as in general means something that we're interested in. In the case of psychology, I claim, or at least the, the types that I'm interested in, in my work are, uh, are objects that are described by terms like implicit memory, working memory, perception, personality, those are the objects, but obviously when we start looking at them, we're gonna try to give a much more fine-grained account, uh, which shows how this object is constituted of multiple phenomena.

22:24 Ricardo Lopes: So what is operationism then and what, what are the origins of the term?

22:32 Uljana Feest: Hm, what is operationalism? OK, right. Uh, I started off by saying that was the motivation for writing this book that I was interested in what operationism is in psychology as opposed to the way in which philosophers talk about it. So you can already see that the question is kind of a loaded question because depending on who you ask, when you ask people might give you different answers. Um, NOW, The term, and maybe this is a good moment to also go back to the distinction between the word operationism and the word operationalism, um, in psychology, especially like in the 1940s at least, many psychologists use the term operationism, but other people also, um, use the term operationalism, and I, um, and especially the latter term is typically. Attributed to the physicist Percy Bridgman, who wrote a book in 1927, he was at Harvard. He wrote a book in 1927 called The Logic of Modern Physics. And in it, he has, amongst other things, the very nice sentence that states something like, the meaning, uh, but no, by a concept, we mean nothing but a set of operations. So that formulation sounds as if he was trying to say that the meaning of a concept is um Completely um determined or there's nothing to the meaning of the concept, but, um, but some, some operations, some measurement operations, that's how it's usually taken, that's what it's usually taken to mean. And of course, phrase like that, it sounds a lot like another very popular position that was like in, in philosophy, uh, in philosophy at the same time or around the same time, namely, um, verificationism. That, that's the, the, the idea that the meaning of a sentence uh consists in the, in the, in, in, in the method of verification or and that the sentence is only meaningful if there is a method of verification, uh, and that a term is only meaningful insofar as it figures in a verifiable, in, in a sentence that is testable in principle. So, um, so given the similarities and also given the fact that, you know, right around that time, um, European immigrants, uh, were, you know, coming to the US, um, from the German speaking world, people like, uh, Karnav and Reichenbach, um, there was an, um, and also actually at the time there was a lot of interaction between psychologists and philosophers which I find really fascinating in the, in the history of, of philosophy of science. Um, IT was natural to assume that these two positions are very closely related. And so in a certain thing, in a certain sense, I think they are, but in a certain sense, I also think they really aren't because as a practicing scientist, you're not primarily interested in the semantics of scientific terms, you're interested in um what, you know, maybe what you're entitled to say in an experimental situation or you're interested in like, how can I use this term to conduct my research? How does this term describe the objects I'm interested in? Um, AND so, Percy Bridgman definitely was trying to express. Um, AND a note of caution. So, so, his, the general gist of where he's coming from is, let's try to be careful to not overextend our concepts. When we have a concept like mass or gravity or whatever, um, That comes out of our particular physical tradition, like classical physics. Um, WE should not assume that it holds in the same way or that it's applicable in the same way in, in other realms, you know, much more macro realms or much or, or, or micro realms. We have to be very cautious about this, and of course, in, in this caution, he was informed by the recent relativistic revolution. Um, SO he, he was very kind of struck by, you know, you really can't over, you really have to kind of, uh, maybe contextualize the use of your terms, uh, a little bit. So that at least is, is my reading of what Bridgman really intended. Um, BUT that's, you know, my book is not about Bridgman, um, but there, there's, um, but, but psychologists at the time were picking up on the notion of operational definition, um. Partly from Bridgeman, um, but I think also using it in a slightly different way. And so in my book, I have in, in chapter Two, I think that's the first substantive chapter. Um, I talk about a number of early experimental psychologists who start saying, using the terminology of operational definition. And, um, and really the kind of the basic gist of it. Uh, IS that they're, that they're saying, if we want to get a grip on our subject matter, we need to have a method of measurement that we're going to treat as um, Providing us with data about this object. Mhm. Now you were asking, sorry, you were asking about the, the history of the term operational definition, the history

28:07 Ricardo Lopes: of operationism, yes,

28:09 Uljana Feest: right, right, so. Yeah, so maybe I'll just stop right here and, and give you a chance to ask another question.

28:16 Ricardo Lopes: Yeah, I mean, I was going to ask you what is an operational definition and what is the connection between operational definitions and experiments.

28:30 Uljana Feest: Right. So, um, yeah, I guess I've already started talking a little bit about operational definitions. They're typically, you know, associated with means of measurement, you know, you have a method of measurement, kind of, there's this very famous quote by a psychologist Edwin Boring, who actually, by the way, said this in early in 1923, i.e., before Bridgman. Um, HE says, um, by intelligence, we're just, we just mean whatever the tests measures. Um, AND this is another reason why, um, Or this let's say this expresses, this is a very provocative statement that expresses a um something that seems intuitively wrong, right? Uh, SO, Why if, if by, if by intelligence you simply mean what you measure, well then, what you measure by a particular test, don't you first need kind of um independent evidence that the test really measures this thing, right? And so I think that's where a lot of people's, people are uncomfortable with operationalism. In part, because they think it expresses some very weird, um, this kind of weird position where, where somehow you've already defined your object in terms of the method of measurement, but then how did you, you know, how do you know that it's a good measure, method of measurement for whatever you're trying to measure? Um, I think we should keep in mind that Boring is really talking, uh, in a, in a very applied context. He's just saying, look, we're using these, these, these intelligence tests. They, they serve a purpose, they have a specific, maybe predictive value, whether or not they do is a, is a separate question. Uh, LET'S just not, let's just stop having this kind of philosophical discussion of whether they're really measuring intelligence. Let, let's just, they're, they're just a tool. Uh, AND I think in certain contexts, that's a reasonable position to take. I think in a, in a more kind of basic science context, it's not a reasonable position to take because we, it, it seems intuitively, no, we, we do want to think about what intelligence really is and What whether we have good reasons to and and whether and and what are its features, uh, and, and how do we go about investigating it um and so that's where I think operational definitions come in in the way in which I use the term and the way in which I use the term is informed by uh, an analysis of the writings of basically in the book, in my book, um, 3. Important figures in psychology in the 1930s and 19 well actually earlier 1920s to 1950s, um, but I'm focusing on writings from the 1930s primarily where, um, and these 33 figures are SS Stevens, psychophysicist, um, Um Um, sorry, I'm blanking on the first name. Uh, Edward Tolman and, uh, and Clark Hull. All three of them, and, uh, Tolman and Hull are known as, nowadays, they'd be referred to as neo-behaviorists. So what I'm saying about all three of them. Is that even though they use terms like operationism and operational definitions, none of them are saying um the meaning of, of psychological terms is exhausted by, by some means of measurement. Rather, what they're all, what they all seem to be saying is, if we want to study things like conscious experience in the, in the case of SS Stevens, or um Demands and cognitions, uh, in, in the, in the case of um Tolman, for example, then we need to have experimental methods or we need to have a way to tie the, the, the use of these terms very clearly to specific empirical um circumstances because we don't, we don't want to lose kind of, um, we don't, we, we don't want our Psychological theorizing to become too detached from, from the kind of empirical basis. Uh, AND that also becomes very clear in the, in the third person I just mentioned, Clark Hull, who basically, I mean, his aim was to produce, to construct a psychological theory from the bottom up. And, and so his idea is, first, we, we, um, We start with our basic concepts that we try to kind of tie to particular experimental paradigms. Uh, AND so then we, we kind of, um, And, and those paradigms involve particular operational definitions. They involve, when I say, so in the case of, of Stevens, for example, when I say experience, I mean discriminative behavior. Uh, I mean, um, I mean, you know, people respond to sensory stimulation in, in a way, in a specific way. Now, this might at first blush sound as if Stevens was saying experience, the term experience means nothing but discriminatory behavior. But importantly, he actually says, for the purpose of science. So, so he's, I think when you actually read his, his work on psychophysics, it's very clear that he thinks experience has a conscious dimension. There's to experience is to have a conscious to have a conscious, uh, what we might say, um, To be in a conscious, uh, state, phenomenal conscious state. Um, BUT How do you access somebody's phenomenal conscious state by uh presenting them with stimuli and then in some way, um, eliciting responses to that, to, to, uh to the stimulation and those responses are then going to be data that tell us something about the experience of the subject. So, so here this is, this is, I think is a nice example of how it sounds as if he were kind of reductively defining, reductively defining experience as nothing but a particular kind of behavior, but on my construal, he's actually operationally defining, um, he's merely saying this is how I'm gonna measure it. Um, HERE'S my object of research, conscious experience, and I'm, I'm measuring it, uh, by, by setting up experiments that, that, um, treat data in response to sensory stimulation as evidence for whatever I wanna say about this, uh this object of research, i.e. experience.

35:32 Ricardo Lopes: So in the book you suggest that the central unit of analysis for our epistemological considerations of psychological research should be concepts and not theories. Could you explain that?

35:47 Uljana Feest: Well, when you think about, I mean, so maybe a little bit of, again, my, my autobiographical background is, I don't really know that, I mean, it's been, there's a little bit of discussion in psychology recently, but it's not that clear what a theory is in, in psychology, uh, and how many theories there are, um, and like what it takes to kind of Focus on a theory. To me, it just seemed much more obvious that we're focusing on things, objects, the kinds of things we're interested in. So if we're interested in a particular objects, so as we've kind of been, I've been indicating in my book, one of the objects that then kind of takes up, like one of my case studies is about memory, and I talk about implicit memory and about working memory. So if my object is memory, um, Then I, I argued that the I mean, yes, maybe in the long run, we would like a theory of memory, but my, my main claim here is that in order to even come up with a theory, we have to explore the object and exploring the object really means working with concepts, uh, with the concepts that we have. We don't necessarily have a big grand theory of memory, but we do have a kind of folk psychological concept of memory that, um, Involves, for example, memory is something that, you know, uh, is the result of learning, it's, it, it, it involves the idea that something is being stored and that something can be retrieved. So I think we kind of go, our researchers go into their experiment with the um With this, these kind of very basic concepts, which inform the way they approach their subject matter. And now of course, we could, we can question whether these concepts, whether that's a good way to go, right? I mean, in philosophy of mind, there's a lot of discussion about folk psychology and whether we should even use our folk psychological concepts to do, to do um empirical research. Um, I have opinions about this. One opinion is that actually the, in, in, when this is discussed, In philosophy of psychology. Often the assumption is that um folk psychological concepts are explanatory concepts. I.E. their concepts that is that people appeal to, to explain behavior. So in my book, I take a different, I have a different take on it. I think they're primarily taxonomic concepts. We, we take kind of memory to be, we classify psychological objects in certain ways. And a lot of research, I claim, is not primarily. GEARED at Explaining behavior, but at at getting a fuller More, a fuller, more nuanced understanding of a given kind of object. Um And So, so, yeah, so back to your question, that's part of why, why I say start with the object, um, about which we want to build a theory, maybe, ultimately, rather than with a theory when we don't even know what the object is. So, it's kind of, there, I'm also really following my historical actors because I just found them very persuasive, that Somebody like Hull, I mean, he's kind of been forgotten, and, I mean, he's, he's not that, uh, he's, he's, he was hugely important, you know, in the 1940s. Um, AND nowadays, he's more, you know, I don't think he's, he's a very central theoretical figure, but I think the kind of the basic insight, um, is still worth that he, that he had is still worth considering, which is, um, start, start with, with You know, the units that we're interested in. In his case, he was interested in habits. How do we conceptualize habits? How do we construct experiments about habits? Um, And that means starting with concepts, concepts of the things that we're interested in.

40:11 Ricardo Lopes: Mhm. So what is discovery and what is justification, and what is the distinction between them?

40:23 Uljana Feest: So in philosophy of science. That's often, you know, a distinction people draw. And again, I think I'm just, um, I, I, I, I assume you're inviting me to say a little bit about um how this distinction is often appealed to in traditional philosophy. So there's this kind of traditional idea that, you know, philosophers are interested in normative questions, philosophers are interested in, like, what are, you know, what are good rational reasons to Uh, to believe something, given the evidence, you know, how well are our theories confirmed, etc. AND those kinds of questions. ARE asked and take place in the context of this uh of justification, right? Because it's all about justifying some idea that we have, that we already have. Um, AND by contrast, traditionally people have, have argued that just that, I'm sorry that I'm getting all the terms confused here, that discovery is whatever way in which People come up with their ideas. Uh, THEY don't have to be logically justified. They can be, you know, people have ideas from all kinds of places and then, in random ways, you know, interesting ideas might come up, come to their minds. Um, AND because of that, we can maybe describe how the ideas can come to people's minds, but it's not properly the subject matter of philosophy of science because there is no logic to this. There is no kind of, there are no rational rules of how you discover. So that's kind of the traditional picture. Now, that picture has actually been kind of under attack for like several decades, um, at least, at least since the work of Thomas Kuhn. Um, But I think it's still very pervasive uh on some level, like, how do you talk about discovery? Now, in my book, I actually kind of wanna get away from that terminology, because I, I, I think it's artificial. Um, I think it is, of course, entirely possible that sometimes people randomly have some brilliant ideas, but I also think there's a lot of um Um, I wouldn't say, yeah, there, there, there's, um, People have good reasons for coming up with particular ideas. They're theoretically motivated. Um, THEY follow from previous, um, experiments. They follow from, from previous theories, they follow from, um, kind of putting various considerations together in a, in a creative novel way. Now, um, So I, so I think Like I, like I just said, the, the, the distinction between discovery and justification certainly doesn't, if it describes anything very well, it certainly doesn't describe kind of a chronological step where, you know, first you have discovery and then you have justification. Um, BUT because there's so much confusion about what these terms really mean and what they, what they're applicable to, I actually prefer the term exploration. Uh, AS you, um, may have noticed, my book has the subtitle in Epistemology of Exploration. And that also gets us back maybe to the, to your previous question about um or your multiple questions, uh, concepts and objects, because what I, what I try to focus on or to the, the way I try to ask the question about discovery, uh, is not as something that somehow happens as distinct from, from, from justification, but rather as in terms of We are trying to explore. Scientists explore a particular domain, and in order to explore this domain, they have to make certain assumptions about the kinds of objects that this domain is populated by. So, the psychological domain, right? Um, INTUITIVELY contains things like, you know, cognition, emotion, perception, uh, Desires. Um, BUT, but also things like maybe, you know, mental disorder or personality or develop, you know, the child development. Uh, SO it, it contains a, a large array of, of things, and in my mind to do psychological research is to explore that domain in a, uh, systematic fashion. And that's just, it's kind of by, by focusing also on the systematic on, on the, on the way in which exploration is systematic, I also kind of want to counteract the narrative according to which discovery is not systematic, um, so, so, so this kind of systematic process of exploration, I think kind of really blends together aspects of what was previously referred to as discovery and as justification.

45:19 Ricardo Lopes: Mhm. Uh, AND what is the interplay between theory and evidence in the investigative process?

45:29 Uljana Feest: So, as, as I'm, I've said at the beginning, I think of, and one reason why I'm really interested in the relationship between operation or operational definitions or operationalizations on the one hand, an experiment is What happens when you're operationally defining your concepts? Is you're, you're thinking about how to Measure your object in a way that produces or to pro how to produce data in a way that are, that, that are informative of the object you're interested in. Um. And so It's not the case. So, so I think the kind of the implication of that is that researchers already put a lot of Mental energy and thought into, um, into their research design before they even have data, um, uh data which then are gonna function as evidence. And this is important because I think it counteracts a particular way of talking about evidence. That always seems to assume like that the evidence is just out there. So what I wanna really highlight in my work is that evidence is produced. It's produced with specific conceptual um presuppositions in mind that have to be in place. And so even, so this is, of course, especially obvious when you, you know, in experiments because you're literally producing the data. I mean, you're, you know, Uh, you have your subjects, you present them with stimuli, you record their behavior, and that, that behavior is then, you know, your data, um, quantified in the appropriate manner or whatever. Um, SO, so clearly that, you know, you're not just out there observing, you are. Observing the data that you produced with a particular question in mind and with particular conceptual presuppositions in mind. So this also kind of picks up on, on one of your previous questions about the kind of how is um concepts and, and experiment intertwined, right? I mean, it's intertwined in the sense that the concepts inform the, the, the experimental design which produces the data and so the data are in an important way kind of conceptually laden. But I'm not sure, like, yeah, sorry, can you remind me of your question?

48:01 Ricardo Lopes: Uh, YEAH, I mean, uh, I was asking you about the interplay between theory and evidence in the, yeah.

48:08 Uljana Feest: Yeah, and of course maybe that, that's also a good moment to kind of go back to the earlier question about why do I, why do I focus on concepts and not on theories, um, because, uh, so I, I see a close interplay between concepts, and, and data, but of course you could ask and maybe rightly so, what's the difference between, you know, theoretical and conceptual, right? Um, AND, and I think that's, that's a, you know, a very legitimate question. Um, AND it kind of, I think, goes back to I, I, I feel that theory is a very broad term, and I'm not always quite sure what it means, and by talking about concepts, I think I'm um uh. So, so there's a sense in which if I, if I say, you know, my concept of memory, you know, involves, you know, that something is learned and something is stored and something can be retrieved, then you could say, well, isn't that also a theory, right? Because there are certain, you know, connections, um, kind of maybe more or less law-like between, you know, various, um, Aspects um of, of, of this domain of research. And so, on a, on a loose understanding of theory, you could say that's also, that, that is, that is, of course, theoretical. Um, I think I've just been kind of, um, and so I think, yeah, in a way, I, I, I did take myself to be answering your question about the relationship between theory and evidence, even though I, I tend to talk more about concepts than about theories.

49:42 Ricardo Lopes: Right. And what are reasoning practices and which of their aspects are you interested in when trying to understand the investigative process?

49:54 Uljana Feest: Great. So reasoning practices. Yeah, I mean, I think I, I, I would say that I probably use the, the, the term in a very intuitive way. I mean, reasoning practices are whatever we engage in when we reason, we make, you know, we make connections between concepts, we make predictions based on things that we have learned in the past, um, we draw uh maybe more, more specifically in science, um. We come up with hypothesis to test. We, um, draw inferences, you know, statistical inferences, I think would also constitute types of reasoning practices. Um, AND as you may have gathered, I think that the reasoning, what the reasoning practice that I am primarily interested in is really the one of, um, the kinds of practices that go into experimental design. Because, um, because I think that's, that's where a lot of Decisions are being made, uh, about, I mean, it, it, it, it, they are informed by specific assumptions that we already have about the subject matter, um. And so a, a lot hangs on them. Um, AND I think, a lot of the time when we talk about, well, what do the data really show and then is, are the, you know, are the inferences really, um, really justified. It's, I think a lot of the time this is not about, it can be, but it's a lot of the time not just about did I apply the statistics correctly. It's about, do the data really, um, do I have good reason to assume that the data really show what I, what, what I, what I want them to show. And what I claimed them to show.

51:40 Ricardo Lopes: Right, and what would you say is the role that philosophy should play when it comes to scientific investigation?

51:51 Uljana Feest: I think that's a very, that's an interesting question, and, and I'm, I might, I, I suspect that some people might say philosophy, you know, Like, why should philosophy play a role in in psychological investigation, right? It's a different discipline. We reflect about science, scientists do their own thing. And to, to be honest, for the longest time, I mean, I, I have a, you know, basic training in psychology, but then when I went into philosophy, and for a long time, um, I was just interested in understanding psychology better. I did not necessarily take myself to aim at improving, you know, psychology, and, and I still don't. I mean, I, I, that would, you know, I, I I am interested in talking to psychologists, but I'm not sure that I would make a strong normative statement about what role philosophy should play. Um, HAVING said that, I do feel that there is a stronger interest. On the side of psychologists in philosophy, science than there was 10 or 20 years ago. So when I kind of left psychology, um, I don't remember kind of psychology, friends and colleagues being very receptive to the kinds of questions, to the kinds of more basic epistemological questions I was interested in. Um, AND I think that's changed a little bit in the last, you know, 10 or 20 years. And so now I do find psychologists interested in philosophy of science, and while I don't think that philosophy should play a role, I do hope that it can play a role. Um, AND that it's, um, like the kinds of, the kinds of things I think about, that those are things that could be fruitful for psychologists to think about as well. Mhm.

53:40 Ricardo Lopes: And what do you make of the replication crisis? I mean, in what ways would you have something to say about it through the kinds of topics you explore in your book?

53:52 Uljana Feest: Um, So the, yeah, the replication crisis, of course, that's, of course, I think I was indirectly already referring to that earlier. I mean, in response to your last question, like why is it that philosophy psychologists are more interested in, in philosophy now? It is in part because of the replication crisis because there Has been due to the fact, I mean, probably most listeners know this, um, many experimental results don't replicate, so it's been, you know, it's, it's impossible or they, people when, when redoing experiments or reanalyzing the data, either one of the two, you don't get the same results, and that has that has led to a lot of soul searching within the psychological community, um. And yeah, as I said, I think it's part of why in they've, they're kind of psychologists, you know, are, are taking a, a, a harder look at their own epistemological foundations and at the kind of philosophical work that's out there. Um, WHAT do I make of it? Well, when I first heard of it, I, I was just really um delighted. I shouldn't say that, that's, that's awful, you know, I shouldn't be delighted at something that is, you know, um a terrible problem, um, but no, let me qualify this and say, I was just, I was intrigued because I had been thinking about experiments. And, and so to me, it was clearly an issue that was interesting from a philosophy experiment, philosophy of experiment perspective. Um. If you think about psychological experiments, what reasons, you know, might there be to expect that replication is hard. Now, to be honest, like in philosophy experimentation, not just in psychology, but more generally, that's just kind of, that's just something. That's a standard piece of knowledge that it's really hard to do experiments, and it's really hard to get them right, and it's really hard, and because it's really hard to get them right, it's also really hard to replicate them. So that's been kind of a long standing discussion within, you know, STS and history and philosophy experimentation. And kind of coming from that, I, I kind of, kind of brought in, I mean, I've, I've written a, a, a few papers about, about, about that, about the replication crisis kind of from the perspective of philosophy experimentation. Um, AS you probably know, there, there is a whole Um, most of the literature on, on, on the replication crisis has focused more on the statistical side, um, of, uh, of psychological research practices, i.e., on, uh, on questions of, um, What might be statistical practices that have led to The inflation of seemingly um significant results, um, such that we now realize that a lot of these results actually, you know, weren't significant. Um, SO there's a lot of debate about that and I'm not a philosopher of statistics, so I just haven't really engaged in that part of the, the debate as much. Um, COMING, but I think the debate about the replication crisis has also really branched out in recent years and people are, so there's still this whole bit about um questionable research practices, how do you improve statistical practices, um, and again, And I think that's a, a valuable thing to be thinking about. Others have pointed out that there's also a problem with the kind of lack of theoretical conceptual depth of psychology, that there's a lot of focus on producing effects. Um, THERE'S a big debate about why, how that focus is, is connected with the current incentive structure in psychology where people have to publish and, and that often I feel, um, Leads to maybe paying less attention to the conceptual side. Um, SO, so that's been, that's been a recent part of the debate. There's also, um, I think renewed interest in the whole question of like validity in psychology. So I think, what do I think of the replication crisis? I think it has really given rise, kind of speaking positively. It has given rise to a lot of methodological questions that I'm very interested in and that are super important for psychology. And so, I guess uh the hopeful part of me thinks, um, yeah, the replication crisis has the potential to really Uh, improve practices, not just with regards to statistics, but also um with regard to other aspects. And going back to my own work, um, what do I think, I think, um, Uh, or at least coming from where I'm coming from, I would draw attention to the, to the more conceptual side of experimentation. Let, let's not just think about what went wrong, you know, how did people misuse statistics once they had their data, but what went wrong in the processes of data generation, um, And of course, people are doing that to some extent when they're talking about, you know, statistical power and like larger, uh, groups, I mean, larger uh subject groups, but also the, like the truly conceptual side of how do we, what kinds of questions ought we be pursuing, how should we be pursuing them, and how, um, Let's spend more time thinking about operationalization of questions.

59:39 Ricardo Lopes: What is a robust finding? What, what does that mean?

59:46 Uljana Feest: That depends on who you ask, I think. Um, IN, in philosophy of science, there is, I mean, I think we use the term probably, again, kind of intuitively, right? A robust finding is a good finding, a reliable finding maybe, but in philosophy of science, there's a somewhat technical meaning that again goes back maybe to the early 80s, where it's been, it's often the, the notion of robustness is often used to Describe um. Having multiple independent ways of of of reaching the same result. So then, so if you have that, that's, that's robust, robust, um. Now, in my book, I talk a little bit about that. So there's in chapter, one of the early chapters, I, um, I talk about the, the 19 40s, 50s where um people in psychology, methodologists uh were thinking about test validity. And validity, um, that's a whole, yeah, huge topic that we can't go into right now, but, but one answer was something called the multi-trait multi-method, uh, approach, where, you know, you, um, have several different methods and you use them to test for the same trait. Um, BUT you also, um, Use these methods for different traits. Um, Just to make sure that Sorry, I have to, I have to um reconstruct the logic here. So, yeah, so, so, I'm gonna just ignore this for now. Anyway, the multi-trade, multi-method uh approach is somewhat similar to the Robust to the, to the, to the logic of robustness reasoning that emerged within philosophy of science in the 80s where it was associated with names like William Wimsatt. Uh Wimsatt incidentally actually gets the idea in part or at least appeals to Donald Campbell, who is one of the architects of the multi-trade, multi-method approach. So, um In my, so, so that's maybe the kind of mainstream way of talking about robustness and also it's, it's one mainstream way of thinking about the relationship between robustness reasoning as it's used in psychology and, and sorry, in philosophy and in psychology where it maybe refers to a particular way of validating tests. Um. Now, my own take is the robustness. Reasoning understood in this way. Still is too wedded to an kind of an older way of doing philosophy of science, which really just tries to kind of validate or confirm an idea that's already there. So let's say, you know, I believe that there is, you know, whatever. Working memory exists or something, um, then I, you know, have, you know, 5 different working memory tests and, uh, and, and the, the tests are validated and the finding can be shown to be robust, um, if they're all correlated with one another. Um, NOW, that might be, um, Important and relevant sometimes, and especially it is relevant probably when you're trying to validate a test that you don't want to use, you know, in particular context, but I don't think it fully does justice to the research situation, which is more than Dynamic where you're not, you don't just want to validate the test, you want to find out something about the object. You want to find out, OK, so now I have the test, but like what else can we say about working memory? What are its features? What is its, its capacity? What is its duration? And all of these things, um, kind of go beyond mere existence claims. They're descriptive questions that we try to answer. And so in my um In, in my book, I draw on a um On A notion that I got also from the older psychological literature called converging operations. Converging operations at first sight, it might sound as if it were just kind of some robustness reasoning, right? You converge on the same thing. But in fact, um, Garner Hack and all, um, who wrote this article, um, had a different idea in mind. What they say, when, what, what they have in mind when they talk about, um, converging operations is the idea that you produce, uh, you have a particular descriptive question, for example, about your, about your object memory, you test it, uh, you get data, um, but As always, the data are in principle underdetermined by the evidence, of the, the sorry, the, uh, the, the, the inference you want to draw is underdetermined by your data. So, what this means is there conceivably, there could be a different explanation for the data. So, if you have that situation, and I think, you know, you frequently have the situation, not only that, yeah, well, in principle, there could be a different explanation, but in psychology, where we have massive, um, a massive degree of underdetermination and, and, and a massive degree of Uncertainty. Um, OFTEN, psychologists have very specific alternative explanations. They say, well, yes, it could be due to, uh, what I think it's due to. I, you know, I manipulate, I, I, I conducted this manipulation, I think I manipulated working memory, but I might have, you know, actually manipulated attention. Um, SO, so, So what do I do? Well, I have to conduct a follow-up experiment which specifically tries to differentiate between these two interpretations of the data. And, and so what we're talking about here then is not, um. It's not just kind of finding evidence in support of a particular interpretation, but it's rather a process by which the concept gets gradually refined um by kind of getting By fine tuning the kinds of descriptive questions we ask in light of the, the different possibilities that any data, um, are compatible with.

1:06:28 Ricardo Lopes: OK, so I have one last question then. Do you think that the analysis you present in your book generalizes beyond psychology?

1:06:42 Uljana Feest: I wanna be careful about that, but of course, I'd like to think, I'd like to think that they generalize in some, in some sense. So I've talked to um I've talked to physicists, like historians of physics, who typically will say something like, yeah, um, Of course, in earlier times, um, there was a lot more uncertainty about the subject matter. Um, AND so, certainly my analysis would be apt for those earlier times. However, I think even At a point in time where we have a lot more detailed, firm theoretical knowledge in other sciences than in psychology, I think the basic fact that you're, you're, um, That your data are very, that the data produced in experiments are very far removed from The kinds of theoretical or conceptual questions you're trying to test. I think that is true of all of science. Um, AND so the kind of drawing attention to what went into the, the way we design an experiment, I think that is probably um, And, and also like, where do we, where do we, um, there, there's so many kind of points at which we can question uh results by asking, well, how was it produced? I, I think that is probably true of others, of other scientists. I would certainly think it is um kind of going back to psychology, which, as I said, is characterized by this huge amount of epistemic uncertainty and openness. I think that's probably true of many scientists even today that deal with complex systems, right? And so, so in that regard, I think probably a lot of the things I say do generalize. Great.

1:08:38 Ricardo Lopes: So the book is again Operationism in Psychology and Epistemology of Exploration. I will be leaving a link to it in the description of the interview. And Doctor Fei, just before we go apart from the book, would you like to tell people where they can find your work on the internet?

1:08:59 Uljana Feest: Hmm. So, yeah, so where can you find me on the internet? I would say probably uh fill papers and fill people. Um, I think it's probably most up to-date because my, my departmental um web page is only as up to date as I keep it, which, and um, I have, I, I tend to be behind, and I think the one place where you're most likely to find me kind of frequently is Blue Sky. So I have a Blue Sky account where I post. Regularly, I mean, not frequently but regularly, so I think that that's where you would find information about me and about current and recent research.

1:09:41 Ricardo Lopes: Great. So thank you so much for taking the time to come on the show. It's been a real pleasure to talk with you.

1:09:48 Uljana Feest: Thank you. It's been a real pleasure on my part.

1:09:51 Ricardo Lopes: Hi guys, thank you for watching this interview until the end. If you liked it, please share it, leave a like and hit the subscription button. The show is brought to you by Enlights Learning and Development done differently. Check their website at enlights.com and also please consider supporting the show on Patreon or PayPal. I would also like to give a huge thank you to my main patrons and PayPal supporters, Perergo Larsson, Jerry Muller, Frederick Sundo, Bernard Seyaz Olaf, Alex, Adam Cassel, Matthew Whittingbird, Arnaud Wolff, Tim Hollis, Eric Elena, John Connors, Philip Forrest Connolly. Then Dmitri Robert Windegerru Inai Zu Mark Nevs, Colin Holbrookfield, Governor, Michel Stormir, Samuel Andre, Francis Forti Agnun, Svergoro, and Hal Herzognon, Michel Jonathan Labrarith, John Yardston, and Samuel Curric Hines, Mark Smith, John Ware, Tom Hammel, Sardusran, David Sloan Wilson, Yasilla Dezara Romain Roach, Diego Londono Correa. Yannik Punteran Ruzmani, Charlotte Blis Nicole Barbaro, Adam Hunt, Pavlostazevski, Alekbaka, Madison, Gary G. Alman, Semov, Zal Adrian Yei Poltontin, John Barboza, Julian Price, Edward Hall, Edin Bronner, Douglas Fry, Franco Bartolatti, Gabriel P Scortez or Suliliski, Scott Zachary Fish, Tim Duffyanny Smith, and Wisman. Daniel Friedman, William Buckner, Paul Georg Jarno, Luke Lovai, Georgios Theophanus, Chris Williamson, Peter Wolozin, David Williams, Di Acosta, Anton Ericsson, Charles Murray, Alex Shaw, Marie Martinez, Coralli Chevalier, Bangalore atheists, Larry D. Lee Junior. Old Eringbon. Esterri, Michael Bailey, then Spurber, Robert Grassy, Zigoren, Jeff McMahon, Jake Zul, Barnabas Raddix, Mark Kempel, Thomas Dovner, Luke Neeson, Chris Story, Kimberly Johnson, Benjamin Galbert, Jessica Nowicki, Linda Brendan, Nicholas Carlson, Ismael Bensleyman. George Ekoriati, Valentine Steinmann, Per Crawley, Kate Van Goler, Alexander Obert, Liam Dunaway, BR, Massoud Ali Mohammadi, Perpendicular, Jannes Hetner, Ursula Guinov, Gregory Hastings, David Pinsov, Sean Nelson, Mike Levin, and Jos Necht. A special thanks to my producers is are Webb, Jim Frank Lucas Stink, Tom Vanneden, Bernardine Curtis Dixon, Benedict Mueller, Thomas Trumbull, Catherine and Patrick Tobin, John Carlomon Negro, Al Nick Cortiz and Nick Golden, and to my executive producers, Matthew Lavender, Sergio Quadrian, Bogdan Kanis, and Rosie. Thank you for all.