#942 Fernanda Ferreira: Linguistics, Fluency, Speech Disfluencies, and Comprehension

00:00 Ricardo Lopes: Hello, everybody. Welcome to a new episode of the Decent. I'm your host as always Ricardo Lobs. And today I'm joined by Doctor Fernando Freya. She is distinguished Professor of Psychology at the University of California Davis. Her area of research is psycholinguistics. She uses basic insights from formal linguistic, special theories in sentence phonology and syntax to develop models of processing. And today we're talking about modifiers, speech disfluencies, misinterpretations and how people deal with them, the study of reading and information processing through wide tracking and some other related topics. So, Doctor Ferreira, welcome to the show. It's a big pleasure to everyone.

00:43 Fernanda Ferreira: Thank you very much. It's really great to be here. Thanks for the invitation.

00:47 Ricardo Lopes: Great. So perhaps to introduce people to the topics we're going to explore here and also to you a little bit more. Could you please start by telling us about the kind of work you specifically do the questions from linguistics you are most interested in and how you got into it?

01:10 Fernanda Ferreira: Ok. Um I guess I'll start at the highest sort of at a high level of, of um generalization. I'm interested in language. I've always been interested in how language works. And I began to be interested in the question of how humans process language. When I was an undergraduate, um I became fascinated by questions of uh you know, human nature and what made humans unique. What are the, what are the, what is it, the balance between nature and nurture and those sorts of questions? And in the course of pursuing those, those topics, I ended up taking classes in philosophy of mind and epistemology as well as classes in psychology of course. And I never actually took a class in linguistics when I was an undergraduate because we didn't really have a linguistics program. But through some of these other courses, I began to realize that there was this area of research this that looked at language processing from a science from or looked at language from a kind of a computational and biological point of view. And then with psycholinguistics, I discovered an area where people were looking at questions from a processing computational psychological view and trying to understand how the language system worked together with other cognitive systems, like memory and uh decision making and attention and how all those cognitive systems that are not specifically dedicated to language combined with language systems to enable us to speak and to understand each other. So uh in my own research, I focused on questions like how we use the grammar of our language to in real time to process sentences. So, uh for example, we know that in English, uh we tend to place our modifiers of nouns um before the noun. Whereas in Portuguese, they, they come after the noun. And so this is um the kind of information that a listener can use online to anticipate what sort of word type is likely to be coming up next. And in production, it's information that would be we would be using to constrain the form of our, of our sentences. So um I guess then my general approach to studying language has been to use primarily behavioral methods and I trying to understand how people generate linguistic representations in real time and how the language system works cooperatively with other cognitive systems to allow these representations to get generated, whether it's in comprehension or in production.

04:14 Ricardo Lopes: Mhm. So let's get into some more specific questions. Uh I would also want to ask you uh some of the methods you use in your research, but perhaps it's easier to get into some of the topics and then we'll, we can go through some of the methods as we talk about them. So uh you are interested, for example, in uncovering the mechanisms that enable humans to understand and generate language in real time and in cooper operation with other cognitive systems. So what are these uh mechanisms and how do you approach these kinds of questions?

04:54 Fernanda Ferreira: Um Well, so one kind of mechanism that people use when they are um trying to understand language is that they uh there's a, so there's a term we use in psycholinguistics and computational linguistics. We refer to the parser and the parser is the component of the language system that takes this continuous stream of input, whether it's auditory or signed or visual um in the visual modality and breaks it up into pieces like words and grammatical phrases and uses those phrases to obtain uh an understanding of what a sentence might mean. And so uh the parser in order to do this is going to consult grammatical information. So information about the rules of the language that allow words to be put together and possibly um in some languages, at least rules about things like um agreement like grammatical agreement. So in English, we don't have a lot of it, but like in Portuguese, for example, we have um gender agreement and that kind of information can help the processor figure out how to chunk this continuous stream of sound or of squiggles on a computer screen into discrete units like words and phrases and so on. And so in the research, I've looked at the question of how this grammatical knowledge is consulted in real time and what the language processing system does when it's faced with more than one possible interpretation that it could draw of a constituent. Um BECAUSE there's a lot of ambiguity in language, there's a lot of situations or, or the parser will often find itself confronted with input that can receive more than one grammatical analysis. Um Things like um I si saw the, I saw the boy, I saw the boy with the binoculars, you know, in that sentence. Um There are two possible grammatical analysis. You used binoculars to see the boy or you saw a boy who had, who had binoculars on him, on his person in some way. And so the question is, you know, which of how does the processing system decide which interpretation to get? And a lot of my work has been centered around questions like that where we focus on information sources like, well, maybe the system tries to compute the simplest analysis. And if that simple analysis is not sufficient or gets blocked in some way, then the system has to construct an alternate, then it goes to it to the more difficult analysis. But the default is to construct a simple analysis. Um OR maybe the system looks at the context and says, well, uh in a situation like this, it's very likely that somebody would be using binoculars as an instrument. And therefore I'm going to compute that analysis. Um MAYBE pros plays a role. So the way that the sentence is pronounced. So if I say um I saw the boy with binoculars, um I might be sort of indicating that actually, as I say that I don't know which one would be more likely. But there are claims in the literature that one way of saying it versus another way will push you in one towards one analysis or another. So in my research, we, we have tried to understand these questions using methods like um anything from acceptability judgments which are very cheap and easy to get all the way to um things like eye tracking. And we have done some F MRI and er P research as well. Although cognitive neuroscience is not my core area of research,

08:50 Ricardo Lopes: uh we will come back to eye tracking and also to how people deal with, as you said, their, the ambiguities of language. But now, uh I have another uh uh aspect of your work that I would like to explore here. So you've also done some work on how the position of modifiers in English specifically influences how words are encoded and, and later on retrieved from memory. So, uh how does that occur? And is there a difference between how people in code and process pre modifiers versus post modifiers? And, and by the way, what are modifiers to begin with?

09:36 Fernanda Ferreira: Ok. So um I'm actually looking at a presentation that I gave on this topic just to help me remember some details. And then a very important um to begin with is that the work on pre nominal versus post nominal modifiers that I've done is entirely led by my former graduate student Hussein Karimi and Hussein is now at Mississippi State University. Um When he worked with me, he was really interested in this question of how modifiers influence our interpretations of language. And so he partly came to Davis to work with me because he wanted to explore this topic with me. But I need to give credit to Hussein that this research was um designed by him, led by him. Um He's the one who kind of formulated the specific research questions that we would ask, but that we asked, but he started from work that I had done before. I got to know him probably when he was still a little boy or maybe not even born yet work that I had done with my partner John Henderson, where we looked at the um interpretation of garden path sentences. So these garden path sentences are sentences that have an, an a syntactic ambiguity in them and they're called garden path because you tend to go one way in your interpretation of the ambiguity and the subsequent material in the sentence contradicts that analysis and forces you to have to go the other way with the sentence. And the classic example is the horse raced past the barn fell which everybody hates and it doesn't really make a lot of sense, but it does have a legitimate analysis. The horse that was raced past the barn fell, that's what the sentence actually means. And in English, we can delete some of those optional function words. So when people get the horse race past the barn, they think they're done they think they have a complete active clause and then they get this verb fell and it's like, what am I supposed to do with this? I can't figure out where this is supposed to go. And so they have, and so they struggle and they finally reanalyze the sentence. Um IN the work that I did with John Henderson, we looked at sentences like uh while Mary bathed the baby played and in a sentence like that uh if you get while Mary bathed the baby, you or would be tempted to treat that as a chunk that Mary's bathing the baby. So you get while Mary bathes the baby and then you get a word like played and you're you've been led down the garden path, it turns out the baby is playing and Mary's bathing herself. And so in that work, John, in the work that I did with John, we showed that if you take that phrase while Mary ba the baby plate, if you take the baby and you lengthen it with modifiers, so you have the baby that was cute and cuddly. So the full sentences while Mary bathes the baby, that was cute and cuddly played, you really make the garden path more severe. The garden path becomes far more uh hard, it becomes far harder to recover from. And yet we show that if you have a sentence like while Mary bathed the cute and cuddly baby played, that's not harder than just the baby. So the core idea here is that we've got a noun phrase the baby and it can be modified with something like cute and cuddly like all babies are cute and cuddly, right? So it can, and in English, we can either say the cute and cuddly baby or we can say the baby that's cute and cuddly. And that's the pre nominal versus post nominal distinction. We can either put the adjectives before the noun which is kind of the default or we can put them after the noun. And so what this work that I did with John Henderson showed a long time ago is that if you put the modifiers after the noun, you make the garden path more severe. So Hussein Karimi came to me in the mid two thousands to work. Uh uh ACTUALLY after 2010 to work with me on this to pursue this issue further where he wanted to look at the possibility that modified nouns in general are richer and that we remember them better. So if you have a sentence, like I saw the cute and cuddly baby versus just I saw the baby cute and cuddly adds information that makes the concept of baby more memorable. And, and so we started out with doing some experiments where we showed exactly that, that any noun in a sentence that's modified will tend to be remembered better than a noun that isn't modified. So that just kind of established this enhanced um memorability of any noun in a sentence that has received a modifier. Um Let me know if I'm going on too much or is this? OK.

14:37 Ricardo Lopes: No, no, no, no, that's perfectly fine.

14:39 Fernanda Ferreira: Yeah. OK. So, so having shown that a noun in a sentence that has been modified is has a richer representation in memory. Um Hussein then wanted to know whether it matters if it's, I saw the cute and cuddly baby or I saw the baby that's cute and cuddly. In other words, is we already know that modification makes the noun richer. But is there a difference in how rich it becomes with modification? Depending on whether the modifiers are pre nominal, that is they occur before the noun or post nominal, they occur after the noun. And so we did a series of experiments where we showed that if you give people sentence with something like uh the dog chased the angry cat, people are more likely to continue to, to write a follow up sentence that is about the cat than about the dog. And we show that that effect is even greater if the modifier is after cat rather than before the cat. And in um other work, um Let me just consult this present this uh summary that I have here just to make sure I don't get anything wrong. Um Right. And um the information and a noun phrase is read faster when it's when it has post nominal modifiers and what has pre nominal modifiers. So, in general, what this seems to suggest is that when you have a, when you have a phrase and you're trying to interpret it as a listener or reader, you get something like the, and then the best thing is to get the noun next because the noun is the the boss of the phrase. So the cute and c if I'm saying in English, the cute and cuddly, you know, your processing system is in a state of limbo of like the cute and cuddly. What um if you say the baby that's OK. Now you, you, you have been told what is the entity that's at issue here and then you get the modifier and you can kind of conveniently attach that modification to something that you already have a token in your mental representation for. So our um work suggests then that the head of the phrase is the boss and is the thing that you're processing is centered around. And so this leads to some really interesting questions like why does English work the way it does? And why in general are there any languages where the the head occurs after its constituent after its arguments or after its modifiers? So English is a language that is um generally heads occur first, but noun phrases are the one place where we actually post can postpone the head. And what we show is that, that actually leads to processing costs. So, um really it would be better if languages worked. So that heads always occurred first and consist recently occurred first. Um But of course, there probably are other tradeoffs that are making head initial languages not necessarily more efficient than head final languages. Um But at any rate, that's kind of the, that's kind of the story about head initial versus uh well about pre nominal versus post nominal modification.

18:18 Ricardo Lopes: So another topic I would like to ask you about is speech disfluency. So, first of all, what are speech disfluencies? And uh if I got this correctly, I got from your work and your of the work of other people that there are four different types of disfluency filled puzzles like mm and uh and stuff like that. Unfilled puzzles, repetitions and repair. So, could you tell us about that?

18:48 Fernanda Ferreira: Um Sure. So no doubt in this um uh video, you've already heard a lot of examples of disfluencies. Um BECAUSE of all of us use disfluencies and, and uh display disfluencies when we're speaking. Uh The idea is that, um see, there we go, there is a disfluency right there. Um Yeah, you basically laid out the, the types of disfluencies that are out there there. The first three, the filled pauses, unfilled pauses and repeats are kind of um tactics for buying time. And then the repairs are the kind of disc fluency that we have to invoke when we didn't quite plan as effectively as we could have. And we've made a mistake and we need to correct it. We could actually even talk about another kind of disfluency, which we could maybe refer to as an undetected error where we say something incorrect. And our interlocutor might say, put the milk in the, put the milk in the stove, you mean the fridge, right? And you say, oh yeah, yeah. You know, you know what I mean kind of thing. Um So uh disfluencies are really fascinating because they are a really great example of the way that we have this, these linguistic representations. We have these grammars that we can consult, that enable us to produce and understand sentences. But when we're applying them in real time, we we often find ourselves not able to retrieve the information that we need as quickly as we would like to, to maintain full fluency. So there might be a word that's uh that's rare or that's escaping us or I can't quite remember your name. So I put in an ah or I pause um or I just say the the, you know, the the as a way of getting extra time. So, disfluencies um um that uh where disfluencies tend to occur can give us some insights into what kinds of constituents or other or words or phrases are hard for the linguistic system to retrieve or from the memory system to retrieve. And in work I did recently with my graduate students. Um uh Nora Beer and Nae Chanta Von. Um BOTH of whom are now now have their phd S and have moved on. We actually try to look at the question of whether people as they become older, become more disfluent. So o older adults are thought to become more our impression is that they become more disfluent as they age, they say um and uh and they pause more than younger people do. Um But the evidence on this point has been really mixed in the literature. And as you can imagine, it's pretty hard to study how disfluencies change with age because what you really want to do is follow somebody longitudinally. You want to start with someone in their twenties, record how many disfluencies they produce, then record them in their thirties and then they're in their forties and then in their seventies and eighties, of course, no one's gonna live long enough to do that to do that study. So it's really a tough question to address. So most of the work that's being done is what we call cross sectional, which means you compare some young people count how many disciplines they produce and some old people. But that introduces all kinds of confounds. It's not the most powerful way to get at the question. So in this study, what we did, um and it was inspired actually by um uh uh an incident where I was listening to a news to the news and they were interviewing our former governor of California. Jerry Brown, who was Governor of California fairly recently. And he was also governor of California in the 19 seventies. And I listened to him in the current times. I was struck by how disfluent he was, not that he was super disfluent, but I remember in the seventies he was just smooth, you know, you just seem to make no mistakes at all. He's talked the way people could, you know, would read a speech or something. And I thought jeez, you know, this would be a really cool way to look at this question longitudinally is to go on youtube and find videos of people being interviewed at multiple times in their lives, you know, take somebody like uh Jerry Brown, they're very well. There are recordings of him being interviewed at different stages of his life. So what we did was to collect youtube videos of people like Oprah Winfrey, David Bowie, Jerry Brown, you know, all kinds of people who by necessity are famous because most people who will be interested multiple times over their lives and you on you and who's would then be up on youtube would tend to have a little bit of, of fame associated with them. And we counted the the disfluencies in their speech at the different times of their lives. And what we discovered is that actually people do not become more disfluent as they age. Um Contrary to what anyone would have expected. What we did observe is that older adults have what's called enhanced lexical diversity. Meaning that if you look at the set of words that people produce as they get older, they come from, they display richer vocabulary. So older people have access to more words and we know this from lots and lots of, of research on aging. There's one thing that happens to you with increasing age, that's good is that, you know, more and more words. Um But as you know, more and more words, they do get harder and harder to retrieve because it's sort of like rummaging in a drawer that's full of stuff and the more stuff that's in there, the harder it can be to find the exact thing that you want. And so the slight tendency for more dis fluency that we did observe in older adults could be pretty much entirely explained in terms of their wish to retrieve a word that was a little bit more precise, a little bit less frequent that um was there probably a little bit hard to retrieve and they needed a little bit of extra time in order to pull it out of their um mental dictionary or lexicon? Um So sorry, that's like a very long winded description. But it tells us that we can look at questions like um how do people age? What happens to the language system with the age? What's the relationship between memory and language uh by examining something as simple as just people's speech and counting up how many uh and ums and repairs they produce in that speech. So that's all on the language production end. Um I haven't talked about language comprehension and fluency. So I'll just tell you really briefly that the question that we can ask. And uh is it OK. Do you want me to continue with it?

25:53 Ricardo Lopes: Yes. Yes. And then, uh we'll also have some follow up questions to all of what you're saying there. But yes, please go ahead.

26:02 Fernanda Ferreira: OK. So the thing with disfluencies and comprehension is that when I um um um when I'm doing that you as a listener are kind of on hold, you're waiting, your, so your parser, your language processing system that's been engaged is now in a state of being on hold waiting for the speaker to kind of resume the, the processing. Um Similarly, if I produce a mistake and then have to engage in a repair, you, you as the comprehend, have to invoke probably inhibition mechanisms that delete the word that I said and replace it with the word that I meant to say. So if I say something like I did this research with Nora, I mean, with Hussein, you initially built a representation where you understood me to be saying that I did this research with Nora. But if I then say no, I mean, Hussein, you need to strike that from the record kind of like a jury that's told by a judge, forget what you were just told, you need to remove that from your mental representation and replace what I said with the correct word that I intended. And what we showed in some really, um I think fun research that I did with a former postdoc named Matthew Lauder who's now at the University of Richmond. What Matt and I showed is that when people get something like put the milk back in the stove, which is going to be corrected because that's a very common kind of speech error that people will make. Um WHAT listeners do is they right away begin to detect that you have made a mistake because it's, it's not plausible to put milk in a stove. And what we showed using something called a visual world paradigm, which is about eye tracking, which I guess is something we'll get to later is that if you show people pictures of things in a kitchen, um and I say, put the casserole in the stove, your eyes go to a picture of a casserole, your eyes go to the image of the stove. But if I say put the milk in the stove, your eyes will go to the milk, but they don't even tend to go to the stove. They go, you, you look directly at the fridge as if your autocorrect brain mechanisms as you're processing language are saying, forget stove, that's not what the person meant. That's a mistake. So we process disfluencies really efficiently because we're, we're able to use what some people in cognitive science refer to as our priors where a or you can just think of it as world knowledge and plausibility information which tells us that milk doesn't belong in a stove, that stove and fridge or words that people are likely to confuse with each other. And what you're trying to do as a listener is to uncover the communicative intention behind what the speaker is saying. You're not only using the literal world words in order to do that. Um So that's another thing then we can learn from by studying disfluencies this time in comprehension, we can get a really good window into the way that our brains autocorrect um and filling in mechanisms work. So it's not just our phones and our devices that are good at um correcting our mistakes, but our brains are good at doing that too.

29:32 Ricardo Lopes: Ok. So several questions about all of that. So the first one is uh is it that every time people use a field pause, an unfilled pause, a repetition or a repair, you classify it as a this fluency? I mean, I'm asking you that because since it's so common that we use these kinds of uh this fluent or have this, this sort of this fluent behaviors while talking, I was wondering if it's always problematic. I mean, cognitively, for example, since you also mentioned there, uh comparing younger people to older people in terms of rates of this fluency and all of that, I was wondering if it would also be tied to perhaps some sort of uh cognitive issues, cognitive decline or something like that. So I I, is it always a cognitively problematic if we use these sorts of these fluence?

30:43 Fernanda Ferreira: So I, that's a really interesting, that's a really important question. So in production, what we have shown in other work that I did, I did with a former colleague named Joel ***, who's an expert on ad DH D. We looked at in, at adults who had been diagnosed with, it had received an official diagnosis of A DH D when they were young and we compared them to demographically demographic control. So people who are same education level, roughly the same age, who had the same um income level and so on who had not ever received an A DH D diagnosis. And interestingly what we showed is that people with a DH D do not produce more disfluencies than people who had never been diagnosed with a DH D except for the repair type. So what that seems to suggest is that if you've had a DH, perhaps as you might expect, your inhibition, inhibition mechanisms aren't working quite as well or as efficiently as they might in other people. And as a result, you'll say the wrong word a little bit more frequently than someone who's never had ad a DH D and have to make a repair. So, what this kind of suggests is that us and, ums and pauses are actually a good way to plan. Ok. So I was talking about the, that in cases of a DH D, uh, we can see that disfluencies are by comparing, uh, individuals who've had a DH D to those who haven't, we can get a picture that tells us that pauses are an effective way to buy time. And that in some ways, it's not a matter of never being disfluent. It's kind of like picking your poison. Do you want to be disfluent by carefully planning and pausing and saying us? And um so that you don't make a mistake or are you gonna wait more heavily, the system operating so that you don't make any pauses? But, and that increases the likelihood of errors. So it's not really a matter of ever. I don't think any mere mortal can ever be perfectly fluent. And it just is a matter of whether you're in a situation where you're going to wait precision very carefully and therefore produce maybe more pauses or whether you're going to wait fluency more heavily and therefore increase the potential for making errors which will necessitate a repair. Um That's on the production side, on the comprehension side, it's super interesting to think about disfluencies as cues. And in fact, if we go back to the work of the pioneers in this field like Herb Clark and Tom Wo this is actually what they've argued that when people are um interpreting language, they use disfluencies as information. Um For example, if I say the um cat like that's a little bit un unexpected because cats are pretty easy word to retrieve. If I say the um feline like that is the um is a little bit more expected because feline is a slightly rarer word. So if you hear me produce an R and um perhaps you can get yourself prepped for a slightly less expected word or concept. And there's really cool research by people like Jennifer Arnold demonstrating that this is true that you can use disfluencies as cues that somebody is about to refer to something that's not, yes yet, maybe introduced or has not been talked about for a little while. And in research I did with Carl Bailey, we showed that disfluencies can tell you that you're about to begin a new clause in a sentence because we tend to produce um before the beginnings of at the beginnings of clauses, nobody's just fluent at the end of the sentence. So nobody says John went to the store um like that doesn't make any sense, but we would say um John went to the store. So these disfluencies can be used as information about what a speaker is likely to say. So they help the listener get a head start on what's about to happen. Plus there's one more function of disfluencies, I should say because we now have disfluencies in written language as well. So if you uh especially in casual writing, like uh social media or um in instant messaging or direct messages to people will often in insert um or an uh because we know that they actually convey a little bit of meaning. If I say something like I heard about your um story, you know, that sort of is um I'm using um there as kind of a word to suggest skepticism about your um uh that story is, you know, about the concept of, of the story here. So uh disfluencies have a, have a number of potentially useful functions.

35:54 Ricardo Lopes: Mhm And with all of those disfluencies that we tend to have and sometimes misspoken words or using even the incorrect words. And uh all of that, how of how, how often do comprehend misinterpret language actually? Does, does it occur often or not? And with all of course, with all the ambiguity surrounding language and all of that,

36:24 Fernanda Ferreira: that's I'll give you two answers to that question. One is we can't really know and the other is that based on what we know, probably less often than we might worry, we misinterpret each other. So, on the one hand, it's very hard to know how often we misinterpret each other because misinterpretations are private, I can't tell by just sort of looking at you, right. Now, to what extent you are understanding each and every word that I've said because I don't have access to your mental states. And we know that uh well, you know, before this interview started, we, we talked a little bit about speaking Portuguese. And um you know, I can understand Portuguese far, far more easily than I can speak it because I can piece things together. I can use context and comprehension and I can fake it. I can pretend that I'm understanding and then hope that something the person says uh subsequently clarifies the word that I don't actually know. And I don't want to admit that I don't know it. So I'm pretending. Um So we really need to do experiments in order to get at misinterpretations. And what experiments like the ones we've done in my lab with people like Kyle Christiansen, for example, have shown is that indeed people can be quite bad at understanding language. They misinterpret the syntactic structure. They even get, get the words in the sentence wrong. We've shown that people interpret a sentence like the dog was bitten by the man to mean that the dog bit the man because it's much more likely that a dog would bite a human than the other way around. Um And so there, and we know from really interesting work going back to the seventies that Lynn reader, for example, did on the, what's the, the so called Moses illusion? Um Where if I ask you how many of each kind of animal did Moses take on the Ark? A lot of people will just say, 02. Um But what they really should say is uh you mean Noah, not Moses because that um you know the way the story goes, it was Noah and not Moses. So the Moses illusion reveals that people often will, we will not understand the language correctly. And these autocorrect mechanisms are so robust that sometimes they work in such a way that they're fixing the input for us and we're not even aware of it. Um And I might be trying to tell you something unexpected, but your uh systems of for processing noisy information are smoothing it out, but it may be distorting the message that I'm trying to convey. So that's one answer is that we know from a lot of research that people misinterpret each other fairly often in our experiments. But now real life, how often do people actually in real life misinterpret each other? Because the thing is in our experiments, we've designed our stimuli to be confusing. We've designed, we know the dog was bitten by the man. I've used the passive structure there. I've told you about something implausible because I want to lure you into misunderstanding that sentence. And the same goes for garden path sentences. So we can't, based on our estimates on how often people misinterpret each other in misinterpret language on the results of experiments because those experiments are designed to elicit misinterpretations. But what we can, but in when we've, what some researchers have done, not me, um I think people like um Doug Roland and uh Jan uh Jassy have done this kind of work is to look at cor big corpora of language, you know, just big chunks, naturally spoken language and looked at how truly ambiguous things are in real context. And what they show is that when you take into account all this additional sources of information that people can rely on language isn't as ambiguous as we might think. And so it's probably the case that we misunderstand each other a lot less. Then our experiments would suggest because we normally have a lot of opportunities to clarify what we meant. And just generally, the signal in real conversations is much richer than it is in an experiment in experiments. People are in a darkened room and they're, you know, looking at single sentences on a computer screen in real life, you can use my speech, but you can also use my, you know, my gestures, you can use um the what I've said in the previous 45 minutes, you know, all of these things to help you and you might also know some things about me. So you might know that certain errors that I tend to make in speech. So it's kind of a different situation in real language. So what I would say is that language. Fortunately, the language systems have evolved to be very, very efficient and to enable us to communicate. Nevertheless, we can say that we always understand each other correctly, right? The world is full of misunderstandings.

41:54 Ricardo Lopes: Yeah, I I was actually going to say and if anything that I'm about to say is incorrect, please correct me. But I was going to say that in real life. I mean, as you said, there, we have many more cues to draw from like for example, non verbal cues, body language gestures. And so for example, if I hear someone using an incorrect word, perhaps they're pointing at something and I immediately understand what, what they meant by what they were saying, right? And sometimes by intonation or something like that or uh uh or perhaps sometimes uh people misspeak, but they the the next sentence, uh they, I mean, clarify what they mean or perhaps with the next sentence, we have added context to what they said previously. And then we understand really what they were trying to say. And also I guess that even the talker, the peop the person who's speaking also react to how the other person reacts to them talking. And perhaps there are instances where we understand that perhaps we said something that the other person might have misunderstood or we might have misspoken in some way and we go back and add more information or correct ourselves. Or sometimes I guess what happens is that we just misinterpret what people said, but it's just something that doesn't matter that much. And so it makes no difference at all for our interaction, for our life and stuff like that.

43:44 Fernanda Ferreira: Right. Right. Uh, I think I, that's all of that is absolutely correct. And it makes a, helps to make a point that I've often tried to communicate in my work, which is that when we look at misinterpretations and ambiguity and how ambiguities are resolved, we're not doing it because we think this is a general problem in language. Garden Path sentences, for example, don't happen very often in real language. What we're trying to do is to uncover the mechanisms that the language processing system uses. And just like we have to invent devices for telling us for giving us measures that we can use to record what's happening. We have to invent sentences or stimuli that kind of crack the system they break. So by the reason we, we like sentences with ambiguities in them or sentences that are likely to be misinterpreted is because in some ways by showing where the system breaks, we can get an idea of the mechanisms that are involved in normal language processing. So we will, we know for example, that uh people uh seem to treat the heads of phrases as particularly important. Um And we know that on the basis of these experiments that we've done that manipulate whether there's a garden path or not and the point isn't what happens in Garden Path sentences. The point is how do people interpret heads of phrases and heads of phrases of course occur, you know, by definition, you know, every phrase has a head, so heads of phrases are all over the place. So that's just one example to make the point that yes, exactly what you say is true, which doesn't mean that it's pointless to study these weird sentences or to look at how people misinterpret each other. But that it tells us what is the architecture that's enabling the language system to be so robust.

45:46 Ricardo Lopes: Uh uh No, of course, the point I was making was just that perhaps we don't have to be too worried about misinterpreting language because someone might misunderstand what you were saying there or what I was saying just a few seconds ago and think that, oh my God, we're misinterpreting language all the time.

46:09 Fernanda Ferreira: Of course, there are situations, you know, like in a courtroom, for example, where you have to be very careful not to get misinterpreted. But fortunately, normally we're not, we're not being um questioned or grilled about anything. So we don't have to worry about that level of precision. You know, when you teach also though, you know, you're very conscious of the fact that you can get misinterpreted. You can say something in a way that's unclear, use a word that a student doesn't know that you haven't bothered to define yet. So a lot of language processing is about not only saying what you want to say, but formulating a model of your interlocutor so that you can uh produce utterances or write sentences that are likely to be understood. So the burden is not just on the comprehend or to understand, but the burden is also on the speaker or the writer or the signer to produce uh sentences that are appropriate to the person they're communicating with.

47:15 Ricardo Lopes: So, still on the topic of uh fluency of language production, does it have any relationship at all with individual differences? Like for example, in terms of intelligence and executive function?

47:32 Fernanda Ferreira: Um Yeah, so that's something that I looked at in some of my earlier work. And um and I mentioned the work that we had done on A DH D and it, there is some evidence that the that's comprehend well. So for, let's take an obvious example, like vocabulary. So a component of language production is producing words and producing words that fit the meaning that you're trying to convey as accurately as possible. Um That's going to be related to the size of your vocabulary. And we already talked about lexical diversity in the context of, you know, older adults who've had more time and experience in which to acquire bigger vocabularies. Um We also know that whatever you might think of the validity of um standardized assessments like IQ tests and um GREs and things like that. A big component of those tests is vocabulary. And so uh the people with bigger vocabularies tend to end up with higher scores on things like IQ tests and um other kinds of standardized assessments. And so that would suggest that there's a relationship between IQ and uh language production, but it's really um probably mediated by other things that we don't really understand very well, like uh how that vocabulary is acquired, what kinds of experiences have gone into the opportunities that people have had to acquire vocabulary. And then some of the factors you've also mentioned like inhibition. Um So, uh the relationship between inhibition and and other cognitive systems is something that we're only beginning to understand. But there are teams of researchers um like Al Kim and Jared Novik, for example, who've been focusing heavily on the role that inhibition might be playing in individual differences in language processing. So, um I think it's a really interesting question. I think it's really, really hard to study because uh there's so many dimensions on which people can differ from each other. But in general, um there's, it's almost certainly going to be the case that individual differences in cognitive skills are going to correlate with language production and comprehension skills as well.

50:03 Ricardo Lopes: By the way, on the topic of comprehension, in what ways does redundant information contribute to it?

50:12 Fernanda Ferreira: OK. So that's a really big question. Um Right. So, redundancy in language has been a topic that we've been, it's motivated linguistic and psycholinguistics theories for decades and decades. And it goes back back to the gray and maxims about, you know, to be brief is, you know, one of the things you're supposed to do and being brief means don't produce any unnecessary words. Uh And there, there are views currently of the efficiency of the language processing system that kind of go along with that idea that, you know, generally speaking uses up metabolic uh is, you know, is metabolically demanding. So I want to communicate my ideas as efficiently as I possibly can. Redundancy would seem to be um inconsistent with that goal. Um So, you know, Bryce would seem to be on the right track when he says that people should not be redundant. But as more and more work is showing speakers are redundant and listeners benefit from that redundancy. Uh So, for example, um what uh Grace would say is that, for example, I should not call this a green pen. I should just call it a pen because there's no other pen right now in the discourse. So to include the color at modifier is pointless. It's, it's information that you don't need. Um But what we now know from a lot of research is that for example, color as a trait is an extremely salient property. And in fact, there's research showing that I'm more likely to call this a green pen in English. Um This is worked by Rubio Fernandez. Um I'm more likely to call this a green pen when I'm speaking in English than I would if I were speaking in Spanish or they say Portuguese because the adjective comes first. And if I'm speaking in English, and I'm going to say the, you know, by producing green, you know what I'm referring to because in this context, I can't quite see what's in my background. But the green things are this pen, maybe that plant and the stuff in the outside the window. So by producing a redundant adjective and or at least an adjective that isn't necessary in the context because it's the only pencil, I actually will help you to identify the referent more quickly. Um So, and we also know that people say things like uh repeat again and repeat again is redundant. Um And I have a phd student working with me named um Casey Felton and he's looking at uh people's production of things like lift up. So in English, we say lift up the table. Um BUT the meaning of lift is to elevate. And so you don't need that particle in theory, it's, it is truly redundant. Um But this is so, so common in com in regular language, we say add in subtract out, mix together uh combine together it's, you know, just all over the place.

53:33 Ricardo Lopes: So all of those expressions redundant expressions we also have in Portuguese like for example, repeat again, we say Ruperto Trevis something like that. So we're always saying that as well,

53:49 Fernanda Ferreira: which suggests that it's a general tendency on the part of humans that they are redundant. So I think then the question gets to be, well, what is efficiency and efficiency isn't just producing the fewest syllables per unit of meaning it could be efficient to provide listeners with redundancy because it does give them multiple opportunities to get your message. And we had just been talking about the uh the opportunities for misinterpretation that are in language and how perhaps in real conversations we're not so bad at understanding each other. Maybe it's because we're redundant and repeat again in is technically redundant. And your teachers would yell at you for saying that. But in real life, if I'm teaching my class and I say, OK, I want you to redo this again. Um I tend not to say that but imagine if I said I'm giving my students two chances to get my meaning. I give them the re on redo and I give them again as a separate word. And now my meaning becomes clear. Um I see this a lot in, I don't know if you ever do like online um exercise videos and things where the instructor will say, OK, so let's repeat that again. And you know, because it's really important that you understand that we're going to now do the same exercise that we just did. And, you know, you need to understand that that's what's coming next. So repeat again, is very healthy or helpful, healthy. Um There, there was this feature. Um So in general, I think redundancy is valuable. Um PARTLY because listeners attention wanders and they uh they might not know what you meant at this point, but they will know what you mean at a later point in the sentence. So there's more and more of an appreciation and a respect for redundancy in language that it actually benefits the comprehend or uh and allows the speaker to convey their message more clearly. It also is just less, less abrupt. So lift, lift that versus lift that up, you know, the up is somehow gentler it g it makes the instruction a little bit gentler and um I don't quite know what's going on with that, but I have that as an intuition.

56:17 Ricardo Lopes: Yeah. So I have two more topics I would like to ask you about. Uh WE'VE already mentioned eye tracking several times here today. So how do you actually study reading and information processing through eye tracking? And what can we learn from it

56:37 Fernanda Ferreira: exactly? So we know that the eyeball is structured so that there is a, so we know about the retina on the back of the eye, you know, the collection of cells that are responsible for our ability to um process visual details, especially the cones, right? The cones are the the cells that are allowing us to, to read essentially or to extract visual detail. And the this foveal area vision is actually quite small. And what that means is that we need to make eye movements in order to process the information in the environment. And eye trackers capitalize on that fact. So when we read and imagine we're reading text like English or Portuguese that's read left to right top to bottom. Um We will, our eyes will land on to right around the middle of a word. Typically, it's a little bit left of center. Um BUT right about the middle of a word and then we move our eyes to the next word and then we move our eyes to the next word and we, and we're basically reading each and every word in a text except for words like the and of which we tend to skip over probably could because we can see them in our peripheral vision and they're also highly constrained contextually. So we, when we look at a visual scene, it's the same thing. So when you look at like what's behind me in order, you can get a sense in in one fixation that you're looking at some kind of a home office. But you in order to identify what are the actual objects in the scene, you have to make fixations all around it. You know that there's a clock over here and that there's a picture on the floor for some weird reason underneath the window, you know, et cetera. So, um what this means then is that the position of the eye is giving us information about what the comprehend is attending to. So, if I want to know what word you're paying, you're currently processing. When you're reading some information on a computer screen, you, the position of your eyeball is gonna tell me that. So what I need is an eye movement monitoring system that essentially records the position of the pupil that tells me where you're looking. And that eye movement recording system allows me to identify the exact word that you're currently looking at and even the character within the world that you're looking at. Um In fact that you can even get more precision than, than a single character on a computer screen. So what we've learned from this kind of eye tracking research is that, as I've mentioned, people tend to fixate on each individual word except for some highly predictable, highly frequent words and that the fixation durations. So the length of an individual fixation correlates with the difficulty of the word that's currently being processed. So a word like cat would be, would receive a shorter fixation than a word like feline because feline is a longer word, but also because it's a less common word. And we also know that when people get garden path sentences, when they get to the part of the sentence where they realize they've messed up, they will exhibit long fixations and they'll also do multiple fixations. So they usually, you only look, you only fixate on a word once or twice. But if this, the sentences become confusing, you'll look at it multiple times and you'll actually also make a backwards eye movement called a regression as if you're sort of going. Oh This is hopeless. Let me start again here and you go backwards and skilled readers make a lot of regressions. It's not a sign that of failure. It's actually what efficient reading looks like. So we use the eye movement monitoring system or eye tracking to give us. Uh YOU know, I mentioned before that language processing comprehension is, but we can actually make it a little less private by monitoring the position of your eyes. I can tell if I put you on an eye tracker, what words you're looking at, how long you're spending on that word. Um How long the eye movement is over to the next word, et cetera. And that gives us a lot of valuable information about information processing uh during language comprehension.

1:01:06 Ricardo Lopes: So I have one final question. Then this is a topic that is I guess unrelated to the others to some extent. But when it comes to language learning, and I guess that many people out there would be interested in language learning because they are parents or educators or they, they are interested in knowing if we can improve language learning, particularly in Children, I guess. So can singing facilitate language learning.

1:01:38 Fernanda Ferreira: Oh, gosh, that's also, that's also a study I probably should have suggested we not try to go into in too much detail because I'm going to tell you. So, what I'm gonna do is I'm gonna hijack this part of our conversation to tell you a little bit about how we often do science. So that study is a complete one off. It's a study that I did with, um, uh Katie Overly and her phd student at the time when I was at Edinburgh and I served on the committee of the graduate student who was working on that project. And what we showed in that study is that, yeah, it looks like um, singing can help people to learn, you know, a second language. Um But that was a study that I worked on um completely as just sort of like I said this one off, I've never really pursued it again. So I couldn't tell you what the current literature on that topic looks like. I also would emphasize that that study focused on learning second language. I believe it was Hungarian. Um THAT, that she was focusing on as, and she used that just because it was unlikely that very many of the subjects we would want to test already learn New Hungarian and, or maybe it was Czech, see, I can't even remember what the second language was that we were looking at. And um, but that's probably a different process from what we see in first language learning in Children. Although we know that teachers and you know, educators, parents like to use songs as a way to teach kids words. Um Maybe it makes them more memorable, memorable, makes them uh the repet repetition helps them to learn. Um Also it helps Children I think to learn phonological patterns. So in, in song, you tend to get rhymes and so rhymes are going to facilitate phonological awareness and phonological awareness we know is really important for learning how to read. Um In fact, you might know, I don't know if you know about the studies that were done a few decades ago on um on Portuguese illite illiterates. Um So people who after the revolution were um put into literacy programs in Portugal. So adults who had never had the opportunity to learn to read. And um so like uh um oh his name is escaping me, a Portuguese researcher who spent his, most of his career in Belgium did a lot of these um studies along with Jos Allegria and some other people anyway, they show that people who are, who do not read and write. Adults who did not know how to read and write had very little phonological awareness. They could tell you how many syllables were in a word. They kind of intuitively understood things like syllables, but they didn't understand the things like phone names. And once they learned to read the same adults could tell you that a word like um stick has four sounds to it. Um Whereas prior to literacy, they would say it had maybe three sounds to it. Um You know, if that anyway, so all of that is to say that phonological awareness which could be something that is conveyed through music, helps people to become literate or is a consequence of becoming literate. It's probably both a cause and a consequence and in that way is sort of related to language learning. But here I'm kind of speculating and I'm in some ways out of my specific area of expertise. So,

1:05:15 Ricardo Lopes: no, that, that's fair enough. So, Doctor Freud, just before we go, would you like just to tell people where they can find you and your work on the internet?

1:05:26 Fernanda Ferreira: Um Yeah, so I, you can find me in Davis, California most of the time. Um OR in San Diego where my brother lives with his child. My brother is also a psycho linguist. Um So here at the University of California, San Diego, um but on the internet, if you Google, my name, um you usually, I will come up pretty, pretty much the first or second item on your search. There is a Fernando Ferreira in Brazil who's a volleyball player. That's not me. Um But uh I, and if you Google Fernando Ferreira Davis, then you, then you will find me pretty easily. So, yeah, I'm easy to find Fernando Ferreira as I tell people is a very common name. Um BUT not so much in cognitive science. So

1:06:17 Ricardo Lopes: yeah, probably in Portugal it would be much more common,

1:06:22 Fernanda Ferreira: I guess. And Brazil, it seems like it's quite common in Brazil. So um yeah, yeah, when people say your name, it's so unusual. It's like only here.

1:06:33 Ricardo Lopes: Great. So I'm leaving some links to your work in the description box of the interview and thank you so much again for taking the time to come on the show. It's been a real pleasure to talk with you.

1:06:45 Fernanda Ferreira: Thank you for the opportunity. I've really enjoyed this conversation. So thanks.

1:06:50 Ricardo Lopes: Hi guys. Thank you for watching this interview. Until the end. If you liked it, please share it. Leave a like and hit the subscription button. The show is brought to you by N Lights learning and development. Then differently check the website at N lights.com and also please consider supporting the show on Patreon or paypal. I would also like to give a huge thank you to my main patrons and paypal supporters, Perera Larson, Jerry Muller and Frederick Suno Bernard Seche O of Alex Adam Castle Matthew Whitting B no wolf, Tim Ho Erica LJ Connors, Philip Forrest Connolly. Then the Met Robert Wine in Nai Zuk Mar Nevs called Hofi Governor Mikel Stormer Samuel Andre Francis for Agns Ferger Ken Herz J and Lain Jung Y and the Samuel K. Hes Mark Smith, J. Tom Hummel s friends, David Sloan Wilson. Yasa dear, Roman Roach Diego, Jan Punter, Romani Charlotte Bli Nico Barba, Adam Hunt, Pavlo Stass Nale Me, Gary G Alman, Samos, Ari and YPJ Barboza Julian Price Edward Hall, Eden Broner Douglas Fry Franca, Bela Gil Cortez Solis Scott Zachary. Ftw Daniel Friedman, William Buckner, Paul Giorgino, Luke Loki, Georgio Theophano Chris Williams and Peter Wo David Williams Di A Costa Anton Erickson Charles Murray, Alex Chao, Marie Martinez, Coralie Chevalier, Bangalore Fist Dey Junior, Old Eon Starry Michael Bailey. Then Spur by Robert Grassy Zorn. Jeff mcmahon, Jake Zul Barnabas Radis Mark Temple, Thomas Dvor Luke Neeson, Chris to Kimberley Johnson, Benjamin Gilbert Jessica. No, Linda Brendan Nicholas Carlson Ismael Bensley Man, George Katis Valentine Steinman, Perlis Kate Van Goler, Alexander Abert Liam Dan Biar Masoud Ali Mohammadi Perpendicular Janner Urla. Good enough, Gregory Hastings David Pins of Sean Nelson, Mike Levin and Jos Net. A special thanks to my producers is our web, Jim Frank Luca stuffin, Tom Veg and Bernard N Cortes Dixon Bendik Muller Thomas Trumble, Catherine and Patrick Tobin, John Carlman Negro, Nick Ortiz and Nick Golden. And to my executive producers, Matthew Lavender, Si Adrian Bogdan Knits and Rosie. Thank you for all.