#1281 Francesca Rossi: The Benefits and Risks of AI

00:00 Ricardo Lopes: Hello everyone. Welcome to a new episode of The Dissenter. I'm your host, as always, Richard Lops, and today I'm joined by Dr. Francesca Rossi. She's an IBM fellow and the IBM AI ethics global leader, and today we're going to talk about AI ethics. So Dr. Rossi, welcome to the show. It's a pleasure to everyone. Thank you. Thank you for having me. Uh, OK, so let me start perhaps with the most basic question here. So what is AI ethics and what kinds of questions does it deal with?

00:35 Francesca Rossi: So AI ethics can be seen as a very multidisciplinary field of study that Includes experts from of course computer science and AI but also from social sciences like you know people that work in sociology or psychology or philosophy or economies and and many others as well as also other stakeholders like consumer rights associations or so and all these people together try to identify and also. So possibly understand how to mitigate AI, you know, impact, uh, possible negative impact on people, society, the environment, and so on. And these mitigations usually are not just technical ones, but are mostly, you know, so technical, so involve, you know, people, decisions, leadership, uh, policies, best practices, and so on.

01:41 Ricardo Lopes: So when you think about AI, what do you think are the core opportunities associated with AI? I mean, what are the most positive impacts that AI can have on society if we develop it correctly, not only from an ethical perspective, but also I guess from Uh, a technological perspective. So, uh, what are the main gains that we can get as human societies from AI?

02:14 Francesca Rossi: Yeah, so, um, so first of all, uh, to identify the opportunities, one has to think about what is the goal, and then you say, OK, then there are opportunities to reach that goal or that vision of the future and and for me, of course, AI is not the goal. The goal, the goal is that humanity, you know, thrives and grows and people learn more and they're more conscious about themselves and possibly they live, and this can lead to the ultimate goal which is living together peacefully and you know, collaborating with each other and so on. So given that goal then. Then one can identify whether AI gives us opportunities that are specific to the capabilities of AI to reach to get closer to that goal, and I think that there are many because AI can besides the usual one that one can think about that AI can help us solve problems that we haven't solved, that can help us people, can help solve, you know. Problems in healthcare that, you know, that uh we, we don't know uh uh still how to solve or that can help us tackle with the climate issues or things like that. So global problems that people so far, they made a lot of prog we made a lot of progress, but we still cannot solve, uh, so that, that's one opportunity for AI of course AI can also, In an enterprise view of the role of AI can also augment, you know, productivity, augment the scientific discovery, augmented efficiency, and so on, but I think that most AI, uh, another thing that is very important for me is that really AI can help humans grow as human beings, and so we can, for example, take our um. We can, they can, it can help us use our cognition in ways that are better suited for humans, that can help humans grow as human beings, be more conscious of what we are, learn more, and therefore learn also how to live with the others, you know, in a peaceful environment, so that. ARE all the opportunities of AI, many, many multifaceted opportunities, whether you are interested in productivity, efficiency or in scientific discovery or in solving global problems, but most of all, an overarching thing for me is really to help humans grow as human beings individually and collectively.

05:03 Ricardo Lopes: Uh, SO you mentioned goals. Let me ask you then, because you said that first, of course, we have to establish the goals we have in mind and then we can look at the opportunities that AI can offer us. So how do we go about establishing those goals and who should participate in that process of establishing the goals we have in mind when it comes to uh applying AI?

05:31 Francesca Rossi: Well, I mean, the governance of AI is I mean it's a global thing, you know, the the governance of AI has to be put in place by many different actors and stakeholders around the technology, which companies that build or use AI, of course they have an important role to play also because they can put together a governance that is agile and faster because it has to go, you know, at the same pace. Possibly as the evolution of the technology, then there is the role that governments may have in putting some baseline rules for everybody to comply with and then there is also the role of global governance, best practices or that are more about the goals like uh what AI role. Be in our society and and then there are more general goals like the UN sustainable development goals that are not about AI but that they define a vision of the future that these organizations have decided that collectively that that that should be what we aspire the future to be.

06:57 Ricardo Lopes: So since we're talking about AI ethics, we need also to consider the potential risks associated with it and not just the benefits or the opportunities we can get from it. So what would you say are the main risks we should worry about?

07:17 Francesca Rossi: Well, of course, over the years these risks have evolved because the capabilities of AI have evolved and with new capabilities and new techniques there come new opportunities, new applications, but also new or amplified risks. So the usual ones that were already, um, you know, were already with the more traditional and narrow machine learning, not even within. You know, we didn't have to wait even for generative AI or risk around privacy, about fairness, about transparency, about explainability, about robustness, you know, these were already there when we even we didn't have generative AI, but with generative AI there are new and amplified risks that have to do with the content that is generated with AI. So where there is a risk of, you know, hallucinations and deep fakes, so the epistemic risk that we no. LONGER know what is true and what is false, and we may, uh, you know, be so confused that by by this effect that we may not know how to make decisions because the informed decisions about facts and then we make our own opinion on top of that. But if we don't know what is true, what is false, then this whole structure you know of our decision making is a bit disrupted. And then of course there are Issues about um agency, also with the use of genetic AI and other AI techniques in agentic AI so we may may risk of delegating too much, you know, over reliance to this agentic AI system that you know. I think that a genetic AI system gives us a lot more opportunities to automate, simplify everything that we want to do, but also they may be vulnerable. They may do unintended actions that may be irreversible, so we We have to be careful about delegating to the systems only when we can make sure that they do what we ask them to do and so we give the agency of them to act on our behalf but only under certain assurances and so this. Um, AND, and that's another, you know, amplification of risks that goes from with with agentic AI that has to do with not just one model like a LLM that can hallucinate or can do things that we don't want him to do, but also with the fact that we were putting together an agentic AI system. Several components. One is an LLM. One is a tool. One is another thing that communicate with each other, pass data from one to the other one. And so the risk can be not just in each component bringing some possible risk, but also in the way these things are combined and they pass information between them. And then of course there is also another risk that has to do with the Impact on the environment, you know, this system requires a lot of resources to be trained and also to be used. And, and because they need so much, you know, energy and resources and funds, then there can be the risk of power concentration which is a risk in the sense that power concentration means that only the opinion and the value. OF few actors are embedded into an AI system. That's why, for example, at IBM and in many others, there is a trend for open source models that allow more, you know, points of view to be possibly embedded into the model or or into the post-training, you know, phases so that Each community can have its own model that is tuned to the needs of that community.

11:48 Ricardo Lopes: Mhm. So whenever people discuss the risks associated with AI, sometimes people bring up, uh, the possibility that maybe somewhere in the future, AI could uh reach a level of development where the AI systems would no longer or may uh could no longer be under our control and start behaving in ways that do not. Go in line with the orders we give them, etc. Uh, DO you think that that is a a risk worth considering? I mean, how likely is it that AI systems could just go against our orders and in a more, I don't know, apocalyptic kind of scenario, take over society? Is that something that we should take seriously at all or not?

12:47 Francesca Rossi: So I think that you know talking about these risks only in the context of a possible future for super intelligence or AI is a bit of a I mean, that's not what I like to do because I think that those kind of risks are also present now with current AI that is not yet super intelligent or intelligent like a human being, um, but it it is much better than human beings in some dimensions and much worse in others, like still not as. Uh, RELIABLE, I think, or, or at least making mistakes that humans would not make, but also being much better than humans and other things. So, so I don't have this linear view of intelligence that AI is getting, you know, better and better at that in that linear. TRAJECTORY until it reaches something because I feel that intelligence is very multifaceted and multi-dimensional. So I still, I think that right now with the current state of AI, AI has capabilities that are Allowing a lot of emerging behaviors, which is good because with this emerging behavior we get amazing, you know, interactions with the chat, you know, with conversational chatbots that can help us, you know, think about new ideas or discoveries and so on, so amazing things, but also these. BEHAVIOR may be unintended sometimes because it's not easy for humans or whether you use a piece of code or natural language, it's not easy for us to specify exactly all the boundaries of the behavior of an AI system and so the emerging behavior can be within the boundaries we. Because we don't specify everything, but it can be unintended for us and then it can be also unintended in a negative way. So say AI, we have seen an AI system that do things that those who designed and built the AI system did not intend and did not want them to do. And so there is already the risk of AI. Going beyond our control in the sense that it generates these emerging behaviors that are not expected and are not intended, so. So I think that again we don't need to wait for a future of super intelligence for being to be really in the presence of these unintended behaviors. So if you want to say things are out of our control because an agent does things that we don't want him to do. It can be seen as something that is out of our control, but not because again of some intentional thing from AI. So AI does not have intentions. AI does not have morality or ethics. It's just designed by us according to certain objectives, objective functions. But, but again, these emerging behaviors are amazing and good, but also they can include some risks.

16:33 Ricardo Lopes: So, uh, with, uh, keeping in mind the risks that you mentioned earlier, I mean, everything from, uh, deep fakes, from the violation of people's privacy, from the spread of disinformation, uh, contributing to climate change as well through, uh, the high consumption of resources like electricity and water. How do you think we should best mitigate these risks?

17:03 Francesca Rossi: Yeah, so as I said, you know, that's all about the governance and the more capabilities you have like going from traditional machine learning to generative AI to agenttic AI, and the more, you know, it's clear that you need the governance in done in a very structured and principled way and and the and the governance. Like, you know, personally, you know, I, I'm very much involved into corporate governance, building inside the corporate company the right way of assessing the risks and then building the tools or the other processes to mitigate them. Uh, AND of course inside a company, uh, governance can mean many different things. It can mean a checklist, which is not ideal as a way of governing a checklist, you know, did you do this test, da da da, done, that's not the right way. Because it needs a lot of pieces to be put in place from the structures that make decisions about AI ethics like the AI ethics boards or things like that, or the internal processes that do the risk assessment. The culture and the education of everybody, you know, around the AI ethics and the risks and the mitigations that are available, um, the leadership culture that should reward and incentivize the right behavior of the teams and incentivize, you know, and not say just speed for delivery that. Does not help in, you know, in actually delivering the right kind of technology. And then the integration of all these pieces of this AI ethics governance into the processes that the company already has in place. So all these pieces are very important, you know, the incentives and the people kind of pieces are even more important than just producing the technical tools to actually do the test or the benchmarks or the mitigation and so on. Um, AND then of course I think that regulations has a role in playing in this governance overall, uh, as well as standards, international standards that really play an important role. Um, AND, and, but they have complementary roles. I feel that regulations can be a baseline for everybody to be on the same level of compliance, and then each company, depending on the business model, can decide which are the risks that are more relevant or less relevant for that business model and therefore build the governance depending on that, um, so. So the mitigation has to do also with the education of the users that of course in an enterprise AI environment is easier in a sense because for example IBM delivers AI solutions to other companies and in those companies we can help those companies to train the AI users so that they use it in the most appropriate. Way you use it, you know, you use it only when it was tested in the environment in the scenario where it was tested, or you use it knowing that it can hallucinate. So you train the users in a more consumer AI environment where you have a conversational chatbot that everybody in the world can use asking any questions about any topic. Then it's a bit more difficult to train all these users, but of course there is the need for governments and societies in general to help everybody understands, not being experts of AI, but everybody understands what's the right way, the appropriate way to use AI, knowing the capabilities, but also knowing the current limitations.

21:23 Ricardo Lopes: Do you think that there's any set of universal principles that should undergird the development and adoption of AI, and if so, what would these principles be?

21:39 Francesca Rossi: Well, I mean, in general, I think that again this idea that AI should augment and support human capabilities, human cognition, human thriving, human growth, that's, I think should be rather universal. But apart from that, there are a lot of values like again we mentioned privacy, we mentioned transparency, credibility, a lot of values that are important for human beings, fairness, no. But that are spelled out a little bit different in different cultures and that are prioritized in different ways because when you make decisions about AI regarding some AI ethics issue, it's never a decision that you can avoid or completely go to zero with the risk. Like consider fairness for example. Fairness has more than 21 definitions. There was a famous tutorial with 21 definitions of fairness, but fairness has many, many different definitions. Some of them are more appropriate in some use cases, some others in other use cases. And so the way a community decides, you know, which is the fairness definition that is most appropriate for that community may be completely different from what is appropriate for another community or another culture. Also, most of the decisions that You make about not just about one value fairness that has a lot of different definitions um but also even if you decide one definition, you may have fairness between different variables like protected variables or race or ethnicity or age or gender and so on and so there is a lot of inter inter intersectionality. Issues meaning that you may reduce bias, you know, on, on some protected variable, but this may improve, increase the bias in another thing so the things are not independent on each other, so it's always almost impossible to reduce and put to zero bias in all the dimensions that you want. So you always have to think about. Decisions in AI ethics as reaching trade-offs. Trade-offs between, in this case, different, you know, fairness on different dimensions, but also trade-off between different values like for example between control and autonomy, between innovation or speed and safety, between fairness or accuracy and privacy, so you always have to reach. A trade-off between different properties or values that you care about and so that's why those that make decisions in governments or in companies about AI ethics need to be very well trained in thinking. It's not just a checklist. You cannot reduce it to a checklist because you need to be able to make these nuanced judgments about these trade-offs. And something that can inform these trade-offs, that's why it's very important the cooperation with the philosophers, for example, you know, it's like because depending on the ethics frameworks that is in your mind or in your community, so it's more the ontologies, then uh then you have some red lines of things that absolutely are wrong independently of what the consequences are, and some actions are definitely wrong. You won't, you don't want to do that. And you don't want the AI to do that. Or if you are more of a utilitarian approach, there are no bad and good actions. I just look at the consequences of these actions and uh and then because of that I make my decisions. So these ethics frameworks or virtue ethics or Confucianism or Ubuntu ethics. So depending on where your community is based in terms of its culture and, and the ethics frameworks that are um. Supporting, you know, in embedded in that culture, then you'll see making different trade-offs among that all different decisions around the trade-offs that are typical in AI ethics.

26:20 Ricardo Lopes: Do you think that with all of that in mind and since AI systems and the principles that undergird them would or are developing in different socio-cultural contexts, the participation of anthropologists would also be warranted. I mean, could they play an important role here? Yeah,

26:42 Francesca Rossi: yeah, understanding how cultures, uh, you know, are, um. Our cultures evolve and are thinking about these issues, definitely can be, and I don't think in the discussion that I've been involved in several years, I don't think I've seen anthropologists, I've seen philosophers, psychologists, sociologists, economists, but not anthropologists, and it may be a good idea to include them as well.

27:09 Ricardo Lopes: So, uh, I have one last topic I would like to explore with you, with you. I read in your work about a good AI society. Could you tell us what does that term mean or, or that idea mean? What is a good or what would be a good AI society?

27:30 Francesca Rossi: Well, to me, I mean, a good AI society is not a society where AI is used everywhere or AI is not used at all, but it's a society where the use of AI is deliberate. So the use of AI is conscious, conscious human consciousness, so that humans use AI in a conscious and deliberate way and where they are. They know where they and they can decide whether they are subject, for example, to automated decisions and so and and they retain their agency, so they have ability to control the AI to not being subject to decisions by AI or yes if they want to to possibly override the decision of AI. So it's really a society where AI augments human agency and judgment and does not really instead quietly or you know little by little replacing that human judgment and agency so where humans retain their agency, their consciousness of how to deliberately and intentionally use AI for what they want and maybe not for what they don't want.

28:51 Ricardo Lopes: Uh, GREAT. So, uh, let me just ask you then, uh, how do you look at the future of AI? Do you see it developing in a proper way? Do you think that we are perhaps, uh, under, uh, exploring some of the, some of its potentialities, or maybe it's, uh, developing, uh, too fast?

29:16 Francesca Rossi: Well, I mean, uh, developing too fast, I'm not sure. I mean, it's just that uh maybe it's it's at the at the deployment phase that we have to be careful about deploying when we have all the right governance structures in place, um, but um I think that for me the future of AI is. Really, first of all, as I mentioned already, is not just the future of AI but it is the future of humans with the support of AI. So then in that picture you have to put together both actors, you know, the technology, but also the human beings, and I think that we are moving in a direction where. WHERE AI is being AI capabilities are the focus, you know, we want to expand and improve AI capabilities, but I think that there is a space for thinking more about how these AI capabilities fit within the goal of augmenting humans' capabilities and humans agency, so. Currently in my project I try to be inspired also by cognitive theories of how humans make decisions or how humans even relate with the technology to build AI, you know, to improve AI capabilities in a way that really supports that vision. So, but overall I mean I think that there is. Not many people that are not amazed by the incredible evolution of these AI capabilities or they're not amazed about, you know, interacting with the conversational chatbots or looking at the realistic images that are produced, so it's really amazing. It's something that even researchers in AI were not really. Participating for many, many years, researchers, for example, tried to harness you know natural language and uh and uh and uh and and this uh the fact that we that now we did do that it was a surprise and uh and and this allows this was a big game changer because it allowed. Allows everybody, not just people that can code to interact with AI systems and to have them do whatever they want them to do in a very easy way, and that changes really the impact of AI that can that AI can have, not just, you know, in supporting those that produce code or those that are in companies that can use AI delivered as a system. OR as a service, but everybody, so this incredible democratization of the delivery of AI capabilities to everybody is going to really to change, and I hope that people take this opportunity to do the best with what they have, you know, before generative AI people had all the knowledge in the world. In their phone, you know, like on, on the web, and they could access it easily and so on, but now they have much more than that. They have all the, all the, I mean, all the is a bit, uh, uh, I mean not really true because AI learns from whatever is on the web, which is not all the knowledge, of course, and that's the problem also, and that's one of the risks because it doesn't represent all the. Communities equally well or the language is equally well, you know, and also there are some, many things that we don't write on the web like common sense reasoning things. So that's why the AI has trouble with those. But, uh, but so everybody has this this cognition, artificial cognition, which is different from a human one available at their hands. So I really hope that they can use it in the best way.

33:40 Ricardo Lopes: OK, so Dr. Rossi, I will be leaving links to your work in the description of the interview and thank you so much for taking the time to come on the show. It's been a fascinating conversation. Of course, thank you. Hi guys, thank you for watching this interview until the end. If you liked it, please share it, leave a like and hit the subscription button. The show is brought to you by Enlights Learning and Development done differently. Check their website at enlights.com and also please consider supporting the show on Patreon or PayPal. I would also like to give a huge thank you to my main patrons and PayPal supporters, Perergo Larsson, Jerry Muller, Frederick Sundo, Bernard Seyaz Olaf, Alex, Adam Cassel, Matthew Whittingbird, Arnaud Wolf, Tim Hollis, Eric Elena, John Connors, Philip Forrest Connolly. Then Dmitri Robert Windegerru Inasi Zu Mark Nevs, Colin Holbrookfield, Governor, Michel Stormir, Samuel Andrea, Francis Forti Agnun, Svergoro, and Hal Herz Agnon, Michel Jonathan Labrarith, John Yardston, and Samuel Curric, Hines, Mark Smith, John Ware, Tom Hammel, Sardusran, David Sloan Wilson, Yasilla Dezarajo Romain Roach, Diego Londono Correa. Yannik Punteran Rusmani, Charlotte Blis Nicole Barbaro, Adam Hunt, Pavlostazevski, Alec Baka Madison, Gary G. Alman, Semov, Zal Adrian Yei Poltontin, John Barboza, Julian Price, Edward Hall, Eddin Bronner, Douglas Fry, Franco Bartolotti, Gabriel Pan Scortez or Suliliski, Scott Zachary Fish, Tim Duffyanny Smith, and Wiseman. Daniel Friedman, William Buckner, Paul Georg Jarno, Luke Lovai, Georgios Theophanus, Chris Williamson, Peter Wolozin, David Williams, Di Acosta, Anton Ericsson, Charles Murray, Alex Shaw, Marie Martinez, Coralli Chevalier, Bangalore atheists, Larry D. Lee Junior. Old Eringbon. Esterri, Michael Bailey, then Spurber, Robert Grassy, Zigoren, Jeff McMahon, Jake Zul, Barnabas Raddix, Mark Kempel, Thomas Dovner, Luke Neeson, Chris Story, Kimberly Johnson, Benjamin Galbert, Jessica Nowicki, Linda Brendan, Nicholas Carlson, Ismael Bensleyman. George Ekoriati, Valentine Steinmann, Per Crawley, Kate Van Goler, Alexander Obert, Liam Dunaway, BR, Massoud Ali Mohammadi, Perpendicular, Janus Hetner, Ursula Guinov, Gregory Hastings, David Pinsov, Sean Nelson, Mike Levin, and Jos Necht. A special thanks to my producers Iar Webb, Jim Frank Lucas Stink, Tom Vanneden, Bernardine Curtis Dixon, Benedict Mueller, Thomas Trumbull, Catherine and Patrick Tobin, John Carlomon Negro, Al Nick Cortiz and Nick Golden, and to my executive producers, Matthew Lavender, Sergio Quadrian, Bogdan Kanis, and Rosie. Thank you for all.