You Can. But Should You? | AI and Ethics with Dr Simon Longstaff

0:00 Georgie Healy: There is a problem at the moment with AI, at least in the systems we can currently deploy. We do not actually know how they are achieving the outcomes which they bring about. Can does not imply should. The fact that you can do something does not mean that you should do it. Any moment that you stop and ask, well, what should I do? Which choice should I make? Well, you're doing ethics.

0:22 Dr Simon Longstaff: Elon Musk has taken OpenAI to court.

0:25 Georgie Healy: For robbing a charity or whatever?

0:26 Dr Simon Longstaff: For robbing a charity. He gave them $38 million when OpenAI was supposed to be non-for-profit and then Sam Altman made it for-profit, and that is what he's in court for. You mentioned before that law and ethics don't necessarily have to be one and the same. Was that behavior ethical?

0:44 Georgie Healy: Not only does law not always align with ethics, it doesn't always produce just outcomes, but that's another topic.

0:52 Dr Simon Longstaff: Look, if, Hello and welcome to In the Blink of AI. I'm Georgie Healy, and this is a special episode. We've been recording for a year and a half now. We've had the most amazing experts in AI, but frankly, we haven't talked about ethics before, and that's been of utmost importance to me, but I didn't want any guest. I wanted the right kind of guest to talk about ethics. And frankly, I got the best person in the country slash the world. I've got Dr. Simon Longstaff. He's the director of the Ethics Centre, where he's been working for nearly 35 years. And his story's amazing. He left school at 16 cleaning toilets on Groote Eylandt. He became a paramedic and a fire officer, but he was adopted by the Aboriginal elders who taught him to see patterns rather than discrete objects. Quite pertinent to this time. And he's still connected to that community 50 years later. We talk about everything. We talk about universal basic income. What could the world look like if we've got no office white-collar jobs? We talk about mythos level capability, what keeps him up at night. And frankly, this is the kind of episode I don't want you to multitask while you listen. I get it. I— that's how I listen to podcasts. But he's a storyteller and he is worth really listening to carefully. I think this episode will stay with you for quite some time. He's a really special kind of guest, so carve out the time. It will be worth it. Let's dive in.

2:33 Georgie Healy: You're listening to a Day One FM show.

2:39 Speaker C: Founders scale faster on Deel. Set up payroll for any country in minutes. Hire anyone anywhere. Get visas handled fast and get back to building. Visit deel.com/dayone. That's D-E-E-L.com/dayone.

2:57 Dr Simon Longstaff: The topic of ethics and technology is incredibly important, but I've never had the right guests to talk about it, frankly. You are the executive director at the Ethics Centre, where you've worked for almost 35 years, correct? Incredible.

3:14 Georgie Healy: Very boring CV.

3:15 Dr Simon Longstaff: It's, it's an incredible tenure. And you've got a doctorate from the University of Cambridge. You have an Order of Australia. I am so excited for this chat. But first, I'm keen to understand you a little bit better. I read, unless my LLM was hallucinating, that you left school at 16.

3:34 Georgie Healy: Correct.

3:34 Dr Simon Longstaff: Started out cleaning toilets in the Northern Territory?

3:37 Georgie Healy: Yep.

3:38 Dr Simon Longstaff: Wow. Can you please tell me about that time?

3:42 Georgie Healy: Well, um, my mum had died when I was 7 and I ended up in boarding school and that was very expensive for my family to pay the fees. And so at, you know, when the time came for me to leave school when I was 16, I was told that I had to understand that everything my family was going to give to me, they had already given to me in the form of my education. And so I was now on my own to do whatever I wanted to do with my life with that as a foundation. So I needed to earn some money and I didn't have any skills. I'd worked school holidays and things like that, just, you know, knocking around family farms and things like that as a jackaroo, learning how to build fences and, you know, sweep a board in a shearing shed and plow and all that kind of stuff. At a time, and in one of the properties, even when it was before electrification where the ironing was still done with flat irons on a wood-fired agra and the donkey was lit in the morning to produce hot water in their kero fridges and things like that. I'd seen a really interesting part of a transition in life. And without any skills, I thought, where can I earn some money? 'Cause I hope one day to go to uni. And so I was able to get a job as what was called a service attendant up on Groote Eiland in the Gulf of Carpentaria in the mid-'70s, a very remote place. And my job was to clean toilets and to empty bins where you might have some old stale milkshake and prawn combo, which was—

5:18 Speaker C: Hmm.

5:18 Georgie Healy: And clouds of mosquitoes and things like that. Things like that. Anyway, I—

5:21 Dr Simon Longstaff: Free heat.

5:22 Georgie Healy: Yeah, yeah, I—

5:23 Dr Simon Longstaff: Good combination.

5:24 Georgie Healy: Anyway, eventually I studied all of the advanced issues around becoming a paramedic and also trained as a fire officer. So I moved from cleaning into emergency services. And being as young as I was, I had to get a driver's license. So I did my driver's license in a red fire truck. I drove ambulances. I had my first encounter of death under my own hands with someone whose life we couldn't save because we didn't always let's have a doctor on the island. And moved into health and safety for the remaining period of my time on Groote Eylandt. Plus, most significantly, I was adopted by one of the clans in the Anendilyakwa people. There are 14 clans, of which one is the Lalara clan. And I was adopted and have my own indigenous name and songline and all the rest, which, and I'm still connected to the island. I go up there, be there again in a month or so. Over 50 years of connection now.

6:24 Dr Simon Longstaff: That's such an incredible story. Being at the Ethics Centre for 35 years, how much were those years so formative to you pursuing something like that, do you think?

6:37 Georgie Healy: The cleaning and stuff? Yeah. Incredibly formative. I mean, firstly, I had some of the best conversations I've had with people about, you know, big issues. Did not actually take place in Cambridge. Some of them took place on Groote Eiland with people who were fitters and turners and truck drivers and people in mining and things like that who didn't have anything like even the education I'd had at that time, which was just limited to secondary education. And so I learned that actually people who you might dismiss as not being especially intellectual in inverted commas nonetheless were fascinated by ideas and had strong views about the world. And that's a very grounding experience. And you know, I still say to people who ask, well, how would I get to do what you do? They think I'm gonna say, oh, well, you need to go to Cambridge and get your PhD or equivalent. I don't. I say, no, go and work in an abattoir or a mine or do something that gives you that practical grounding. But the other thing that happened, which I didn't realize for years and years, occurred at the end of a wharf on the shiploading port in Aliangula, which is where the manganese ore would be shipped off around the world. And the local Aboriginal people who, those could become my family and friends, said, "Well, we're going to take you out there. We want to teach you something." And I thought, "Oh, okay, fair enough." So we go to the end of the wharf and they say, "Now we're gonna teach you how to see things." And I say, "Well, I'm tall." I didn't wear glasses and I had pretty good vision. [LAUGHTER] They said, "Now can you look down there and can you see?" the dolphins. And I said, and I looked and I couldn't see the dolphins. And then they said, hey, you know, Bungee, look down there, you're gonna see the dolphins. And I looked again, I still couldn't see. And then they start laughing and there's a really kind of warm distinctive way that this mob used to laugh at things, you know, really like a belly laugh. And they laugh and they say, oh, you know, the problem is you are trying to see the dolphins. Now I'm starting to get a little bit frustrated now 'cause I'm saying, yeah, you told me to see, look for the dolphins and now you're telling me the mistake is I'm looking for the dolphins. And that's when they explained to me that their view of the world is not one in which there are objects, say a cup on a table or a dolphin in the sea where you're looking for discrete objects in relationship to each other. They said, no, how you see is that you need to see the patterns that are made in the world. So there's a pattern that the ocean makes without a dolphin in it. And then when the dolphins come in, they change that pattern in a way which is distinctively the pattern of dolphins in the sea. Mm-hmm. And likewise, I learned that when you look at the world, not to look at it as a series of discrete objects, but as these patterns. Anyway, that's something I, a lesson I took on and I started to learn how to do it and various other bits and pieces they taught me. And then later on, I was doing this interview with Richard Fidler on The Conversation Hour, and I was saying how one of the great things about the work I've been doing at the Ethics Center is that you move across this spectrum of issues from preparing soldiers to go to war at one end to end-of-life decisions in hospitals at another and everything in between. And I said, "For me, it's all about the patterns." And then I suddenly thought, "Wow, I've carried with me throughout the whole of my professional career as a philosopher, this ability to see the patterns in things." And that was what I was taught at the end of the shiploading wharf in Alangula.

10:14 Dr Simon Longstaff: Absolutely incredible story. I feel like I've already got a sense for what drives you and what you care about.

10:27 Speaker C: Founders scale faster on Deel. Set up payroll for any country in minutes, hire anyone anywhere, and get visas handled fast so you stay focused on scaling. Deel takes care of onboarding, HR, IT, EOR, benefits, and compliance, so your team can grow without borders. It's why more than 40,000 fast-growing companies trust Deel to move fast. Visit deel.com/dayone. That's d-e-e-l.com/dayone.

10:57 Dr Simon Longstaff: Randomly, and not to make everything about AI, Simon, but one of the reasons AI in healthcare apparently is brilliant is because it helps see patterns that humans don't see. Have you heard about this?

11:16 Georgie Healy: No, absolutely. In fact, I spend a lot of time in, in my work around AI, uh, working with people in the medical profession. In fact, I'll be speaking to a group of orthopedic surgeons in New Zealand later this year, and I spoke to some in America last year 'cause they're very interested in this. And of course, what we know is that its most effective use in healthcare at the moment has probably been in the area of pattern recognition around things like pathology where with an accuracy greater than any human eye can achieve. It can spot the early signs of cancer in tissue and things of that kind. So yes, it's, it's a lot of it's about pattern recognition there as well. But there's a, there's a fascinating element to it in that the pattern recognition that's being done within AI, at least as it's currently constructed, is through digital recognition. So you have to digitize every image in order to produce something which is able to be analyzed and the patterns recognized. And yet the actual world we live in is an analog world. So the difference is best captured by the idea of that a sine wave in the analog world is a perfectly smooth curve. When you digitize a sine wave, it is rendered into bits, which means there are jagged edges.

12:32 Dr Simon Longstaff: Mm-hmm.

12:33 Georgie Healy: No matter how fine it is, It's like a problem of an asymptote. You get closer and closer, but you never reach the edge because there's always an edge, a jagged bit. And so there's something different about these two worlds at the moment, at least as long as one remains digital and we remain in an analog world. And I think that's a very interesting question as to what amounts, what it amounts to in that difference.

12:57 Dr Simon Longstaff: Absolutely fascinating. I'm going to get more pointed on some specific AI-related questions, but—

13:05 Georgie Healy: Go for it.

13:06 Dr Simon Longstaff: Before then, a little bit naive, I'm in uncharted territory with the ethics space here. How does someone become an expert in ethics? The people that are like, "Oh, I want to be like you.

13:18 Georgie Healy: How?" I'm not sure you ever become an expert. I think you've got to maintain a real sense of humility and curiosity. Partly because in ethics, it starts off with a premise, at least in the Western tradition, that the unexamined life is not worth living. That's a very big claim about what it means to be a human, or at least to participate in that form of being which we call human being. And it says we have this remarkable ability to go beyond just what instinct or desire might drive us to do through our animal nature. We've got a little bit extra which allows us to say, no, we won't just be driven by that. We can actually stop and think and make conscious choices. So this notion of examining life is a key requirement in ethics, which is in a sense what distinguishes it from morality. You can live a moral life, which is one where you might just habitually apply a set of values and principles in ways that have been taught to you by your parents or where you've grown up within a community, whether it's a faith group or just a general society.

14:25 Speaker C: Mm-hmm.

14:26 Georgie Healy: And so you might learn as a matter of habit to be truthful or as a matter of habit to be generous and so forth. When people ask you, why do you do that? Oh, that's just what we do. That's what I've been raised to do. Whereas an ethical life as opposed to a moral life is by definition examined. You're always thinking about it. You're always trying to make sure that you have the right values and principles, applying them in a proper manner to the things that you do. And so to become expert, is in a sense to learn to see through the different lenses which societies in the West and the East have developed over thousands of years of trying to understand choice. I mean, the most powerful force on this planet is human choice. Everything could be different but for the choices we make, limited only by the laws of physics. And so philosophers have asked, well, what lies behind it? And ethics is the— is the process of trying to answer what is the deepest structure that gives rise to human choice and therefore the way we come to make the world. And that's what you learn to do when you study ethics. You learn to understand the structure of choice, the various arguments that have been made about what is the right way to decide in favor or against a particular question. And so you learn about different perspectives on the world. You learn about a kind of intellectual history of thinking about these things. You learn—

15:51 Dr Simon Longstaff: Mm-hmm.

15:51 Georgie Healy: Some of the techniques by which you recognize the patterns I mentioned before that occur in different areas of life and how they might be applied in one place and then usefully used in another. These are all sorts of things you do. And so some of it's through study, some of it's through experience, some of it's through your own process of reflection and refinement over time.

16:11 Dr Simon Longstaff: I don't know if it's because I'm getting older or because I have children now or because it's such a fast-moving time in technology, Feels like a revolution of sorts, that it seems to be a topic, ethics in general, that I just keep hearing about, having presented to me, it's in the headlines. Have you noticed this or are you too close to it? You're like, I always notice it.

16:36 Georgie Healy: So I do notice it, but I also worry that often it's treated as a non-serious addition rather than actually a fundamental set of questions.

16:46 Dr Simon Longstaff: Like an optional—

16:47 Georgie Healy: Yeah, an optional extra. Or something you bolt onto life. Whereas for me, I sincerely believe that it lies at the heart of everything. Whenever you ask, if you ask yourself, might be something like just you go to the supermarket and you're standing there looking at which eggs to buy, you know, do you buy the eggs which are slightly cheaper from birds that live in cages or the slightly more expensive free-range ones? If you stop and ask yourself, which one should I buy? You're doing ethics. Any moment that you stop and ask, well, what should I do? Which choice should I make? From the clothes you choose to wear in the morning through to the technology you employ, you are doing ethics. And so when it comes to these really transformational changes, such as being introduced by modern new forms of technology, often they'll bring up older questions in a new form. But you can ask, What is the purpose of this thing? Is it a good purpose? Is it something which should exist for this purpose? And then you can also, if I agree this is the purpose for which it can be used, how should it be used? What are the limitations? And hopefully when people raise these ethical questions against that background, they're not doing it just, oh well, maybe we'll take it as an optional extra side consideration, but actually understand it goes to the heart of the choices they're making.

18:08 Dr Simon Longstaff: So beautifully said. I am so proud that my household is free-range egg only.

18:15 Georgie Healy: All right, there you go.

18:16 Dr Simon Longstaff: I had pet chickens growing up, and so it made it very clear to me why that was important. But for my husband, who had never had chickens before, I'm sure it just wasn't so obvious to him. And look, I have so many questions to ask you. High-level clarification before I get into that. Does the law enforce ethics, or are they two distinct things? Or like, how— where does the law and ethics come from?

18:43 Georgie Healy: So they are different, but they can be related. So we know, for example, that there are unjust laws. So in South Africa, they had the laws of apartheid, and we had similar laws in Victoria in Australia, anti-miscegenation laws. Anyone looking at those then and now could have told that they were unjust laws. But when you say that they are unjust, you're making an ethical claim about their deficiency. Other laws such as those which condemn one, criminalize murder, take an ethical insight about the intrinsic right to life and they say, well, if you violate that right to life, there'll be a penalty enforced by the state. And the main purpose of law in those situations in which we all stand equal before the law is to say we don't want a world in which every individual person or group or clan or family is taking the law into their own hands where it becomes a contest about who is the strongest or wealthiest or whatever. Actually, we want to have a situation in which everything is evened out and an impartial person in the form of the Crown in a case here in Australia or the state more generally will say, we will through, in our case, a democratic process define these laws. And in the criminal area, ensure that they are enforced. So you want to have laws which have a strong ethical foundation, and you want to have an ability to challenge and ideally repeal or remove those laws which are unethical, unjust in what they require. And of course, this is what we look to our legislators and our courts and others to do, is to ensure as far as possible that is the situation.

20:26 Dr Simon Longstaff: What is best practice then in this exceptionally fast-moving time, you know, Claude's mythos are being released. And I feel like the law is scrambling, and even ethically, I am scrambling to figure out where to stand. What do you suggest? What is best practice? How do we approach this?

20:49 Georgie Healy: Well, clearly societies have evolved to use a range of different mechanisms to try and manage situations, of which law is one. One tool. You also rely though, for the most part, on the good intentions and practices of people. I mean, the fact that we don't rob each other day to day in the streets of a place like Sydney is not because there is a policeman on every corner surveilling us and ready to intervene. Most of us walk around and we think, well, it'd be, well, it's unthinkable that we would rob each other. We recognize and respect each other's intrinsic dignity.

21:26 Speaker C: Mm-hmm.

21:27 Georgie Healy: Legitimate claim to the property we have, to be free from violence and so forth. So there's different elements there. At the moment with, stick with AI, come back to that at the moment, the law is certainly struggling to keep up and society is yet to have the very serious conversations that it needs to have about the ethical and political dimensions of this. My own view is that it's entirely possible to do these things though. I mean, I'm an optimist about it and I'm optimistic in a way that doesn't necessarily have to constrain or curtail innovation. Instead, I think that a really strong ethical foundation provides the platform on which one needs to build. So if you think about the artistry of the world's greatest ballet dancers, you know, They're moving freely, they're doing extraordinary things. It would be impossible for them to do that if they were trying to do it on a bed made out of jelly, you know?

22:28 Dr Simon Longstaff: Yeah.

22:29 Georgie Healy: They need a firm platform.

22:30 Dr Simon Longstaff: Yes.

22:31 Georgie Healy: So that they can then do all of the extraordinary things that they can do with their bodies. Likewise, I think there needs to be a really strong ethical foundation on which to build all of our AI and to have it do the remarkable things it can do. Without having what we've got at the moment, which is like jelly where no one quite knows. So where do you find the foundation? You find it in principles, general principles and approaches which allow you to address both what you aim to do and how you do it. So what is the purpose to be served and how do you go about it? And both of those have got potential answers in ethics. So one really simple principle is that can does not imply should. The fact that you can do something does not mean that you should do it. So even starting with that as a kind of a foundational idea, which I think most reasonable people would say, "Well, yeah, that makes sense to me. There's some things we really shouldn't do even if we could. Like I could rob a bank, but doesn't mean I should rob a bank." Yeah. I could do, you know, litter the streets. Doesn't mean I should. I might have the power to bring about some kind of damage to an institution or an individual. We shouldn't do it. So this distinction between what you can do. Now, a lot of people in tech say, "Oh, because I can do it, well, that means it's okay that I should do it." Well, no. One should think about the affordances which are created by technology. And how you can restrict the affordances to those which are reasonable. So to take a different technology, something like a handgun, yes, you can pick up, make a handgun, and at the moment, for most cases, anyone who picks it up can use it.

24:20 Dr Simon Longstaff: Mm.

24:21 Georgie Healy: As long as they know how to, you know, set the trigger and pull it and aim. Now they can use it perhaps for a legitimate purpose, or they could use it for some criminal intent. Imagine though that we introduced into the butt of a handgun a fingerprint reader so that only the person who is licensed to use this weapon can do so. And let's assume for the sake of argument that that is a responsible person who's been through training and other things. What we've done then is we've taken a handgun which has got unlimited affordances and we've restricted it using a technological means so to do. So thinking about the affordances that you create with the technology is something you can do. And the more you do that, you might say as a society, well, we've got a well-established framework, for example, to do with the release of pharmaceuticals. We don't just let anything loose.

25:14 Dr Simon Longstaff: Mm-hmm.

25:15 Georgie Healy: If it's going to have a potentially significant impact on people's health and wellbeing, what we'll do is we will evaluate it. We'll ensure that it is safe. And effective, that it does the thing it's supposed to do, and it does so in a way without too many adverse side effects. And then we'll say it can be released to the public at large. Well, we could think about AI in the same way. You might say that there are some things which are so basic, such as just general automation of routine tasks, where that doesn't matter. But if as a society we're thinking, well, how do we actually protect ourselves from the most extreme harms, like Claude Mythos, You might say, well, if it was clear as to what its intended purpose was and it really is effective and it's safe, then it can go through the gate. Now, none of this limits innovation because none of this is saying you must build into your AI a specific set of values or principles. Instead, what you might do instead is say, whatever you build, you must be able to tell us What are the core values and principles which are informing the design of this system? You must be able to show us how what you say you intend to do is present in the system that you have developed. You must be able to put it together in a way so that a disinterested third party can actually assure it to see that those things are still present. And you must make use of a tool like blockchain or others to ensure that the provenance that you have created is preserved over the life of its application. So I'll leave you as the technologist to work out which core values and which principles, that's entirely up to you. I will leave it to you as to how you give this implementation so that you are entirely free to do it in ways which might be novel and confer different advantages. But you must be able to tell me, that is society, those things so that we can reasonably rely upon the fact that it is doing what it's intending to do in ways which you have specified, which isn't being corrupted or changed. And in the more serious cases, it may have to go through some kind of approval process, such as we do with pharmaceuticals.

27:27 Dr Simon Longstaff: I love the framework. I love the technology provider, say it's, you know, Google's Gemini or OpenAI having the framework. What do you say and who is responsible when, you know, the model doesn't know what a deepfake is, it's just creating images based on patterns and based on, you know, skin tone goes with skin tone. Yeah. And then we saw on Grok, you know, users doing horrible deepfakes. Whose fault? Grok or the users?

27:59 Georgie Healy: Well, this comes back to that point I was making about affordances. I mean, you might ask it, and we'll take a physical case like the handgun I mentioned. Who's responsible for allowing the handgun to have so many affordances that a child can pick it up and accidentally kill another kid because they didn't have any limiting factor?— you'd say in that case the manufacturer shares at least part of the responsibility because they had available to them the means by which to limit those affordances. So if you know, for example, that an LLM is being loaded up with a whole lot of human biases which are going to be amplified, whether it's around things like skin tone or postcodes or any of the other things which are, you know, drawn— which cause these unintended but nonetheless predictable outcomes and you do nothing about it, then you are responsible. At the other end of the scale though, there is a problem at the moment with AI, at least in the systems we can currently deploy, in that we do not actually know how they are achieving the outcomes which they bring about. We know what the inputs were. We know roughly what the weightings were, if it's something with the transformer or other things in it, but— No one can precisely say what was the chain of reasoning or the process that specifically led to that outcome. So it's a bit like, unlike Boolean logic, where A gives B, but then for C and things of that kind. And that creates a really interesting problem for people who use these systems because if something goes wrong and you literally cannot know exactly what happened, then does that excuse you from responsibility? And the answer is no. What we have to do in relation to the use of AI is rely upon what I would describe as necessary fictions. So a necessary fiction in this case is one in which we must assume that you did know.

29:59 Dr Simon Longstaff: Mm-hmm.

30:00 Georgie Healy: Even though you couldn't know. Now, there's a precedent for this in government. In our system in Australia, we have a doctrine called ministerial responsibility, where if you are, I don't know, the Minister for Social Services or some other part of government, we know it's literally not possible for you to know what every single public servant is doing, what every single system is causing to happen, because it's just vastly too great a number of people or systems for any human being to be able to monitor. And so we say, we know it's impossible for you to know everything that's happening in your department. And yet we impose a necessary fiction that you as the minister are ultimately responsible for what goes wrong. And you'll be held accountable through the parliament to the people. Otherwise there's no way to hold power accountable in a democratic society.

30:57 Dr Simon Longstaff: You have to incentivize them to really take it seriously, I guess.

31:00 Georgie Healy: Well, you, yeah, I mean, whether it's a positive or a negative incentive, I mean, they're all volunteers by the way. No one— Yeah. No one is forced to be a minister in a government.

31:09 Dr Simon Longstaff: True.

31:09 Georgie Healy: You know what you're in for.

31:10 Dr Simon Longstaff: No one has a gun to their head.

31:10 Georgie Healy: Which includes being subjected to this necessary fiction. AI requires the same thing. What it requires is that if you are the CEO of a firm deploying AI and you're its user, to the extent that you have received a system where people have, the makers of it, if you like, have discharged all of their ethical obligations in the way that I outlined before, now you must live with the fact that you are ultimately responsible even though you cannot know what's happening inside the black box. And we as a society need to understand that. And we need to say, okay, if you're going to use these systems, you're going to have that degree of ignorance fundamental to the way the system operates, but you will still be responsible for what it produces.

31:51 Dr Simon Longstaff: I think you're highlighting very clearly that there is precedent for this. I think a lot of us in tech are like, oh, there's no precedent for this. How could we possibly navigate this time?

32:02 Georgie Healy: Nearly everything that's likely to happen has its precedent. Let me just take another example. This time, not so much in terms of the operation of the technology, in its immediate effect in terms of performing tasks and helping to support or make decisions. But in terms of what many people foresee as a likely displacement of people from employment, there's a big debate. I mean, you get people who are extreme dystopians and extreme utopians, but I think any reasonable person would have to say that lots of things which are currently done by people many times dangerous, difficult. Other works will be replaced by a stack of technology, which includes AI, robotics, and a whole lot of other things. So all this could be terrible. And I think of all these people who lose their jobs and, you know, and there's a real reason why you need to manage it because particularly if these are middle-class jobs in accounting and other areas, you know, clerical work, we know from history that every revolution that's ever troubled the world has been led by a disgruntled middle class. The French Revolution, the American Revolution, the Russian Revolution, they're all led by the middle class—

33:12 Dr Simon Longstaff: Yikes.

33:13 Georgie Healy: —who are feeling really annoyed about their circumstances. And they mobilize the working class to become their foot soldiers. So if it goes wrong, it goes very badly wrong, but it doesn't have to. And we know because there's a precedent for such a society in times when there was the pernicious institution of slavery. Ancient Athens was largely supported by a whole array of slaves. It wasn't as bad as say somewhere like Sparta, but this was a society where slaves were doing huge amounts of laborious work. Now you ask, did the Athenian citizen's life therefore become degraded and meaningless? And no, it didn't. Because what they then did, the Athenian citizen, was they spent their days doing extraordinarily meaningful work as citizens. They were going to the courts, they were legislating, they were doing things. Now, of course, it's a small scale, a city-state, not as big or complex as whole countries or societies. But it tells us that human beings have been in places before where rich and meaningful life was possible without jobs in the way we understand. Even on this continent prior to colonization, If you went into an indigenous society, there weren't people who had jobs in the sense that there was an employer who contracted with you to do certain work and then you were paid a wage and things like that. You had a life in which you spent some of the time providing for the means for survival, food and things like that, but you were also engaged in family activities, artistic endeavors, spiritual components to life. Mm-hmm. So you had this kind of really rich and meaningful life, but there wasn't a single job in the way that we understand it. So we'll need to evolve our understanding, say, okay, there'll be lots of work to be done in a society which has all of these machines and other systems, but there won't perhaps be as many jobs. So then you'd say, well, how do we sustain a society? And that raises questions about taxation and the distribution of wealth and a whole lot of other things. But the point is, it's not radically new for human beings to engage in questions about these things and to find solutions. And we should be optimistic that if we actually think about it in advance, talk it through in advance, then we can come up with some wonderful solutions which could enable a country like Australia to become the most just, prosperous society the world's probably ever seen.

35:42 Dr Simon Longstaff: My brain is exploding in the best possible way. I just never, you know—

35:46 Georgie Healy: A boom, a boom.

35:46 Dr Simon Longstaff: I went from full-time work all my career to just a little bit more flexible hours, and even that's kind of like hard for me to get my own head around because I'm so used to that structure. And so when ideas like, or propositions like universal basic income are proposed, I think absolute madness. How could that possibly work? Do you have a specific take on universal basic income as well?

36:13 Georgie Healy: Yeah, look, I think it's got to be rationally grounded, and there are ways to do that. So a fellow called Alan Schwartz and I worked for a number of years on this concept of the universal commons. And that is that if you think about typical companies, what they do is, say it's a manufacturing company or start with a farming company. So they say, well, if I want to grow potatoes in order to sell them into the marketplace for a profit, first of all, I need to get some land. So I either buy it or I pay rent for it. Then I need some people who are going to help me grow the land, grow the potatoes. So I'll have to pay wages and then, you know, there'll be a series of other costs and eventually I will I'll take all of those costs and hopefully sell my product for a profit, which I then get to keep. So it's the basic model. The trouble is with that model, for most of human history, all of the things which allow you to do that come at something less than their true value. So you are benefiting from the clean water, the clean air, the peaceful environment in which you operate, all of which are provided by society. And as the person who is seeking to profit most from that, you pay a relatively small amount by way of taxation. And so you might ask then, well, should there be a reasonable percentage of the profits in the form of tax or otherwise, not only going back into the system to maintain all of the things which allow your commercial activity to be possible, but also should a share of that be redistributed to the people who own these things, which are ultimately held in common? Because the great mineral resources, the clean air, the quality of society is produced by all of us. And so should there be a share in that? Now you're starting to think, well, maybe there is a basis in which a distribution of all of the incredible wealth that can be produced out of this society can be distributed, not just because you happen to be living there and don't have a job, but because you have some kind of reasonable and legitimate claim. on a share of the commons, which are being drawn upon. Now, seen in those terms, you might say, well, okay, maybe then we all contribute to this. We receive our share. That might be in the, we might call it the equivalent of a universal basic income. And then that allows us to do work, caring for people in the community, a whole lot of, I mean, you might have a situation where some people who do have jobs, they go off to the city, they do their work, if that's where they go, they come home. There are other people who've got You used to have chooks in the back garden.

38:49 Dr Simon Longstaff: Yeah.

38:49 Georgie Healy: You've got eggs and things. There might be somebody else in your suburb who's a great cook. There might be more of a barter exchange where money exchange and there's still value gain. So you can see, again, older forms, older ways of living being given a chance. Maybe suburbs take on a particular character of their own within this little small market economy based on the exchange of goods and things from people's labor. Supported by the vast productive capacity of a society which takes seriously what could be done with AI, robotics, and other things. Like lithium cycle. In Australia, we mine lithium at the moment, 1% of the value. Is it 1% or it's maybe up to 4% of its value now stays here. The rest goes off where it's processed and manufactured, partly because it's really dirty and dangerous work.

39:41 Dr Simon Longstaff: Mm.

39:42 Georgie Healy: But if you could automate it, if you could have it being done by machines and you retain all of that value back here, then it's available to be used for other purposes in support of a flourishing society.

39:52 Dr Simon Longstaff: I feel like with all this technology revolution, I assumed that we wouldn't have these communities and connections and we'd just be wired on our devices. But the picture you paint with this potential universal basic income structure seems, if anything, more collaborative than we've seen for many decades.

40:13 Georgie Healy: Well, it's kind of a non-accidental element of what AI does. So if you— we go back, you were talking about pattern recognition earlier on, and expert systems can do better than many pathologists. Now, you haven't seen a reduction in the number of pathologists. Robots will eventually be better able to perform certain forms of surgery.

40:33 Dr Simon Longstaff: Can I just tell you, I heard that someone who had— someone based in the US performed open-heart surgery on a little baby from Chicago remotely.

40:44 Georgie Healy: Yeah, and you probably won't. These could even be autonomous. And the reason being because with machine vision, it can see for, just for example, if you're excising cancer, you want to have as small a margin as possible between what's healthy tissue and the diseased tissue. Human eyes won't be able to see as well, won't have the same degree of dexterity. So does that mean the end of surgeons? They'll disappear? No. Why? Well, if I receive back the report from the pathologist, the system, let's say it's just a purely automated system, which says that you have got a terminal illness, that you only have months to live. Only another human being can put their hand on your shoulder and say, Georgie, I have some terrible news, where you know and I know what it means to be mortal.

41:31 Dr Simon Longstaff: Mm-hmm.

41:31 Georgie Healy: We understand what it means to be facing one's death in a way that no machine knows. Now, it may do a very good simulation of what it's like to put a hand on a shoulder with a consoling touch, but it doesn't know. There's a qualitative difference here.

41:48 Dr Simon Longstaff: Yes.

41:49 Georgie Healy: Part of what it means for us to live in this analog world with our own mortality and things like that. Now, that is just one example where the question that is begged by the expert system, the AI system, is, Well, what then is distinctive about our humanity that cannot be replicated? And this is gonna be a question asked of us in businesses and organizations of every kind. What is it distinctively human thing that we hold onto that we need to nurture? And I think it's that question and the answer that it produces around things like The difference between what, what, what, when, why do we have hand-cut chips on the menu?

42:32 Dr Simon Longstaff: Oh yeah.

42:34 Georgie Healy: Because we think, wow, somebody has, or artisanal beer or artisanal bread. There's something about the thing which bears the mark of its maker.

42:43 Dr Simon Longstaff: Yes.

42:43 Georgie Healy: The thumbprint in the old convict brick or the person who hand-cut the potato or whatever. That is non-accidental. I think, which only another person could do. I mean, I was in a conversation recently about using AI and surveillance systems for improving safety in the workplace. You could easily have a system which is able to look at a person who's about to put their hand near, say, a dangerous conveyor belt, say, "Stop," stops the belt and all the rest. What you get there is a functional outcome. The belt stops, the hand is saved from injury or damage. What you don't get is someone, if it was another human being, who says, "I care about you enough to tell you stop." And so the culture of an organization which achieved the functional outcome of safety without, just by through machines, it would progressively lose that attitude of care and reciprocity that comes when a person cares enough to say, "Hang on, I'm looking out for you." Mm-hmm. So I think these tools, which I think are fantastic, I'm a great supporter of AI. I want it to be used as widely as possible as a support. But we need to think then about what is distinctive of our humanity and talk about it and work it and recognize it and value it and support it and then see how it's worked through in the kind of societies we build.

44:11 Dr Simon Longstaff: I don't know if it's because I just haven't had breakfast yet, but if a robot tried to pat me on the shoulder if I got a horrific health scare, I think I would lose my mind.

44:24 Georgie Healy: Yeah. I mean, there's, I mean, the developments in robotics and AI at the moment, I mean, it's progressing at such a rate. It won't be that long until you'll have very convincing robots that, you know, will look and feel and sound. They'll have a, they'll look you in the eye as I am now and think, but they won't have that indefinable, maybe it is definable 'cause we are talking about it, element that comes with that notion of our mortality and what it means to be a fragile human.

44:56 Dr Simon Longstaff: So beautifully said. And I think we, like, even hearing it explained, that gut sense of, oh yes, that seems obvious, but you almost need those real case study examples to be like, oh, of course I would prefer a human to say they care or to high-five me if I achieve something. Or the like. I'm going to get a little bit trivial now, Simon.

45:21 Georgie Healy: Right. Are you going to get spicy too, you said?

45:22 Dr Simon Longstaff: Yeah.

45:23 Georgie Healy: All right. Okay.

45:24 Dr Simon Longstaff: Are you ready?

45:24 Georgie Healy: Yeah. Okay. Is this trivial or spicy we're going to?

45:27 Dr Simon Longstaff: Trivial, then spicy. Okay. Okay. Let's go with trivial. There are some headlines in AI at the moment. Number one is Elon Musk has taken OpenAI to court. I believe he's in court right now.

45:41 Georgie Healy: For robbing a charity or whatever?

45:42 Dr Simon Longstaff: For robbing a charity. He gave them $38 million was when OpenAI was supposed to be non-for-profit, and then Sam Altman made it for profit, and that is what he's in court for. You mentioned before that law and ethics don't necessarily have to be one and the same. Was that behavior ethical, to go from nonprofit to profit after getting that donation?

46:09 Georgie Healy: Not only does law not always align with ethics, it doesn't always produce just outcomes, but that's another —topic. Look, if Sam Altman and others accepted money on the basis that it was going to be applied for a charitable purpose, then there was a kind of covenant made with those who subscribed to do that, which is a presumption in favor of preserving the charity. Now, the problem here, the ethical problem is that when Altman Altmann sought those funds, it wasn't merely just to have a charity for the sake of a charity. It was to do a particular thing, namely to work towards developing AI. And I suspect that what happened was that Altman and co reached a point where the ability to invest in that purpose was severely circumscribed by it being a charity in that people said, well, there's only so many dollars, $38 million or whatever it is that we're going to give you. For a charitable purpose, whereas what your project requires is billions.

47:16 Dr Simon Longstaff: Mm.

47:17 Georgie Healy: And we will only give you billions if you are prepared to commercialize. So there's a dilemma for people at that point. And so often in ethics, the choice is not between good and bad, right and wrong. It's good versus good. It's right versus right. So there's the project that needs to be done. And then there's the commitment to do it in a charitable way. And it turns out that that is not possible. There's a maxim that comes from a German philosopher called Immanuel Kant that says, ought implies can. That is, you're not obliged to do that which is not possible. So if I said to you, look, unless you fly around this studio without any aid at all, just, you levitate. Unless that's the case, then you're a very bad person.

48:07 Dr Simon Longstaff: You're gonna say, "That's rubbish." Right.

48:09 Georgie Healy: I'm not a bad person because I can't do that. It's simply not possible. So there's no moral obligation upon me to do that which is impossible. And Altman might've said, "Well, I'm seeking to do this really important thing and it's not possible to do it as a charity." Therefore, I'm not morally obliged to continue with that. But does that mean he's completely off the ethical hook? No. So there's a couple of things. Firstly, he should have been really honest with all of those people who had donated money up front and said, look, I'm unable to realize the objective, the purpose as a charity. Will you consent to my changing it to a corporation or will you accept a return of your funds?

48:56 Dr Simon Longstaff: Ah.

48:57 Georgie Healy: Which can be applied to some other charitable endeavor. Secondly, he could also have said, and what I can promise is that the original purpose will be honored in its intent, which includes putting very significant constraints around what we do or don't do to ensure that our pursuit of this objective isn't driven by the need to generate a profit, but instead to secure the ultimate good that we originally claimed. In other words, we'll accept the profit and distribute the profit as a condition for funding, but we won't do it because of the profit.

49:34 Dr Simon Longstaff: I see.

49:34 Georgie Healy: So there's a subtle distinction here in terms of what he could have done. And I expect that if Altman had gone to Elon Musk and said, look, Elon, I still believe with a passion in thing that you agreed to fund. I can't, we can't do it like this. I'd like your approval, your consent to change to it. And if need be, I'll give you your money back. But I also, whatever we do, it won't just be for profit. It's going to be, in other words, the money will be a means to an end, not an end in itself. Then I don't think you'd be in court today.

50:08 Dr Simon Longstaff: Yeah.

50:09 Georgie Healy: But you need to think it all through like that.

50:11 Dr Simon Longstaff: Yes.

50:12 Georgie Healy: To to get to the point where you've got a path ahead.

50:16 Dr Simon Longstaff: And then if, you know, he had given the money back and Elon was just salty that this success story didn't have his name affiliated with it anymore, then we would know. It's just—

50:27 Georgie Healy: It's Elon's choice. But it would've been an honorable process. Now, I don't know, maybe that all happened as far as I— and that this is just a grievance that Elon Musk harbors even still. But in the absence of knowing, that's generally how I would have done it.

50:46 Dr Simon Longstaff: Okay, here's another headline for you. In March, the Pentagon argued that domestic surveillance is fine because it's legal, Simon. It's, you know, it's legal.

50:58 Georgie Healy: Anthropic. Yeah, that's the Anthropic issue. Yeah.

50:59 Dr Simon Longstaff: Yes. Just to remind the listeners, Anthropic were partnered with the Department of Defense and then refused to agree to use their models for autonomous warfare, essentially.

51:13 Georgie Healy: And you can understand now that we know about Mythos why Anthropic was not prepared to give the kind of blank check to the Pentagon, remembering that Mythos could have been used as an extremely offensive weapon in order to find and exploit vulnerabilities in any system. That may or may not have been lawful. Even if it was lawful, the idea that a weapon of such power could be used, and remember that weapon doesn't have its affordances constrained. It's like the gun with the— No thinking about it. No, it can be used to attack anything.

51:56 Dr Simon Longstaff: Hospital.

51:57 Georgie Healy: Civilian infrastructure, whatever. And the Pentagon at present in ways that some have argued is not lawful. So, you know, in breach of the laws of warfare, such as the attacks made on boats in the Caribbean and the destruction of people who were clinging onto wreckage. There are a number of other areas where the law says there's a high question to be answered. If you then give them an unlimited power, you would say that's not a responsible thing to do, particularly when those in charge of the Pentagon are saying, you must give us this to use without restriction. Now, it may be that other suppliers who are making available to the Pentagon different tools might say, the tools we provide lack the kind of power that Mythos would've released. And even if they use it within the strict American administration's interpretation of law, that's something we can live with. But you'd hope that everybody else, and not just Anthropic, are thinking carefully through the ethics of their engagement in providing services, goods and services to a department which may potentially be found to have used it in violation of international law, which the US doesn't recognize to the extent it might, but even their own domestic laws.

53:31 Dr Simon Longstaff: I wanna speak to you for another 3 hours. We're just about to get to the rapid-fire spicy questions.

53:37 Georgie Healy: Okay.

53:38 Dr Simon Longstaff: You've been looking forward to, but before that, do you have an LLM of choice, Simon? Do you choose a specific model?

53:45 Georgie Healy: Yes. I've been enjoying using Claude. I'm starting to use it more deeply. I use it for a range of different functions, and I also subscribe to another one, which gives me a choice of systems.

53:57 Speaker C: Yep.

53:58 Georgie Healy: Where I can use multiple ones if I wanted to. I haven't really made as much use of that because I've been exploring what Claude can and can't do, at least in support of my work. I haven't yet got to the point of having it ingest everything I've ever written or said or done in order to create an avatar, partly because I think although it may functionally perform as well, or if not better than me, the personal component of being engaged for the reasons we were discussing before is important. But that's, that's where I am at the moment.

54:31 Dr Simon Longstaff: I'd rather speak to you than your avatar.

54:33 Georgie Healy: How are you? How are you? Have you got one that you choose?

54:35 Dr Simon Longstaff: I actually have 3, which is ridiculous.

54:37 Georgie Healy: For different purposes or?

54:39 Dr Simon Longstaff: I started with ChatGPT, like a lot of the Australian ecosystem. Switched over to Claude mostly because of this Pentagon-related stuff.

54:48 Georgie Healy: Yeah, that factored in my thinking.

54:51 Dr Simon Longstaff: Yeah. And I'm an ex-Googler, so Gemini—

54:54 Georgie Healy: Right.

54:54 Dr Simon Longstaff: —is incorporated into a lot of what I do, and I'm already on a pro tier, but I use Claude probably 90% of my time. Even if it is great to have the same prompt across 3 tools, who has the time? Like, maybe that's—

55:08 Georgie Healy: Well, it's interesting stuff. I mean, The relationship that's developing between Microsoft and Anthropic around having two systems both checking each other.

55:18 Dr Simon Longstaff: Yeah.

55:19 Georgie Healy: This is a really interesting idea. And any of the orchestration layers, which sort of get different systems doing that, one researching, one checking, confirming, it takes away so much of the risk of hallucinations and other things that come with it. It may not be perfect, but it's gonna be so much better than it has been. And that cooperative environment is, I think, a really positive development.

55:41 Dr Simon Longstaff: I have a conspiracy theory that Anthropic didn't expect to have this much demand, and so rate limits are getting squeezed.

55:50 Georgie Healy: Yeah.

55:50 Dr Simon Longstaff: And sometimes for questions that aren't, like, are a little bit more trivial, like, help me edit this LinkedIn post, I'll use a ChatGPT or a Gemini just to not use up those rate limits. Rapid fire.

56:01 Georgie Healy: Okay.

56:02 Dr Simon Longstaff: 15-second answers if you can, but you don't have to. Or shorter. I won't cut you off because I would like to hear your answers. Big picture, what keeps you up at night as an ethics expert that we might not have on our radar?

56:17 Georgie Healy: The lack of discussion about what kind of society we want to have when we harness AI to its greatest potential.

56:22 Dr Simon Longstaff: If you could wave a magic wand and change one thing about how AI is being deployed or developed, what would that be?

56:31 Georgie Healy: I would educate the people who are building it to understand core ethical frameworks so that they can build it into what they construct.

56:39 Dr Simon Longstaff: Are you going to be purchasing a bunker? No. Could be fun, right? Does it mean anything when big tech companies say they're prioritizing safety and ethical practices?

56:52 Georgie Healy: It means something if it was true. And I suspect that it's often made as a glib expression without fully understanding what is implied by that commitment.

57:03 Dr Simon Longstaff: What is the most unexpected joy of living amongst the manganese mines in the Groote region?

57:11 Georgie Healy: It was my relationship, which continues to this day, with the indigenous people of Groote Eiland.

57:16 Dr Simon Longstaff: How special. And last question, if you have one piece of advice for the listeners, they're in tech, they're in startups, but they're also increasingly people just navigating this time in technology about ethical AI use, what would you tell them?

57:33 Georgie Healy: Look for an organization that can tell you when you ask, what are the underlying values and principles embedded in the system? And how they monitor whether it's preserved or not. If they can't tell you, then find someone who can.

57:47 Dr Simon Longstaff: This has been such a joy. Thank you so much. Uh, Dr. Simon, how can people find your work? Where can they follow you? And yeah, take it away.

57:57 Georgie Healy: Well, they can find me in all of the usual places, LinkedIn and things like that, where I tend to be more active. I don't post a lot because I only feel like doing so when I have something to say rather than just keeping up a regular stream. So I'm a bit of a disappointment on that front. But you can also go to the Ethics Centre's website, which is www.ethics.org.au, and there's a range of material there. And then if you really want to engage in ways which will stretch your brain and look out for later this year in August when we'll be presenting the latest in the Festivals of Dangerous Ideas, which is something that we put on.

58:33 Dr Simon Longstaff: Oh, I adore that. Is that at the Sydney Opera House?

58:35 Georgie Healy: No, it's now at Carriageworks, the Town Hall, and a number of other venues around the city. Amazing.

58:40 Dr Simon Longstaff: Yeah. Thank you so much. This has been an absolute joy.

58:43 Georgie Healy: Well, thanks for having me.

58:46 Dr Simon Longstaff: Thank you for listening to In the Blink of AI. You can check out the the show notes for anything discussed in this week's episode, and we will be back next week. This podcast was produced by Day One with music by Dan Hansen and visual artwork by Sophie Tyrell. If you loved the episode, please tell your mates, and I love AI news. Please share your thoughts and suggestions to georginarosehealy@gmail.com.