Responsible AI - AI with a Conscience | Episode 4 | Learning Kerv | Learning Kerv

Description

In this episode, we tackle some of the most pressing questions surrounding the ethical, privacy, and security concerns of AI systems. From safeguarding your personal data to understanding the environmental impact of generative AI, we explore the steps individuals and organisations must take to ensure AI is developed and used responsibly.

Whether you're a tech enthusiast, a data privacy advocate, or someone curious about the future of AI, this episode provides valuable insights on how to navigate the complex landscape of Responsible AI.

Key Highlights:

Security vs Data Privacy: Why It Matters: Understand the key differences between security (protecting systems from threats) and data privacy (safeguarding personal information). Learn why both are crucial when developing and using AI systems.
Concerns for Your Private Life: Explore how AI may impact your privacy, from data collection to digital tracking, and what you should be aware of to protect your personal information in an AI-driven world.
Security Risks in AI: Delve into the major security concerns surrounding AI, including vulnerabilities in AI models, data breaches, and how malicious actors could exploit AI systems.
Relevant Regulations: Learn about current laws and regulations governing AI, data privacy, and security. We discuss frameworks like GDPR and emerging policies to ensure AI development remains ethical and compliant.
Addressing AI’s Environmental Impact: Discover the environmental challenges posed by Generative AI, such as high energy consumption and carbon emissions, and explore ways organisations and developers can reduce AI’s ecological footprint.
Ensuring Adherence to Ethical Guidelines: Find out how organisations and individuals can ensure adherence to responsible AI practices, including transparency, fairness, and bias mitigation, to keep AI systems aligned with ethical standards.

Tune in to this episode on Responsible AI to learn how to navigate the ethical, privacy, and security challenges of AI.

Hosted on Ausha. See ausha.co/privacy-policy for more information.

Transcription

Rufus Grig
Hello and welcome to The Learning Curve, the podcast from Curve that delves into the latest developments in information technology and explores how organisations can put them to work for the good of their people, their customers, society and the planet. My name is Rufus Grigg and in this series, with the help of some very special guests, we're looking into all things generative AI. Now the world's massively excited about Gen AI and most of this series has been about the technology itself and its applications. But with a few episodes under our belts, we wanted to... pause and consider some of the risks, the challenges and the cautions associated with the tech and its expected widespread adoption. So today we're going to focus on four particular aspects. We're going to look at security, privacy, sustainability and ethics. Perhaps all of them could be taken together and described as responsible AI. And for that, I'm joined by two more of my brilliant Curve colleagues. Firstly, I've got Tony Leary. Tony is Curve's Chief Information Security Officer or CISO. How are you doing, Tony?
Tony Leary
Well, yeah, thank you, Rufus.
Rufus Grig
Good. Thank you very much for joining us. And I'm also joined by Francis Thomas, who is our Chief Sustainability Officer. How's things, Fran?
Fran Thomas
Yeah, good. Thanks, Rufus.
Rufus Grig
Brilliant. Thank you. So let's dive straight in with the security and privacy piece. Security and privacy are obviously very closely linked. But Tony, can you sort of perhaps unpick what the difference is? Is there an important distinction and how we need to think about each one?
Tony Leary
Sure. I mean, sometimes they're used interchangeably, I think, but they are different, for sure. I think most people know what information security is. It's about keeping data safe. I think most people have that concept of what it is. And for the most part, we rely on other companies, even as individuals who rely on companies, whether it's Curve, Microsoft, Google, to protect data. And ultimately, that's what security means for most people. Privacy, on the other hand, it relates to really surround choice and our ability to exercise choices on how data is used and processed. So that could be that you want one company to take a particular action with data, but another not to. And also the right to change your mind about that as well. And data protection regulations, GDPR, of course, from a UK perspective, we follow GDPR. And that's put that whole concept of choice and power really for individuals into law. And I think just one further point about the relationship between the two is that these days, I think it is linked very closely and effectively, they're mutually dependent. dependent on each other. But I'm not sure that was always the case. We're probably old enough to have been using the internet for 30 years in my case. And certainly, if a system is privacy supporting, it has to be secure. I think that direction of relationship is definitely always there. But you could certainly have a secure system that doesn't have privacy, a public system. But I think these days, even public sites, whether it's Facebook, even X and others, have privacy policies and have some level of... privacy enforcing controls. These days, I think we can say they're very closely linked together.
Rufus Grig
Okay, thank you. Interesting. And some very high profile companies there that have been never quite that far from controversy over potentially some of those privacy issues. So let's start with privacy then. In an enterprise context, thinking about AI and generative AI in particular, what are the main concerns?
Tony Leary
I think it's people using public tools. So these platforms like ChatGDP in particular, There's huge amounts of publicity when it was launched. I think even now there's 100 million people also registered for that service. A lot less now using it regularly, but certainly a lot of people have got access to it. And lots of people saw some benefit from using it, not just students cheating at their homework, but people in business, whether it's helping to write things. We've all used it. And these tools are actually quite good just for first drafts and things. So I think they've had a lot of take up there, but there's always a risk that you share that. data about yourself or about Curve as a company or a customer even with a service. And then that services privacy policy allows them to use that data in a model. An example of that is someone putting data into ChatGDP, might be corporate data. They may be doing so for them for valid reasons to use ChatGDP to draw some insights about data. But that then may be used by OpenAI in their model. And let's say a competitor was just asking a really broad question about Curve, that data may be returned and that data is confidential. And certainly if you're on a free version of ChatGDP, their privacy policy allows them to reuse data in their model. So that's absolutely a concern for any enterprise and even individuals, really, in terms of what you share with these services.
Rufus Grig
So if I just get that clear, I mean, there were certainly some examples early on when ChatGPT first sort of became really big in the public consciousness of, say, be a customer service agent using ChatGPT to say, please, here's the details about this customer. Please write me an email that I can respond. The concern, I guess, is that I've said, here's Tony, here's his financial situation, here's his medical problems. Please give me an answer. And someone else then gets served up that data because the model has trained on it and ingested it and learned it. Is that right?
Tony Leary
Indeed. Yeah, that's a real risk. Yeah. And I think, as you say, there's been examples of that actually happening. And I think more generally, just from a privacy standpoint with these systems, they're very hard to test. And the examples we've mentioned, that's almost people finding the edges and the flaws in these services once they're live, which it was ever thus, I guess, with a lot of free internet services. But these systems are particularly dangerous from a privacy perspective, just because of the amount of data they're aggregating. And... the ability for people just to craft responses and get data out of them. And I think that's just overall, it's just always a challenge with these large language models, that ability for companies really to properly test them. And no doubt as time moves on, their tools will improve. But right now, I think there's some real risks there. And also just geography. So as mentioned, I mentioned the GDPR, that's really strict about where data is processed. And it could be, for example, we may have customers that obligate us to process data in the UK, but certain services, they may well be hosted in the EU or the UK, but AI services may be happening somewhere else. They may outsource it to someone like OpenAI, that data may be going off to the US, for example. And that would be a breach from a GDPR perspective in terms of an obligation to understand where data's going and particularly around offshore processing. So lots of concerns around that. with privacy and certainly with anything that's public. I think that's where I'd start.
Rufus Grig
Okay. So you've certainly scared us a little bit in terms of the potential for things that can happen. So what can organisations do? What sort of guardrails should they put in place? I guess, you know, it's fairly easy to say to your staff, don't use ChatGPT, but how do you manage that around the services that you talked about just now, where, you know, a service that the company is perfectly happy using suddenly bolts on a fancy new GenAI service. that happens to do its processing in the States or somewhere else. How does an organization keep hold of these things?
Tony Leary
I think it's training, education to start with. But I guess from a sort of a backend perspective, it's around inventory. It's really just understanding what systems you're using and where these things, if there are AI services being used, where is that processing happening? For any organization that's got an information security management system, this type of approach is completely normal. No, it's about knowing where your information assets are. and measuring risk, managing risk around those things. So there isn't really anything that new with these sorts of services, but I think it's more the genie out of a bottle issue, I suspect, with users using things like ChatGDP in their, I guess, let's say their personal life, finding them useful and wanting to use them for corporate work. And I think in that case, it's just making sure they've got an alternative, which within Curve, everyone would have access to Copilot, Microsoft Copilot. So that is effective. effectively a version of ChatGP that they can use to get similar results you would expect.
Rufus Grig
Okay. And how do we get hold of that copilot? Because I think obviously Microsoft has a lot of different copilots and there's also some reassuringly expensive ones. I guess if you're talking about a ChatGPT equivalent, how would they access that?
Tony Leary
Yep. So anyone that's got a Microsoft license even can go to copilot.microsoft.com and they'll get some view of, I guess it's a version of Bing effectively. But it's a version of Copilot that works across the internet. Some users within Curva are piloting Microsoft Copilot, the M365 Copilot version. And that gives you, I guess, a switch where you can choose to search the web or internally. So there's a couple of different approaches depending on what license you have. But certainly, if you go to that website, it'll be obvious what you're able to do.
Rufus Grig
Okay, thank you. So that's good advice for enterprises, though, and how they sort of start worrying about that. What about for anyone listening who's concerned about... privacy in their private lives. Any particular guidance there?
Tony Leary
Well, yes. Yeah, lots. Probably too much for the time we have today. But I mean, ultimately, whatever you share online, it has some level of risk. And certainly if it's in any sort of public forum, cookies are quite an old school thing. But I still personally worry or look at cookie banners when they come up. And I saw one earlier today and the site or this particular cookie management system this site had. said that they shared data with 1,568 different companies. all of which are gathering data about how you're using that site. And it's all going off, being aggregated, it's being traded, it's being shared with other companies that want to sell to you. But ultimately, there's a profile of all of us online, and it's really hard to avoid sometimes these types of cookies and what's being shared. And a lot of sites, not normally in the EU or the UK due to GDPR, but certainly elsewhere in the world, cookies always feel not optional. put it that way. And there's this idea or this concept of dark patterns, how the sites are designed to basically to make it very difficult to disable things like cookies.
Rufus Grig
Does generative AI in particular enhance that risk? Are there services that people are using that they might not otherwise have used? Does the generative AI in itself create tools to people who want to do bad things with your data? Does it make that easier? I mean, I'm just thinking specifically about what the implications of Gen AI's availability are on that personal privacy.
Tony Leary
I think so. Just again, it comes back to aggregation. Huge amounts of data has been ingested by these models. They're apparently going to run out of all human created text within the next no more than 10 years, apparently. Everything that anyone's ever written that's in the public domain will be inside a large language model of some sort. So that means pretty much anything you've ever written in a public place, on a public website, will be in a model and available for someone to question. And that gives those models a degree of insight about you. And it allows people to use those to draw more information out. So you could just go into ChatGTV now and ask about yourself, but so could anyone else that's using that service. And that may be someone that just wants to target you. They may want to send you an email that's really very well crafted to get you to click on it. And perhaps they can discover something about you that's personal, something about your hobbies, just something that... It's just a trigger that means that you're likely to click on something or open something that you wouldn't otherwise do. So very much at risk.
Rufus Grig
Yeah, so that's really interesting. So moving on, I guess, into the security concerns then, where what you've described is meaning that it's possible to craft a much more personalised and therefore much more realistic and higher likelihood of being clicked on phishing email, for example.
Tony Leary
Indeed, yeah. And of course, these LLMs work in two directions in a way. So we've... often talk about things being a dual use technology and pretty much all of IT is, but certainly AI is a tool. There are definite benefits to people that want to be bad actors and use data for malicious reasons. And that could be either, as I described, using prompts or getting data out of, say, a model like ChatGDP or a service that's public, but it could also be just building their own. They can use the same tools that anybody else can to build a service that learns about certain people, certain companies. and gives them the ability to, whatever checks and balances may be within services like ChatGDP, then obviously anything they build won't have those. So that, of course, is the other side of this. Whatever we can use for good, somebody else can use for malicious reasons as well.
Rufus Grig
I guess if it's good at making good people productive, it makes bad people productive to do more bad things more quickly at the same time. Are there specific things that enterprises deploying generative AI themselves need to worry about from a security? perspective?
Tony Leary
First of all, it's getting governance people involved from the very start. And it's now with the emergence of legislation around this with certainly in the EU with the EU AI Act, I think it's treading quite carefully with these services. Certainly our own policies is graduated. It talks about use of certain, I guess, very simple services like ChatGDP, albeit we don't permit people to use that because they can use Copilot. But then there's this tiered policy then deals with... services that are provided by what you might call a trusted partner like Microsoft or AWS, then moving through to services that we completely build ourselves, that are completely bespoke. And I would say that those are the highest risk from an enterprise perspective. Anything you're building from the ground up, I think that's something that needs a lot of care and a lot of oversight of.
Rufus Grig
Great. Thanks very much, Tony. I'd like to move now to the environmental impact of generative AI. And this has received quite a bit of coverage in the press. Fran, can you just talk us through what is the challenge here? Are we seeing much more power being consumed because of Gen AI?
Fran Thomas
For sure. It's estimated that the world's data centres probably consume somewhere between 1% and 1.3% of all global energy demand. And that's on the rise, which is crazy, isn't it? And there's a whole bunch of different dimensions to how Gen AI impacts. the environment. But just speaking about energy specifically for a minute. I sort of feel like we can break this down into a bunch of different phases. And I think the obvious one is the training phase where we take huge amounts of data, petabytes at times, so certainly those big headline grabbing LLMs. That's a huge amount of computational intensity to build that model. And there are ways that we could address how those models are built, and that's an emerging discipline. So looking at the model efficiency, considering building smaller models where appropriate. something called pruning, which is parameter reduction. I know quite often we talk about this model's capable of a billion parameters, but every one of those parameters that it's trained upon, there's a cost to that, isn't there, in terms of the computational power needed to process that parameter. So more parameters isn't always better. I know that's one of those benchmarks that we've come to look at within the news, especially as this new model has this new capability.
Rufus Grig
Size is everything, I guess,
Fran Thomas
is what you're saying. Size is not everything. That's probably a good way to put it. And then selecting the appropriate underlying models. As we know, most of LLMs are built on other LLMs. You know, the universal truth, isn't it? There's nothing ever new. So picking the underlying model and picking something which has been optimized is often quite important when looking at training phase. So most of us, though, end up living and using AI within the inference phase, which is the bit that we use to get an answer from an AI while also training, as Tony was talking about earlier. We are using the model, but we're also contributing to its ongoing evolution. Some of those same principles apply with the training phase using a model that perhaps is more efficient if it is capable of giving you the result that you want. It's not always the best idea to use the most capable model because that most capable model will have more parameters. it will require more computational intensity to actually give you the answer that you want. And that can be seen as wasteful, I guess. But some of the other things that we can look at there are around the same principles as good, efficient code, making things event-driven, analyzing the code, exiting process at the earliest opportunity, using compression, queue versus scale. So there's a desire, isn't there, for everything now. That might not necessarily be the best way to minimize energy consumption. Waiting might be the best thing to do sometimes. And, you know, in lots of models, that is okay. So reasonable availability targets as well. I know many times we're using off-the-shelf services that deliver these results in AI, but there is a lot of bespoke model building still going on because that's where you often get the best results. they require infrastructure. So think about the availability targets. And if you are doing that and specifying your own infrastructure to run models on, selecting specialist infrastructure, so things that have been built to run LLMs are much more efficient than those that haven't. Leverage caching, obviously, in terms of results, server-side optimization, things like TensorFlow serving. So there's an abstraction layer emerging now that is all about performance. with regard to consumption with the LLM. And then we arrive at the ground floor, I guess, which is the underlying infrastructure operation. And as ever, data center selection is going to be really important, or infrastructure provider is really important from a privacy and security perspective, but it's also important from an environmental impact management perspective as well. Some of the obvious decisions there are pick the right partner that is focused on the right objectives. Look at the location that you source that computational power from. I think there's an emerging understanding that hosting infrastructure in places in the world where cooling demand isn't quite as high is probably one of the good things that we can do. But there's also even wherever that data center is, the quality of that data center, quite specifically sometimes around cooling performance, that's often overlooked. How you cool the data center is quite often a really big part of the emissions performance of that data center. And if you go... Specifically looking for this, you can find really unique examples of things which are doing things differently. For example, in Stockholm, they have heat recovery system, which sends the excess heat to nearby residential homes, which is quite a cool idea. So there are lots of innovative ways that that can be done. I think many have heard about Microsoft's endeavor to try and put infrastructure on the bottom of the ocean as one way to deal with it. So there's lots of interesting things going on in that space. And then again, ultimately, if you are specifying the actual hardware in the data center, you picking that accelerated hard we mentioned already, and perhaps even using AI as well to help solve that problem around the orchestration of data centers. That's something that is emerging. I know the big hyperscalers are effectively using that to improve cooling performance, reduce energy consumption by building models of those things themselves. It's worth mentioning that the energy consumption Online, when we're using that computation infrastructure, it's just one element of that. We have to think about the manufacturing process associated with the infrastructure, considering 50% to 60% of all the emissions associated with IT infrastructure is actually embodied within the manufacturing process. A big part of that is sourcing the right equipment that has the right ethical supply chain, because it's not just the environment impact in terms of the emissions, but also there's human rights abuses involved in those rare earth mining situations. DRC comes up all the time, but that's still true. It comes up all the time because it's still true. And that's something we should also think about who we partner with from a hardware perspective, if that's a decision that you're also making.
Rufus Grig
Okay. So there's an awful lot to unpack there. I guess a lot of the sorts of decisions that you're talking about in terms of where are we sourcing the hardware, where are we placing the data center? Some of those are the jobs that for most of organizations, that's a decision they're outsourcing to their hyperscaler provider. So they are putting trust. in a Microsoft or an AWS or a Google to do that. But they're still, even given if they're not building it from scratch and they're certainly not mining the materials, they can still look at the credentials and the sustainability quality of these different hyperscalers. Is that a very active area? Are there people you can go to for help in terms of determining who better partners are in this space?
Fran Thomas
Yeah, I think that's certainly something that is becoming more... to the forefront. So within Curve, as an example, we have a set of sustainable service design principles that attempt to force you to think about all of these nuances about how you architect a solution. And while I think there's sort of a working assumption with organizations, ours included, that all of our workloads are in the cloud. And I think that is broadly true, but I would estimate there's still five, maybe 10% of infrastructure that, yes, okay, we maybe haven't bought and put things in our own closet. but we have co-hosted in a way that's not using the hyperscaler. So it's using those, let's say a second tier hosting provider that doesn't really have a great net zero strategy, doesn't have a science-based target. And they're the ones that we need to think about and capture with these sustainable services I am principal. So I think as an organization, it's really important to code expectations around how you engineer, not just for sustainability, but for privacy and security. And that's a well-accepted practice in privacy and security, I guess. That's something that we often do. We think about the impact really early on in the process as part of our standard architectural practice. I think more of that needs to happen around sustainability and not just in the environment as well. What are the risks? What are the opportunities from a social value perspective when we deploy an infrastructure or service? So yeah, I think more or less going on, one of the conversations that I seem to be having more frequently with our industry partners is exactly what really good looks like. from a software engineering principle perspective, from an infrastructure architecture perspective, a green networking architecture perspective, sustainable program delivery, sustainable logistics. And that differs depending upon the vertical that you exist within. That's different within the tech industry that we're in, that it is from a goods delivery industry. Some things are going to be more or less important that you can have more or less control over. And sometimes it's about looking for those things where you have the greatest impact on first and working your way down the list.
Rufus Grig
But there are huge numbers of decisions you can make from Do I use AI or not in this project? Do I use an enormous model or do I use a smaller model? Right down to which infrastructure provider, how do I provision, where do I provision? Lots and lots of decisions that seem even within the remit of an individual user or an architect or a CIO can move the dial significantly in what the environmental impact is of what we're building.
Fran Thomas
Exactly. And I think that that's a really important idea there that Quite often, organizations will see that sustainability may be important. There's some people looking at it, but that's the thing that's happening over there. When in reality, your project managers, your technical architects, your developers, your project office administrators can all have an impact on the solution with a good set of principles that are well communicated. I think that's probably a really important point.
Rufus Grig
Okay, thank you. I guess all of this is throwing up. Lots of opportunity for regulation and rules and ways of governing the use of AI. Tony, this is new technology. Regulators are often slow to catch up. What is the regulatory position at the moment?
Tony Leary
Well, I think you can say that there is one, but it depends where you are. So within the EU, the EU AI Act came into force in August, albeit it won't be, it's law now, but certainly it will be phased in. over the coming years. What's interesting in the UK is that the previous government, the previous Conservative government in the UK, decided to take a, actually a contrarian approach to Europe, where Europe were choosing to regulate. and control quite closely, the UK decided that they called it their pro-innovation approach. That still stands. The new Labour government, I don't think has done anything about that yet. But I think given our proximity to Europe, obviously the GDPR applies to us, which I think right now is probably still the most important regulation actually around AI. But in time, I expect we'll either just reuse the AI Act perhaps in the UK, Or more likely, what will definitely happen is that given the fact that these regulations that the EU creates are extraterritorial, which means they apply to any one of their citizens anywhere in the world, and they have a lot of their citizens in the UK, that will assume or comply with the EU AI Act, even though obviously it won't be law in the UK.
Rufus Grig
Yeah, I mean, it's extraordinary to think that the last Conservative government would not want to do what Europe was doing. What a surprise. It does seem to me, you know, GDPR... principles are generally followed pretty heavily by US firms now because they know if they want to trade in Europe, they need to have offer choice on cookies, back to our earlier conversation. So that does sound fairly likely. So are there any frameworks or standards that organisations can work to that helps potentially prepare them for whatever the regulatory regime eventually turns out to be and also help them navigate the environment or the privacy, the security and other concerns?
Tony Leary
Yeah, they're definitely emerging around the world. I think the most mature is in the US. The US standards body is called NIST, and they released a risk management framework for AI. I think it's probably just in the last few months, it's actually become sort of version one, so to speak. And actually, the US is interesting, because again, you'd expect with their history with Silicon Valley, they'd maybe have taken that approach that conservative government did in this country to be pro-innovation. But actually, the Biden administration issued an executive order in October. last year around safe and secure and trustworthy AI. So the UK is somewhat out of step here. And of course, NIST being a US government body, then responded by creating these frameworks for organizations to use, which of course are available to anyone, anywhere. In the UK, the Information Commission's office has also released some guidance. They've got some workbooks in Excel that you can use around assessing risk, et cetera. So right now, I think the NIST framework is the most mature, but... But with the EU AI Act Now law, I suspect we'll start to see lots of things emerging from Europe as well.
Rufus Grig
Okay, thanks, Tony. Let's wrap up this session with a bit of a conversation around ethics, because there have been concerns raised about fairness, about transparency, about equality. Fran, can you talk us through some of the ethical considerations around generatory AI in particular and its increasing use?
Fran Thomas
Yeah, so there are many, as you've pointed out. I think there's an overarching thing here is that wherever the risk exists, it exists more with the application of AI. So that probably says something about the technology in itself, which is not unexpected. And you mentioned transparency and accountability and whether or not, where's the fairness if there's no transparency and who is accountable? I think they're really important questions, but there's a couple of things that really stand out for me. One is the training bias. And this is something that has been actively worked on, but sort of way one models and we're well past that now, we're incredibly biased. And that's because the data that was available was biased. And there's a really good example that I've used a few times there around an engine, which was supporting large candidate application set resume filtering. In technical roles, the majority of the resumes historically have been male individuals. And Using and applying that training data to build a model is going to continue to perpetuate the underrepresentation within that situation. So we have to be really careful about the inputs that we use, the training data that we use to not perpetuate biases that exist in society. It's really easy to do.
Rufus Grig
So I guess we've reframed the old adage, instead of garbage in, garbage out, it's bias in, bias out, is it?
Fran Thomas
Exactly. I like that a lot, Rufus. I'm going to use that. Thanks.
Rufus Grig
You're welcome.
Fran Thomas
So that's one, that bias and that bias is inherent and it's not going to go away anytime soon. There are lots of sort of clunky guardrails added to the large LLMs to deal with this at the moment without fundamentally dealing with it. Because the only way to fundamentally deal with it is to improve the quality of the data we put in, which is a long-term problem. The other one though, is what I'd call weaponization. And I'm not necessarily thinking about the application of AI within military circles and talking about the application of AI for use of disinformation. So what do we fundamentally get from AI? We get some distilled, sometimes new information, but largely distilled.
Rufus Grig
The ability to create information, which is highly believable, has a huge impact on society. I don't think we understand the impact that's had in the early days of this technology so far. I think there has been a huge number of events. And I think in the future, we'll be looking at documentaries about the application of this technology, cause and effect of this outcome within a society. I think that that's something that we're heading for. There'll be another review of... the guidelines that are associated with this technology beyond those that have just been published once we really understand the impact, because I don't think it's been seen just yet. Pulling all that together though, it's that overarching responsibility. It's a tool. As technical specialists, as an organization charged with building systems which have an impact on society fundamentally, we have to be really aware of the decision-making that we use to build those systems. And as the old adage isn't there, with great power comes great responsibility. And never can that be truer. And I think I've said this once already, but it's an amplification engine AI really, isn't it? It amplifies your ability to get something done. So make sure the thing you're trying to do is pointed in the right direction, is there for good. Consciously as well, because sometimes it's easy to make a decision that you think is for good. But if you're not aware of the impact of those decisions, then...
Tony Leary
potentially that makes it more difficult okay really really interesting stuff i'm going to ask you each one final question which is for anyone listening to this podcast you know if you could do one thing as a result of listening to this what would it be tony you first i think for enterprises i think it's just being aware of where these tools are being used and where people may want to use them coming
Fran Thomas
from a csa won't be surprised to know it's about control it's about organizations if they're doing these things it's doing them with some level of approval and knowledge rather than just letting people do whatever they like. And for individuals, I think just that similar view about having that awareness and presence around when you're using these tools. For example, ChatGDP, which we've spoken about already, you can go on there and you can tell them that you don't want your data used in a training model, even if you're on a free tier, but you have to go and do that. So I think it's taking care of your data, which was true before AI came along. But as Fran says now, I think the risks of... have increased. So it's even more important.
Tony Leary
Okay. Thanks, Tony. And then finally, Fran.
Rufus Grig
So I'm going to cheat here and get two for one. I think I'm also saying the same thing as Tony as well. It's sort of pick who you work with, pick your partner, pick your tools carefully, make sure that the systems, the models, the organizations that you're engaged with are consciously making decisions about the environment, social value, the impact of those things. If you pick the right partner, then they will be thinking about the environment for you, and you won't necessarily be generating more emissions by using the tool. If you pick the right partner, they'll be thinking about bias. They'll be thinking about whether or not that tool can be weaponized.
Tony Leary
Will Barron Brilliant. Look, thank you both. Really, really interesting conversation. If you've been interested in what we've had to say, then please do get in touch. Tell us what you think. Agree, disagree. We'd love to know what you're thinking about. You can find out more about Curve and about AI in general by visiting us at curve.com. Please do listen out for the next episode. You can subscribe, you can tell all your friends. So it remains for me to thank our guests very much, Tony Leary and Francis Thomas, and to thank you for listening. And until next time, goodbye.

Description

Key Highlights:

Security vs Data Privacy: Why It Matters: Understand the key differences between security (protecting systems from threats) and data privacy (safeguarding personal information). Learn why both are crucial when developing and using AI systems.
Concerns for Your Private Life: Explore how AI may impact your privacy, from data collection to digital tracking, and what you should be aware of to protect your personal information in an AI-driven world.
Security Risks in AI: Delve into the major security concerns surrounding AI, including vulnerabilities in AI models, data breaches, and how malicious actors could exploit AI systems.
Relevant Regulations: Learn about current laws and regulations governing AI, data privacy, and security. We discuss frameworks like GDPR and emerging policies to ensure AI development remains ethical and compliant.
Addressing AI’s Environmental Impact: Discover the environmental challenges posed by Generative AI, such as high energy consumption and carbon emissions, and explore ways organisations and developers can reduce AI’s ecological footprint.
Ensuring Adherence to Ethical Guidelines: Find out how organisations and individuals can ensure adherence to responsible AI practices, including transparency, fairness, and bias mitigation, to keep AI systems aligned with ethical standards.

Tune in to this episode on Responsible AI to learn how to navigate the ethical, privacy, and security challenges of AI.

Hosted on Ausha. See ausha.co/privacy-policy for more information.

Transcription

Rufus Grig
Hello and welcome to The Learning Curve, the podcast from Curve that delves into the latest developments in information technology and explores how organisations can put them to work for the good of their people, their customers, society and the planet. My name is Rufus Grigg and in this series, with the help of some very special guests, we're looking into all things generative AI. Now the world's massively excited about Gen AI and most of this series has been about the technology itself and its applications. But with a few episodes under our belts, we wanted to... pause and consider some of the risks, the challenges and the cautions associated with the tech and its expected widespread adoption. So today we're going to focus on four particular aspects. We're going to look at security, privacy, sustainability and ethics. Perhaps all of them could be taken together and described as responsible AI. And for that, I'm joined by two more of my brilliant Curve colleagues. Firstly, I've got Tony Leary. Tony is Curve's Chief Information Security Officer or CISO. How are you doing, Tony?
Tony Leary
Well, yeah, thank you, Rufus.
Rufus Grig
Good. Thank you very much for joining us. And I'm also joined by Francis Thomas, who is our Chief Sustainability Officer. How's things, Fran?
Fran Thomas
Yeah, good. Thanks, Rufus.
Rufus Grig
Brilliant. Thank you. So let's dive straight in with the security and privacy piece. Security and privacy are obviously very closely linked. But Tony, can you sort of perhaps unpick what the difference is? Is there an important distinction and how we need to think about each one?
Tony Leary
Sure. I mean, sometimes they're used interchangeably, I think, but they are different, for sure. I think most people know what information security is. It's about keeping data safe. I think most people have that concept of what it is. And for the most part, we rely on other companies, even as individuals who rely on companies, whether it's Curve, Microsoft, Google, to protect data. And ultimately, that's what security means for most people. Privacy, on the other hand, it relates to really surround choice and our ability to exercise choices on how data is used and processed. So that could be that you want one company to take a particular action with data, but another not to. And also the right to change your mind about that as well. And data protection regulations, GDPR, of course, from a UK perspective, we follow GDPR. And that's put that whole concept of choice and power really for individuals into law. And I think just one further point about the relationship between the two is that these days, I think it is linked very closely and effectively, they're mutually dependent. dependent on each other. But I'm not sure that was always the case. We're probably old enough to have been using the internet for 30 years in my case. And certainly, if a system is privacy supporting, it has to be secure. I think that direction of relationship is definitely always there. But you could certainly have a secure system that doesn't have privacy, a public system. But I think these days, even public sites, whether it's Facebook, even X and others, have privacy policies and have some level of... privacy enforcing controls. These days, I think we can say they're very closely linked together.
Rufus Grig
Okay, thank you. Interesting. And some very high profile companies there that have been never quite that far from controversy over potentially some of those privacy issues. So let's start with privacy then. In an enterprise context, thinking about AI and generative AI in particular, what are the main concerns?
Tony Leary
I think it's people using public tools. So these platforms like ChatGDP in particular, There's huge amounts of publicity when it was launched. I think even now there's 100 million people also registered for that service. A lot less now using it regularly, but certainly a lot of people have got access to it. And lots of people saw some benefit from using it, not just students cheating at their homework, but people in business, whether it's helping to write things. We've all used it. And these tools are actually quite good just for first drafts and things. So I think they've had a lot of take up there, but there's always a risk that you share that. data about yourself or about Curve as a company or a customer even with a service. And then that services privacy policy allows them to use that data in a model. An example of that is someone putting data into ChatGDP, might be corporate data. They may be doing so for them for valid reasons to use ChatGDP to draw some insights about data. But that then may be used by OpenAI in their model. And let's say a competitor was just asking a really broad question about Curve, that data may be returned and that data is confidential. And certainly if you're on a free version of ChatGDP, their privacy policy allows them to reuse data in their model. So that's absolutely a concern for any enterprise and even individuals, really, in terms of what you share with these services.
Rufus Grig
So if I just get that clear, I mean, there were certainly some examples early on when ChatGPT first sort of became really big in the public consciousness of, say, be a customer service agent using ChatGPT to say, please, here's the details about this customer. Please write me an email that I can respond. The concern, I guess, is that I've said, here's Tony, here's his financial situation, here's his medical problems. Please give me an answer. And someone else then gets served up that data because the model has trained on it and ingested it and learned it. Is that right?
Tony Leary
Indeed. Yeah, that's a real risk. Yeah. And I think, as you say, there's been examples of that actually happening. And I think more generally, just from a privacy standpoint with these systems, they're very hard to test. And the examples we've mentioned, that's almost people finding the edges and the flaws in these services once they're live, which it was ever thus, I guess, with a lot of free internet services. But these systems are particularly dangerous from a privacy perspective, just because of the amount of data they're aggregating. And... the ability for people just to craft responses and get data out of them. And I think that's just overall, it's just always a challenge with these large language models, that ability for companies really to properly test them. And no doubt as time moves on, their tools will improve. But right now, I think there's some real risks there. And also just geography. So as mentioned, I mentioned the GDPR, that's really strict about where data is processed. And it could be, for example, we may have customers that obligate us to process data in the UK, but certain services, they may well be hosted in the EU or the UK, but AI services may be happening somewhere else. They may outsource it to someone like OpenAI, that data may be going off to the US, for example. And that would be a breach from a GDPR perspective in terms of an obligation to understand where data's going and particularly around offshore processing. So lots of concerns around that. with privacy and certainly with anything that's public. I think that's where I'd start.
Rufus Grig
Okay. So you've certainly scared us a little bit in terms of the potential for things that can happen. So what can organisations do? What sort of guardrails should they put in place? I guess, you know, it's fairly easy to say to your staff, don't use ChatGPT, but how do you manage that around the services that you talked about just now, where, you know, a service that the company is perfectly happy using suddenly bolts on a fancy new GenAI service. that happens to do its processing in the States or somewhere else. How does an organization keep hold of these things?
Tony Leary
I think it's training, education to start with. But I guess from a sort of a backend perspective, it's around inventory. It's really just understanding what systems you're using and where these things, if there are AI services being used, where is that processing happening? For any organization that's got an information security management system, this type of approach is completely normal. No, it's about knowing where your information assets are. and measuring risk, managing risk around those things. So there isn't really anything that new with these sorts of services, but I think it's more the genie out of a bottle issue, I suspect, with users using things like ChatGDP in their, I guess, let's say their personal life, finding them useful and wanting to use them for corporate work. And I think in that case, it's just making sure they've got an alternative, which within Curve, everyone would have access to Copilot, Microsoft Copilot. So that is effective. effectively a version of ChatGP that they can use to get similar results you would expect.
Rufus Grig
Okay. And how do we get hold of that copilot? Because I think obviously Microsoft has a lot of different copilots and there's also some reassuringly expensive ones. I guess if you're talking about a ChatGPT equivalent, how would they access that?
Tony Leary
Yep. So anyone that's got a Microsoft license even can go to copilot.microsoft.com and they'll get some view of, I guess it's a version of Bing effectively. But it's a version of Copilot that works across the internet. Some users within Curva are piloting Microsoft Copilot, the M365 Copilot version. And that gives you, I guess, a switch where you can choose to search the web or internally. So there's a couple of different approaches depending on what license you have. But certainly, if you go to that website, it'll be obvious what you're able to do.
Rufus Grig
Okay, thank you. So that's good advice for enterprises, though, and how they sort of start worrying about that. What about for anyone listening who's concerned about... privacy in their private lives. Any particular guidance there?
Tony Leary
Well, yes. Yeah, lots. Probably too much for the time we have today. But I mean, ultimately, whatever you share online, it has some level of risk. And certainly if it's in any sort of public forum, cookies are quite an old school thing. But I still personally worry or look at cookie banners when they come up. And I saw one earlier today and the site or this particular cookie management system this site had. said that they shared data with 1,568 different companies. all of which are gathering data about how you're using that site. And it's all going off, being aggregated, it's being traded, it's being shared with other companies that want to sell to you. But ultimately, there's a profile of all of us online, and it's really hard to avoid sometimes these types of cookies and what's being shared. And a lot of sites, not normally in the EU or the UK due to GDPR, but certainly elsewhere in the world, cookies always feel not optional. put it that way. And there's this idea or this concept of dark patterns, how the sites are designed to basically to make it very difficult to disable things like cookies.
Rufus Grig
Does generative AI in particular enhance that risk? Are there services that people are using that they might not otherwise have used? Does the generative AI in itself create tools to people who want to do bad things with your data? Does it make that easier? I mean, I'm just thinking specifically about what the implications of Gen AI's availability are on that personal privacy.
Tony Leary
I think so. Just again, it comes back to aggregation. Huge amounts of data has been ingested by these models. They're apparently going to run out of all human created text within the next no more than 10 years, apparently. Everything that anyone's ever written that's in the public domain will be inside a large language model of some sort. So that means pretty much anything you've ever written in a public place, on a public website, will be in a model and available for someone to question. And that gives those models a degree of insight about you. And it allows people to use those to draw more information out. So you could just go into ChatGTV now and ask about yourself, but so could anyone else that's using that service. And that may be someone that just wants to target you. They may want to send you an email that's really very well crafted to get you to click on it. And perhaps they can discover something about you that's personal, something about your hobbies, just something that... It's just a trigger that means that you're likely to click on something or open something that you wouldn't otherwise do. So very much at risk.
Rufus Grig
Yeah, so that's really interesting. So moving on, I guess, into the security concerns then, where what you've described is meaning that it's possible to craft a much more personalised and therefore much more realistic and higher likelihood of being clicked on phishing email, for example.
Tony Leary
Indeed, yeah. And of course, these LLMs work in two directions in a way. So we've... often talk about things being a dual use technology and pretty much all of IT is, but certainly AI is a tool. There are definite benefits to people that want to be bad actors and use data for malicious reasons. And that could be either, as I described, using prompts or getting data out of, say, a model like ChatGDP or a service that's public, but it could also be just building their own. They can use the same tools that anybody else can to build a service that learns about certain people, certain companies. and gives them the ability to, whatever checks and balances may be within services like ChatGDP, then obviously anything they build won't have those. So that, of course, is the other side of this. Whatever we can use for good, somebody else can use for malicious reasons as well.
Rufus Grig
I guess if it's good at making good people productive, it makes bad people productive to do more bad things more quickly at the same time. Are there specific things that enterprises deploying generative AI themselves need to worry about from a security? perspective?
Tony Leary
First of all, it's getting governance people involved from the very start. And it's now with the emergence of legislation around this with certainly in the EU with the EU AI Act, I think it's treading quite carefully with these services. Certainly our own policies is graduated. It talks about use of certain, I guess, very simple services like ChatGDP, albeit we don't permit people to use that because they can use Copilot. But then there's this tiered policy then deals with... services that are provided by what you might call a trusted partner like Microsoft or AWS, then moving through to services that we completely build ourselves, that are completely bespoke. And I would say that those are the highest risk from an enterprise perspective. Anything you're building from the ground up, I think that's something that needs a lot of care and a lot of oversight of.
Rufus Grig
Great. Thanks very much, Tony. I'd like to move now to the environmental impact of generative AI. And this has received quite a bit of coverage in the press. Fran, can you just talk us through what is the challenge here? Are we seeing much more power being consumed because of Gen AI?
Fran Thomas
For sure. It's estimated that the world's data centres probably consume somewhere between 1% and 1.3% of all global energy demand. And that's on the rise, which is crazy, isn't it? And there's a whole bunch of different dimensions to how Gen AI impacts. the environment. But just speaking about energy specifically for a minute. I sort of feel like we can break this down into a bunch of different phases. And I think the obvious one is the training phase where we take huge amounts of data, petabytes at times, so certainly those big headline grabbing LLMs. That's a huge amount of computational intensity to build that model. And there are ways that we could address how those models are built, and that's an emerging discipline. So looking at the model efficiency, considering building smaller models where appropriate. something called pruning, which is parameter reduction. I know quite often we talk about this model's capable of a billion parameters, but every one of those parameters that it's trained upon, there's a cost to that, isn't there, in terms of the computational power needed to process that parameter. So more parameters isn't always better. I know that's one of those benchmarks that we've come to look at within the news, especially as this new model has this new capability.
Rufus Grig
Size is everything, I guess,
Fran Thomas
is what you're saying. Size is not everything. That's probably a good way to put it. And then selecting the appropriate underlying models. As we know, most of LLMs are built on other LLMs. You know, the universal truth, isn't it? There's nothing ever new. So picking the underlying model and picking something which has been optimized is often quite important when looking at training phase. So most of us, though, end up living and using AI within the inference phase, which is the bit that we use to get an answer from an AI while also training, as Tony was talking about earlier. We are using the model, but we're also contributing to its ongoing evolution. Some of those same principles apply with the training phase using a model that perhaps is more efficient if it is capable of giving you the result that you want. It's not always the best idea to use the most capable model because that most capable model will have more parameters. it will require more computational intensity to actually give you the answer that you want. And that can be seen as wasteful, I guess. But some of the other things that we can look at there are around the same principles as good, efficient code, making things event-driven, analyzing the code, exiting process at the earliest opportunity, using compression, queue versus scale. So there's a desire, isn't there, for everything now. That might not necessarily be the best way to minimize energy consumption. Waiting might be the best thing to do sometimes. And, you know, in lots of models, that is okay. So reasonable availability targets as well. I know many times we're using off-the-shelf services that deliver these results in AI, but there is a lot of bespoke model building still going on because that's where you often get the best results. they require infrastructure. So think about the availability targets. And if you are doing that and specifying your own infrastructure to run models on, selecting specialist infrastructure, so things that have been built to run LLMs are much more efficient than those that haven't. Leverage caching, obviously, in terms of results, server-side optimization, things like TensorFlow serving. So there's an abstraction layer emerging now that is all about performance. with regard to consumption with the LLM. And then we arrive at the ground floor, I guess, which is the underlying infrastructure operation. And as ever, data center selection is going to be really important, or infrastructure provider is really important from a privacy and security perspective, but it's also important from an environmental impact management perspective as well. Some of the obvious decisions there are pick the right partner that is focused on the right objectives. Look at the location that you source that computational power from. I think there's an emerging understanding that hosting infrastructure in places in the world where cooling demand isn't quite as high is probably one of the good things that we can do. But there's also even wherever that data center is, the quality of that data center, quite specifically sometimes around cooling performance, that's often overlooked. How you cool the data center is quite often a really big part of the emissions performance of that data center. And if you go... Specifically looking for this, you can find really unique examples of things which are doing things differently. For example, in Stockholm, they have heat recovery system, which sends the excess heat to nearby residential homes, which is quite a cool idea. So there are lots of innovative ways that that can be done. I think many have heard about Microsoft's endeavor to try and put infrastructure on the bottom of the ocean as one way to deal with it. So there's lots of interesting things going on in that space. And then again, ultimately, if you are specifying the actual hardware in the data center, you picking that accelerated hard we mentioned already, and perhaps even using AI as well to help solve that problem around the orchestration of data centers. That's something that is emerging. I know the big hyperscalers are effectively using that to improve cooling performance, reduce energy consumption by building models of those things themselves. It's worth mentioning that the energy consumption Online, when we're using that computation infrastructure, it's just one element of that. We have to think about the manufacturing process associated with the infrastructure, considering 50% to 60% of all the emissions associated with IT infrastructure is actually embodied within the manufacturing process. A big part of that is sourcing the right equipment that has the right ethical supply chain, because it's not just the environment impact in terms of the emissions, but also there's human rights abuses involved in those rare earth mining situations. DRC comes up all the time, but that's still true. It comes up all the time because it's still true. And that's something we should also think about who we partner with from a hardware perspective, if that's a decision that you're also making.
Rufus Grig
Okay. So there's an awful lot to unpack there. I guess a lot of the sorts of decisions that you're talking about in terms of where are we sourcing the hardware, where are we placing the data center? Some of those are the jobs that for most of organizations, that's a decision they're outsourcing to their hyperscaler provider. So they are putting trust. in a Microsoft or an AWS or a Google to do that. But they're still, even given if they're not building it from scratch and they're certainly not mining the materials, they can still look at the credentials and the sustainability quality of these different hyperscalers. Is that a very active area? Are there people you can go to for help in terms of determining who better partners are in this space?
Fran Thomas
Yeah, I think that's certainly something that is becoming more... to the forefront. So within Curve, as an example, we have a set of sustainable service design principles that attempt to force you to think about all of these nuances about how you architect a solution. And while I think there's sort of a working assumption with organizations, ours included, that all of our workloads are in the cloud. And I think that is broadly true, but I would estimate there's still five, maybe 10% of infrastructure that, yes, okay, we maybe haven't bought and put things in our own closet. but we have co-hosted in a way that's not using the hyperscaler. So it's using those, let's say a second tier hosting provider that doesn't really have a great net zero strategy, doesn't have a science-based target. And they're the ones that we need to think about and capture with these sustainable services I am principal. So I think as an organization, it's really important to code expectations around how you engineer, not just for sustainability, but for privacy and security. And that's a well-accepted practice in privacy and security, I guess. That's something that we often do. We think about the impact really early on in the process as part of our standard architectural practice. I think more of that needs to happen around sustainability and not just in the environment as well. What are the risks? What are the opportunities from a social value perspective when we deploy an infrastructure or service? So yeah, I think more or less going on, one of the conversations that I seem to be having more frequently with our industry partners is exactly what really good looks like. from a software engineering principle perspective, from an infrastructure architecture perspective, a green networking architecture perspective, sustainable program delivery, sustainable logistics. And that differs depending upon the vertical that you exist within. That's different within the tech industry that we're in, that it is from a goods delivery industry. Some things are going to be more or less important that you can have more or less control over. And sometimes it's about looking for those things where you have the greatest impact on first and working your way down the list.
Rufus Grig
But there are huge numbers of decisions you can make from Do I use AI or not in this project? Do I use an enormous model or do I use a smaller model? Right down to which infrastructure provider, how do I provision, where do I provision? Lots and lots of decisions that seem even within the remit of an individual user or an architect or a CIO can move the dial significantly in what the environmental impact is of what we're building.
Fran Thomas
Exactly. And I think that that's a really important idea there that Quite often, organizations will see that sustainability may be important. There's some people looking at it, but that's the thing that's happening over there. When in reality, your project managers, your technical architects, your developers, your project office administrators can all have an impact on the solution with a good set of principles that are well communicated. I think that's probably a really important point.
Rufus Grig
Okay, thank you. I guess all of this is throwing up. Lots of opportunity for regulation and rules and ways of governing the use of AI. Tony, this is new technology. Regulators are often slow to catch up. What is the regulatory position at the moment?
Tony Leary
Well, I think you can say that there is one, but it depends where you are. So within the EU, the EU AI Act came into force in August, albeit it won't be, it's law now, but certainly it will be phased in. over the coming years. What's interesting in the UK is that the previous government, the previous Conservative government in the UK, decided to take a, actually a contrarian approach to Europe, where Europe were choosing to regulate. and control quite closely, the UK decided that they called it their pro-innovation approach. That still stands. The new Labour government, I don't think has done anything about that yet. But I think given our proximity to Europe, obviously the GDPR applies to us, which I think right now is probably still the most important regulation actually around AI. But in time, I expect we'll either just reuse the AI Act perhaps in the UK, Or more likely, what will definitely happen is that given the fact that these regulations that the EU creates are extraterritorial, which means they apply to any one of their citizens anywhere in the world, and they have a lot of their citizens in the UK, that will assume or comply with the EU AI Act, even though obviously it won't be law in the UK.
Rufus Grig
Yeah, I mean, it's extraordinary to think that the last Conservative government would not want to do what Europe was doing. What a surprise. It does seem to me, you know, GDPR... principles are generally followed pretty heavily by US firms now because they know if they want to trade in Europe, they need to have offer choice on cookies, back to our earlier conversation. So that does sound fairly likely. So are there any frameworks or standards that organisations can work to that helps potentially prepare them for whatever the regulatory regime eventually turns out to be and also help them navigate the environment or the privacy, the security and other concerns?
Tony Leary
Yeah, they're definitely emerging around the world. I think the most mature is in the US. The US standards body is called NIST, and they released a risk management framework for AI. I think it's probably just in the last few months, it's actually become sort of version one, so to speak. And actually, the US is interesting, because again, you'd expect with their history with Silicon Valley, they'd maybe have taken that approach that conservative government did in this country to be pro-innovation. But actually, the Biden administration issued an executive order in October. last year around safe and secure and trustworthy AI. So the UK is somewhat out of step here. And of course, NIST being a US government body, then responded by creating these frameworks for organizations to use, which of course are available to anyone, anywhere. In the UK, the Information Commission's office has also released some guidance. They've got some workbooks in Excel that you can use around assessing risk, et cetera. So right now, I think the NIST framework is the most mature, but... But with the EU AI Act Now law, I suspect we'll start to see lots of things emerging from Europe as well.
Rufus Grig
Okay, thanks, Tony. Let's wrap up this session with a bit of a conversation around ethics, because there have been concerns raised about fairness, about transparency, about equality. Fran, can you talk us through some of the ethical considerations around generatory AI in particular and its increasing use?
Fran Thomas
Yeah, so there are many, as you've pointed out. I think there's an overarching thing here is that wherever the risk exists, it exists more with the application of AI. So that probably says something about the technology in itself, which is not unexpected. And you mentioned transparency and accountability and whether or not, where's the fairness if there's no transparency and who is accountable? I think they're really important questions, but there's a couple of things that really stand out for me. One is the training bias. And this is something that has been actively worked on, but sort of way one models and we're well past that now, we're incredibly biased. And that's because the data that was available was biased. And there's a really good example that I've used a few times there around an engine, which was supporting large candidate application set resume filtering. In technical roles, the majority of the resumes historically have been male individuals. And Using and applying that training data to build a model is going to continue to perpetuate the underrepresentation within that situation. So we have to be really careful about the inputs that we use, the training data that we use to not perpetuate biases that exist in society. It's really easy to do.
Rufus Grig
So I guess we've reframed the old adage, instead of garbage in, garbage out, it's bias in, bias out, is it?
Fran Thomas
Exactly. I like that a lot, Rufus. I'm going to use that. Thanks.
Rufus Grig
You're welcome.
Fran Thomas
So that's one, that bias and that bias is inherent and it's not going to go away anytime soon. There are lots of sort of clunky guardrails added to the large LLMs to deal with this at the moment without fundamentally dealing with it. Because the only way to fundamentally deal with it is to improve the quality of the data we put in, which is a long-term problem. The other one though, is what I'd call weaponization. And I'm not necessarily thinking about the application of AI within military circles and talking about the application of AI for use of disinformation. So what do we fundamentally get from AI? We get some distilled, sometimes new information, but largely distilled.
Rufus Grig
The ability to create information, which is highly believable, has a huge impact on society. I don't think we understand the impact that's had in the early days of this technology so far. I think there has been a huge number of events. And I think in the future, we'll be looking at documentaries about the application of this technology, cause and effect of this outcome within a society. I think that that's something that we're heading for. There'll be another review of... the guidelines that are associated with this technology beyond those that have just been published once we really understand the impact, because I don't think it's been seen just yet. Pulling all that together though, it's that overarching responsibility. It's a tool. As technical specialists, as an organization charged with building systems which have an impact on society fundamentally, we have to be really aware of the decision-making that we use to build those systems. And as the old adage isn't there, with great power comes great responsibility. And never can that be truer. And I think I've said this once already, but it's an amplification engine AI really, isn't it? It amplifies your ability to get something done. So make sure the thing you're trying to do is pointed in the right direction, is there for good. Consciously as well, because sometimes it's easy to make a decision that you think is for good. But if you're not aware of the impact of those decisions, then...
Tony Leary
potentially that makes it more difficult okay really really interesting stuff i'm going to ask you each one final question which is for anyone listening to this podcast you know if you could do one thing as a result of listening to this what would it be tony you first i think for enterprises i think it's just being aware of where these tools are being used and where people may want to use them coming
Fran Thomas
from a csa won't be surprised to know it's about control it's about organizations if they're doing these things it's doing them with some level of approval and knowledge rather than just letting people do whatever they like. And for individuals, I think just that similar view about having that awareness and presence around when you're using these tools. For example, ChatGDP, which we've spoken about already, you can go on there and you can tell them that you don't want your data used in a training model, even if you're on a free tier, but you have to go and do that. So I think it's taking care of your data, which was true before AI came along. But as Fran says now, I think the risks of... have increased. So it's even more important.
Tony Leary
Okay. Thanks, Tony. And then finally, Fran.
Rufus Grig
So I'm going to cheat here and get two for one. I think I'm also saying the same thing as Tony as well. It's sort of pick who you work with, pick your partner, pick your tools carefully, make sure that the systems, the models, the organizations that you're engaged with are consciously making decisions about the environment, social value, the impact of those things. If you pick the right partner, then they will be thinking about the environment for you, and you won't necessarily be generating more emissions by using the tool. If you pick the right partner, they'll be thinking about bias. They'll be thinking about whether or not that tool can be weaponized.
Tony Leary
Will Barron Brilliant. Look, thank you both. Really, really interesting conversation. If you've been interested in what we've had to say, then please do get in touch. Tell us what you think. Agree, disagree. We'd love to know what you're thinking about. You can find out more about Curve and about AI in general by visiting us at curve.com. Please do listen out for the next episode. You can subscribe, you can tell all your friends. So it remains for me to thank our guests very much, Tony Leary and Francis Thomas, and to thank you for listening. And until next time, goodbye.

Embed