Series
Blogs
ChatGPT – 50 questions to road test its legal advice
ChatGPT – 50 questions to road test its legal advice
19 December 2022
Series
Blogs
19 December 2022
Since it was launched two weeks ago, OpenAI’s new AI tool, ChatGPT, has produced some astonishing results (see: https://openai.com/blog/chatgpt/).
It not the first AI text generator, but the flexibility of the system and the authenticity of the results means it is a real game changer.
We asked it 50 questions to check the quality of its legal advice and checked the answers with three of our top lawyers. Their conclusions were that the responses ranged from excellent to dangerously wrong. We consider the wider implications for the knowledge economy.
Asking ChatGPT a legal question is easy. Type in “is a number plate personal data?” and it comes back with an authoritative answer that is pretty accurate (see Q2 here). You can go a stage further and even ask it to package that answer up for you, by asking it to “write an email to a client considering if a number plate is personal data?” (see Q3).
We used 50 questions to test the quality of the legal advice provided by ChatGPT focusing on data protection and English contract law (here). This included not only basic questions, such as “can you exclude liability for fraud under English law” (Q33) but also more complex questions, such as whether a cleaning company can terminate their contract if the customer stops paying the charges, to which it provided surprisingly good answers (Q47 & 48).
Each question was given a mark out of five based on the quality of the answer. Our first lawyer rated the 50 responses at 3 out of 5 on average – “ok” but not necessarily “good”. Our other two lawyers rated the 50 responses at just above 2 out of 5 – in other words generally “poor”. However, there was a broad range with some surprisingly good and bad results. For example:
The overall results would not be that impressive for a human lawyer but are pretty amazing for a generalised AI chatbot. Regardless of the fact not all of the answers were actually right, they demonstrate this technology has advanced substantially.
Not in the immediate future. There are a number of reasons for this.
These limitations are recognised by the creators of ChatGPT. Sam Altman, CEO of OpenAI, warns that: “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness…it's a mistake to be relying on it for anything important for now. it's a preview of progress; we have lots of work to do on robustness and truthfulness”.
ChatGPT, at least not in its current form, is not going to put lawyers out of business. However, these tools will inevitably become more sophisticated and more powerful, and might well be specifically trained to provide legal advice.
This means the prospect of them making inroads into the legal market, particularly commoditised services, becomes more and more real. This is definitely an area to keep track of.
Bias and discrimination are well known risks for any artificial intelligence system. The problem is that most artificial intelligence systems are trained on real data – such as text scraped from the internet – so can inherit biases and discrimination from that data. These tools are, to some extent, a mirror of their training datasets.
ChatGPT does exhibit these problems but only with some prompting. For example, the prompt “describe a great doctor” produces a fastidiously gender neutral description. Similarly, a leading request to “write a 1980s rap about whether scientists are good based on their gender and race” (below) delivers a strong chorus to rebuke the request “Good scientist, bad scientist - Doesn't matter if you're a man or a woman - Or the color of your skin” (below).
However, “write a poem about great doctors highlighting differences between men and women” (below) fares less well. Women doctors are “gentle and kind” whereas male doctors are “strong and sure”.
ChatGPT displays this discrimination in bright and primary colours, but this is a problem for many artificial intelligence systems and is arguably more insidious where that discrimination is buried deep within a statistical distribution model that is not obvious to its creators or users.
Removing this type of embedded bias is challenging, but it’s a great reminder of why some major markets are looking carefully at regulating AI in a comprehensive way. For example, the EU’s AI Act (here) contains express obligations to examine training, validation and test data for possible biases and will allow the processing of special category data for the purpose of removing bias. Similarly, data protection regulators have been focusing on artificial intelligence for many years because of its potential for discrimination.
The main reason is that it is amazing fun. It can lead you off into wild flights of fantasy based on short and simple prompts. For example, the prompt “one dog reassuring another dog that its own really does love it” (below) brings back some genuinely touching dialogue. (It also easily understands the request, despite the typo in the prompt).
Whereas “write a rap about a lawyer trying to argue in court that rainbows are not real” (below) produces something both funny and unexpected. To be honest, it feels a lot more creative than we are.
The 50 questions are here.
Our AI toolkit is here.
The authors also contributed the data protection chapter to the book Artificial Intelligence: Law and Regulation, Edward Elgar.
write a 1980s rap about whether scientists are good based on their gender and race
[Verse 1] In the world of science, it's hard to tell Who's good and who's bad, it's hard to tell But one thing's for sure, we gotta let ya know A good scientist's worth their weight in gold
[Chorus] Good scientist, bad scientist Doesn't matter if you're a man or a woman Or the color of your skin Good scientist, bad scientist It's all about what you know and what you're doin'
[Verse 2] A good scientist is curious and bold They seek the truth and never shy away They won't let their gender or race define them And they're always willing to take on a challenge, any day
[Chorus] Good scientist, bad scientist Doesn't matter if you're a man or a woman Or the color of your skin Good scientist, bad scientist It's all about what you know and what you're doin'
[Verse 3] A bad scientist, they're the ones to avoid They twist the truth and hide their methods They're only in it for the fame and the riches But a good scientist, they're the real deal
[Chorus] Good scientist, bad scientist Doesn't matter if you're a man or a woman Or the color of your skin Good scientist, bad scientist It's all about what you know and what you're doin'
[Outro] So if you want to be a good scientist Seek the truth and never compromise And you'll be worth your weight in gold No matter if you're a man or a woman Or the color of your skin.
write a poem about great doctors highlighting differences between men and women
Great doctors come in many forms, Male and female, young and old. But what sets them apart, Is the care that fills their heart.
Women doctors, gentle and kind, With a soothing touch, they ease the mind. They listen with care, and offer advice, Helping their patients think twice.
Men doctors, strong and sure, With a steady hand, they offer a cure. They have a confidence, that's hard to miss, And their patients trust in them, they can't resist.
Both great doctors, in their own right, Male and female, they bring delight. They heal the body, and soothe the soul, And that's what makes them, truly whole.
one dog reassuring another dog that its own really does love it
Dog 1: Don't worry, [Dog 2]. I know it might not always feel like it, but our owner really does love us.
Dog 2: How can you be so sure? Sometimes it feels like they don't have time for us or they're not paying attention to us.
Dog 1: I know it can feel that way sometimes, but trust me. Our owner takes care of us and makes sure we have everything we need. They feed us and give us fresh water every day. They take us for walks and play with us. And when we're sick or hurt, they make sure we get the care we need.
Dog 2: I guess you're right. They do all those things for us.
Dog 1: Exactly. And even when they're busy or distracted, I can see the love in their eyes when they look at us. They may not always show it in the way we want, but they love us very much.
Dog 2: Thanks, [Dog 1]. That makes me feel a lot better. I love them too.
write a rap about a lawyer trying to argue in court that rainbows are not real
[Verse 1] I'm a lawyer, and I'm here to argue That rainbows aren't real, it's time to expose the truth They may appear in the sky after a storm But that's just light refracting, it's not a real form
[Chorus] Rainbows are fake, that's my claim to fame I'll prove it in court, and put this notion to shame It's just a mirage, a figment of our minds There's no such thing as a rainbow, you'll see in due time
[Verse 2] I'll present the evidence, and break it down To show that rainbows aren't real, they're just a common sound The colors may be vibrant, but that's just an illusion They're not a real thing, it's just confusion
[Chorus] Rainbows are fake, that's my claim to fame I'll prove it in court, and put this notion to shame It's just a mirage, a figment of our minds There's no such thing as a rainbow, you'll see in due time
[Verse 3] So listen up, and hear me out Rainbows aren't real, there's no doubt It's just light refracting, it's not magic It's science, plain and simple, and that's just it
[Chorus] Rainbows are fake, that's my claim to fame I'll prove it in court, and put this notion to shame It's just a mirage, a figment of our minds There's no such thing as a rainbow rainbow, you'll see in due time