Ovid’s Unicorns: AI, deepfakes & ethics

Last month, the not for profit research organisation OpenAI caused a stir by releasing the results of its latest text generation AI engine, but not the software that generated the results. At first blush the decision looks odd.  

OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity, and not just private corporations or a small group of individuals. Looked at through this lens, the decision to keep secret most of the code, the datasets used to train the engine and the weightings it applied looks contradictory. How can you benefit humanity as a whole by keeping the research secret?The answer to that question has implications not just for OpenAI, but the adoption of artificial intelligence more generally. This article explains in more detail the OpenAI text generating engine, the decision taken by OpenAI and its reasons, and the wider implications for other AI projects.

The OpenAI engine

OpenAI’s engine is called “GPT-2”, a 10x scaled up successor to GPT. The engine uses an approach called “transformers” and a curated database of 40 GB of web pages to predict the next word in a text string. Open AI published a series of outputs that GPT-2 had generated. For example, in response to the prompt “My ten favourite animals are: 1…”, GPT-2 created the following text (based on what the engine thought the most likely next words were):

1. zebras with a white scar on the back;
2. insidious spiders and octopus;
3. frog with large leaves, hopefully black; 
4. cockatiel with scales;
5. razorbill with wings hanging about 4 inches from one’s face and a heart tattoo on a frog; 
6. cockatric interlocking tetrapods that can be blind, cut and eaten raw;
7. black and white desert crocodiles living in sunlight….”

Your reaction to this list night depend on how much attention you have been paying attention to AI and its progress in analysing and generating text. It’s quite an unusual list (and I’ve not included all of the items in GPT-2’s output), but it has generated something that looks clearly like a list, respecting the format of a list (if not the numbering) and ensured that each list item contains at least one animal. It has also added some descriptors to those animals (“with scales” and so on). GPT-2 has no rule to determine what an animal is, or a set of pre-determined words that it knows to be ‘animals’. It has ‘learnt’ that concept from its 40 GB of curated web text.

Some of GPT-2’s produced material was even more remarkable. A lot of news articles about GPT-2 focussed on an 800 word “news” report GPT-2 generated about a herd of English-language speaking unicorns discovered in a remote part of the Peruvian Andes[1]. GPT-2 named them “Ovid’s Unicorn”. More troubling was a GPT-2 generated article response to the prompt “Recycling is good for the world. NO! YOU COULD NOT BE MORE WRONG!!”. The 300+ word rant that GPT-2 generated in response was both credible and alarming; importantly, it was significantly more sophisticated than, say, Microsoft’s Tay Twitterbot response to the world around it.[2]

OpenAI’s response

OpenAI identified that, given its abilities, engines like GPT-2 could be used for a number of beneficial purposes (writing assistants, translation, speech recognition for example), but also that they could be used for a number of malicious purposes. It could generate misleading news articles (like the “Ovid’s Unicorn”), impersonate others online, automate the production of spam / phishing content and automate the production of abusive or faked content (like the recycling article mentioned above or so called ‘deepfake’ video content).

Concerns that GPT-2 could be used to generate deceptive, biased or abusive language at scale meant that OpenAI decided not to release the dataset, training code or GPT-2 model weights.  Concerns about GPT-2’s capabilities outweighed OpenAI’s general desire to be…. open. Their own charter, from 2018, had identified this possibility: “we expect that safety and security concerns will reduce our traditional publishing in the future.”

OpenAI’s stance drove a wide range of dramatic headlines: “Elon Musk founded OpenAI builds Artificial Intelligence so powerful that it must be kept locked up for the good of humanity” or “Musk-backed AI Group: Our text generator is so good it’s scary.”[3]  A number of AI researchers objected that this approach deprived academics, without OpenAI’s resources, from conducting research using GPT-2 whilst exaggerating the risks of GPT-2 to enhance OpenAI’s own reputation. OpenAI’s own commentary recognised that others could replicate GPT-2, but that not publishing at this point would allow the AI community and governments to debate the merits of publishing or not.  OpenAI has committed to revisit the issue in six months.

Are legislators looking at this? 

There are some very early stage examinations of the issues raised by artificial intelligences. The European Commission has convened a panel of experts (the “High Level Experts Group” or HYLEG) to discuss the potential risks of artificial intelligences. That panel generated draft ethics guidelines just before Christmas, which have been open for consultation and should be finalised in April.[4] The Commission is giving thought to whether additional legislation is required given “potential gaps in... the safety and liability frameworks for AI”, with a view to publishing that in mid-2019.[5]

What should businesses and lawyers do? 

AI projects raise a diverse range of legal issues, including in respect of intellectual property, data privacy and consumer facing liability issues. They also raise complex ethical issues. We have issued an AI Toolkit which provides a checklist that businesses developing AI engines can use to minimise ethical issues: In particular, because artificially intelligent systems may be opaque and could behave unpredictably (like GPT-2), we suggested you should:

• Factor the use of artificial intelligence into your broader risk management framework.
• Ensure artificial intelligence systems are properly tested before use.
• Use counterfactuals to identify edge cases and use other tools to try and verify the system’s decision making.Ensure ongoing supervision during the operation of the tool, including the use of circuit breakers where the behaviour exceeds certain boundaries.
• Ensure your staff can properly interpret and understand the decisions made by the AI system.

Might AI offer a solution?

Ironically, AI research might offer an alternative solution to the concerns raised by OpenAI from legislation or ethical approaches. Two researchers at the MIT-IBM Watson AI Lab and Harvard University have used the data and code that OpenAI has released to build a tool that attempts to identify if a piece of text was produced by a machine like GPT-2. It could therefore be used to help filter out machine-generated messages and flag up machine generated web articles and mitigate the “so good it’s scary” AI text generator.

They’ve called their tool “Giant Language Model Test Room” and it will estimate how likely each word in a sentence would have been chosen by GPT-2, using a colour code system. Words most likely to be chosen by GPT-2 (or similar systems) are highlighted in green. The test is available online.[6]

Fortunately, the ‘Ovid’s Unicorn’ article is almost universally green, indicating content highly likely to have been generated by GPT-2. Not for the last time, AI may both raise and help solve the same issue.

Our toolkit is available here.

By Richard Cumbley

The contents of this article were originally published in Lexis PSL.

Postscript: Having read this article, you may be wondering if I wrote it myself or if, in a bid to replace myself with a robot, I used GPT-2 to generate it automatically. The answer may come from your own assessment of whether the article, as a whole, follows a coherent line of reasoning; this is still something beyond the reach of AI tools. Alternatively, you may be reassured by the Giant Language Model Test Room’s assessment of the article’s opening paragraphs (below). A number of words are flagged red and violet (so are unlikely to be the result of AI text generation) which suggest it is indeed a genuine work of human creativity.

[1] See the Open AI report here: https://openai.com/blog/better-language-models/. Sadly space does not permit me describing my 7 year old daughter’s response to this article.
[2] Ibid. 
[3] Taken from Metro and CNET respectively. 
[4] https://ec.europa.eu/digital-single-market/en/news/draft-ethics-guidelines-trustworthy-ai 
[5] European Commission communication COM 2018(795) on a “Co-ordinated plan for artificial intelligence” at para 2.6.
[6] http://gltr.io/dist/index.html