European privacy advocacy group noyb is suing OpenAI because ChatGPT repeatedly generates inaccurate information about people and won’t fix it.
Despite its advances, ChatGPT still suffers from a tendency to hallucinate when it doesn’t know the answer to a question.
The unnamed public figure whom noyb is representing felt aggrieved when ChatGPT repeatedly got their date of birth wrong. When prompted for it, the model simply made up a date instead of saying that it didn’t have the information.
It may sound trivial to some, but misleading personal information can have serious consequences, which is why it’s considered a big deal in Europe. GDPR rules state that published information about individuals must be accurate.
If publicly available information is inaccurate, an individual has the right to request that it be corrected or deleted. They also have the right to know where their information was gathered from and where and how it is stored.
OpenAI says it can’t help with any of that. It can’t (or won’t) say what training data it used to gather information about the person and says it can’t stop ChatGPT from making factually inaccurate statements about the person, or others.
The company said it could filter requests about the person but not without blocking all responses related to them. OpenAI says that, “factual accuracy in large language models remains an area of active research”.
In other words, they know it’s a problem but they’re not sure how to fix it.
In its filing to the Austrian data protection authority (DSB), noyb says OpenAI “stated that there is no way to prevent its
systems from displaying the data subject’s inaccurate date of birth in the output, if the user asks for this piece of information.”
noyb, which stands for “none of your business”, says “Simply making up data about individuals is not an option,” and that OpenAI’s response to the issue is not good enough.
Maartje de Graaf, data protection lawyer at noyb said, “The obligation to comply with access requests applies to all companies. It is clearly possible to keep records of training data that was used at least have an idea about the sources of information. It seems that with each ‘innovation’, another group of companies thinks that its products don’t have to comply with the law.”
OpenAI may be the target of this lawsuit, but the problem ChatGPT has with making up false information is a problem other models have too.
Until AI models can learn to say “I don’t know,” and their creators can be open about their training data, issues like this will keep happening.
European users will need to decide whether the utility of a tool like ChatGPT is more or less valuable than their rights enshrined in the GDPR.
For now, they can’t have both.
