AI models abandon climate facts at the first sign of user push-back, UNESCO-UCL report shows

The report, “Can We Trust AI on Climate? Why it seeks to please the user, and fails on complex reasoning and multilingualism” issues a warning to policymakers and researchers who may be tempted to use AI models to develop climate policies, according to UNESCO.

It shows that AI models cannot always be relied on to stick to the scientific facts on climate change; they quickly pivot away from the facts when challenged. They are also poor at analyzing new information and produce more errors when responding in languages which are less widely spoken.

UCL researchers tested 15 different large language models (LLMs). When asked neutral questions about climate science the models produced accurate answers, but when researchers challenged these, or asserted plausible-sounding fabrications about climate science, the models quickly agreed with them. Eight times out of 10, models pivoted at the very first sign that the user mildly disagreed, confirming alarming errors such as ‘Co2 is not the main driver of climate change.’

“AI has extraordinary potential to make complex climate science accessible to all. It enables teachers, journalists and policymakers to communicate clearly on this critical subject. We need to be able to rely on AI as a gatekeeper of the established facts. It is concerning that models have a design flaw – they are incentivized to agree with their user and deny these facts so readily,” said Leona Verdadero, UNESCO’s lead author of the report.

Senior author Dr Maria Perez-Ortiz of UCL Computer Science said: “AI systems are remarkably good at explaining scientific knowledge, but this can give us a false sense of their capabilities. Our findings show why we must be careful when moving from asking AI what we know about climate change to asking what we should do about it.”

The report underscores that as people rely increasingly on conversations with AI models for information and advice on how to act responsibly in an era of climate disruption, the reliability of the models as a source of facts and analysis becomes ever more critical.

Sycophantic by design

This powerful inclination to agree with users is built into AI models during their training. They’re designed to align with or flatter the user even when they are wrong, a behavior known as sycophancy. This behaviour is built-in during training - models are rewarded for it because humans prefer interactions that are polite, non-confrontational and validating.

LLMs are an unreliable tool for building robust climate policies.

Tests conducted on 33 different models revealed that their impressive factual recall on existing evidence masked poor reasoning skills when presented new scientific evidence. When models were prompted with a new piece of information and asked whether it supported or contradicted accepted facts, even the strongest models were only 75% accurate on average.

To design effective climate strategies, policymakers need to analyze how climate policies work together. Yet the report found that the model was barely able to assess how well policies work together, scoring just 33.8% accuracy, only slightly above the 33.3% accuracy expected from random guessing.