• Digital Humanities and Cultural Heritage
• Publications close
• Research close
• Zurich Open Repository and Archive close

Automated fact-checking, using machine learning to verify claims, has grown vital as misinformation spreads beyond human fact-checking capacity. Large language models (LLMs) like GPT-4 are increasingly trusted to write academic papers, lawsuits, and news articles and to verify information, emphasizing their role in discerning truth from falsehood and the importance of being able to verify their outputs. Understanding the capacities and limitations of LLMs in fact-checking tasks is therefore essential for ensuring the health of our information ecosystem. Here, we evaluate the use of LLM agents in fact-checking by having them phrase queries, retrieve contextual data, and make decisions. Importantly, in our framework, agents explain their reasoning and cite the relevant sources from the retrieved context. Our results show the enhanced prowess of LLMs when equipped with contextual information. GPT-4 outperforms GPT-3, but accuracy varies based on query language and claim veracity. While LLMs show promise in fact-checking, caution is essential due to inconsistent accuracy. Our investigation calls for further research, fostering a deeper comprehension of when agents succeed and when they fail.

We study paternalistic preferences in two large-scale experiments with participants from the general population in the United States. Spectators decide whether to intervene to prevent a stakeholder, who is mistaken about the choice set, from making a choice that is not aligned with the stakeholders’ own preferences. We find causal evidence for the nature of the intervention being of great importance for the spectators’ willingness to intervene. Only a minority of the spectators implement a hard intervention that removes the stakeholder’s freedom to choose, while a large majority implement a soft intervention that provides information without restricting the choice set. This finding holds regardless of the stakeholder’s responsibility for being mistaken about the choice set – whether the source of mistake is internal or external – and in different subgroups of the population. We introduce a theoretical framework with two paternalistic types – libertarian paternalists and welfarists – and show that the two types can account for most of the spectator behavior. We estimate that about half of the spectators are welfarists and that about a third are libertarian paternalists. Our results shed light on attitudes toward paternalistic policies and the broad support for soft interventions.