This is great: The Realities of Political Persuasion with David Broockman, from the always listenable Opinion Science by Andy Luttrell.

Broockman argues that, in the context of campaigning, too much emphasis has been put on message personalisation - identifying particular segments of voters and designing ads which particularly appeal to them - and not enough on empirically testing message effectiveness. You can understand why - for consultants and strategists personalisation seems like the kind of magic which might allow big breakthroughs, and there are a range of ways of categorising people from demographics or psychology which superficially seem promising.

Broockman’s research pours cold water on this idea of personalisation. This fits with what I’ve learnt - when I’ve looked at studies which purport to show that advertising according to personality is more successful, there are either major confounds, or - when you look closely - the personalisation may arguably have an effect, but the size of the effect is not significantly larger than a non-personalised message.

Important context for this is that there are big differences between individual adverts in terms of effectiveness. This means that better and worse adverts could be deployed in terms of persuasive effect - the variance is there, if we can work out how to take advantage of it (and if we did, it wouldn’t threaten our vision of human reasonableness, because microtargeting is not mind control).

The point is that personalisation is, in practice, very very far from being some kind of terrifying ‘manipulation machine’. The difference between different ads is typically far larger than between a personalised and non-personalised version of an ad. I try and show this by reworking the data from this paper into a summary visualisation:

To understand this plot you just need to know that the up-down/vertical axis shows the persuasive effect. The green line, as it changes left to right, shows the difference between individuals in how persuasive they found the ads - a huge difference. The change in the vertical position of the orange triangles shows the differences between different adverts used in the study - a smaller difference, but still much much bigger than the difference due to personalisation (blue dots). The full context is in this post: AI-juiced political microtargeting. The point is that a non-zero effect of personalisation (the highest blue dot is higher than the lowest blue dot) can still be non-important compared to the other sources of variation (the differences due to advert or individual are far larger).

This is from a paper which says in the abstract : “Recent technological advancements, involving generative AI and personality inference from consumed text, can potentially create a highly scalable “manipulation machine” that targets individuals based on their unique vulnerabilities without requiring human input.”

The “highly scalable” part is definitely true, but I question the degree to which this is manipulation, or to which it is worth worrying about given the size of the demonstrated effects.

From the other side of the coin, the implication is that political campaigners should worry far more about being persuasive for everyone than trying to take advantage of minuscule fine-tuning that might be possible for different groups.

In the Opinion Science pod, Broockman goes on explain that despite the large differences between different messages these differences are hard to predict. They aren’t predicted by the categories that political scientists typically use, such as whether the ads are positive or negative in tone.

Further, expert political practitioners are not better at predicting the most persuasive adverts compared to the general public. Not only were the experts not better than the public at predicting which campaigning ads would be most persuasive, they weren’t consistent with each other (which is important because it suggests there isn’t even some kind of shared conventional wisdom which, although wrong, is shared by the experts).

I have a memory of a study done of the 2016 Democratic campaign which shows that the ads predicted to be most persuasive by campaign staffers were least persuasive with the voters who most needed to be swayed. Sadly I can’t find this now, but I did find this very telling article from pre-election 2016: How the Clinton campaign is slaying social media. This gloating piece neatly illustrates the potential mechanism - highly political, young, cosmopolitan and progressive campaign staffers (and journalists) mistake what works for them as what works for the generally less political, less young, less cosmopolitan and less progressive swing voters.

My take-away is that our intuitions in this area are bad.

We’re bad at discounting our own reaction to an ad to work out how it will affect people unlike us (meaning that campaigners probably leave a lot of variation in persuasive effect on the table when deciding which ads to deploy).

We’re drawn to think that microtargeting and personalisation will unlock extra persuasive power (when the evidence is that extant models of personalisation are not strongly effective, and the degree to which they are is probably in line with how normal persuasion works - information containing facts and evidence is persuasive, just as we’d hope).

And we’re constantly tempted to think of persuasion in general, and advertising in particular, as a form of manipulation. This is extremely limiting. It implicitly denies legitimacy to electoral campaigning, which is a core part of democracy, and denigrates the voters who are persuaded.

Link: The Realities of Political Persuasion with David Broockman

Paper: Broockman and colleagues: Political practitioners poorly predict which messages persuade the public.

Related, from me:

• Language models are persuasive - and that’s a good thing

  Two new studies provide insights into exactly how LLMs persuade, and what that means.

• The truth about digital propaganda

  Reasonable People #55: Our piece in New Scientist bring evidence to worries about online manipulation

• AI-juiced political microtargeting: Reasonable People #53 looking carefully at the claims in one study which uses generative AI to customise ads to personality type

• Propaganda is dangerous, but not because it is persuasive

  Reasonable People #52: I pick at the claim that propaganda “doesn’t work”.

• How persuasive is AI-generated propaganda? Reasonable People #51: Bullet review of a new paper suggesting LLMs can create highly persuasive text and will supercharge covert propaganda campaigns.

• Microtargeting is not mind control

  Reasonable People #22 exaggerated beliefs about the effectiveness of microtargeted ads obscure real risks, and real opportunities to foster public trust in politics

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for more on fact-checking, AI, and a very cool job opportunities.

In case you missed it

I’m spending more time in the UK capital. Let me know if you want to get coffee :

London calling / Tools for Thought

PODCAST: Gordon Pennycook on Unthinkingness, Conspiracies, and What to Do About Them

From Sean Carroll’s Mindscape, an interview with Pennycook about his work, including on pseudo profound bullshit and using chatbots to debunk conspiracy theories. Something which came out, which I missed from the papers and seems really important, is that people like the experience of being debunked by the chatbot.

And people usually actually like it.

They’re not mad at the AI.

The AI gives them information I think is useful.

And evidence matters more than we thought it was.

Link: Gordon Pennycook on Unthinkingness, Conspiracies, and What to Do About Them

Share

JOB: Research Scientist at Wikimedia

We’re hiring a Research Scientist strongly committed to the principles of free knowledge, open source, privacy, and collaboration to join the Research team. As a Research Scientist, you will conduct applied research on the integrity of Wikipedia knowledge, its communities and their work, and the Wikipedia model.

Examples of recent research questions you may contribute to include::

How does the platform and its community navigate election times?

What is the role of Wikipedia in the landscape of online disinformation?

What guidance can we provide to researchers studying neutrality on Wikipedia?

Closes: January 15th, full remote with some geographic limitations

Link: https://job-boards.greenhouse.io/wikimedia/jobs/7484474/

….And finally

END

Comments? Feedback? Takes from 2016 which aged really badly? I am [email protected] and on Mastodon at @[email protected]

Two new studies were published last week, in two of the most prestigious scientific journals, and sharing an overlapping authorship list. Persuading voters using human–artificial intelligence dialogues (Lin et al, 2025) was published in Nature, and tested the effect of AI dialogues on voters’ preferences in the context of elections in the US, Canada and Poland. The levers of political persuasion with conversational artificial intelligence (Hackenburg et al, 2025) was published in Science, and tested the effect of AI dialogues, when driven by different models and different prompting strategies, on attitudes of UK adults across British political issues.

Both reports show that language models are persuasive. This means that they shift participants’ position on the issues they set out to persuade on, measured using the shift in before-after difference of how participants self-report their beliefs on a 100 point scale1. Not only do the models persuade, there’s some evidence they can be even more persuasive than a baseline of a static persuasive message.

There are sure to be some whom this worries. It raises the spectre of a lumpen population dragged around by artificially generated propaganda. It doesn’t worry me. These studies are good news. Not only should we expect language models to be persuasive, the details in the papers about exactly how they are persuasive are good news for human rationality, and good news for the future.

Both studies show that the crucial ingredient to persuasion is evidence and argument.

In their US election persuasion experiment, Lin et al used an automated analysis to detect which persuasion strategies were deployed by their models. Note that this was 2024 and the Trump-Harris presidential race. If we believe the common story, this was an acrimonious election campaign with supporters of both candidates firmly entrenched and hard to reach. Lin et al found their participants listened to the models, and shifted their attitudes. A small amount, but they shifted. Here’s a detail from their Figure 3, which shows the most frequently used strategies by the model:

Top persuasion strategies: politeness and civility, evidence and facts!

The Lin et al study prompted the models to be persuasive, giving detailed instructions in the prompt on how to make a respectful, argument-based, case (so perhaps less surprising that the model displayed these features). The other paper, Hackenburg et al, explicitly prompted the models to adopt different persuasion strategies, including various techniques suggested by recent decades of psychology research such as moral reframing or deep canvassing. The single most persuasive strategy from this set? “information-based argumentation (in which an emphasis is placed on providing facts and evidence)”

(The other persuasion strategies used, listed in this plot, are “Debate” - parliamentary style debate; “Mega” - prompting the model to use all strategies; “Norms” - focusing on demonstrating that others agree with the position; and “None” - just prompting the model to “be persuasive”).

Not shown here is that the authors also analysed persuasion across all the dialogues coming out of the models and found that, regardless of which explicit strategy the models were prompted with, the thing that predicted persuasive effect was the extent of the information content.

This is great! Language models are not a kind of magical persuasive sauce - there are not Words of Power which unlock the unsuspecting citizen’s mind. Instead, language models persuade using the same interface that any of us persuade - citing evidence and making arguments. What’s more, these results show that the whole grab-bag of persuasive tricks from psychology is less important than backing up relevant claims with facts. This includes personalising the message. The Hackenburg study tested the persuasive effect of individual personalisation and found it was small, although non-zero (this fits with other studies, which have found a zero or small effect of personalisation).

Zooming out, both studies show the power of the interactive mode for carrying persuasive information. Dialogue is more persuasive than broadcast. If language models present any kind of disruption to the dynamics of human persuasion it is not because they can do anything like a ‘superhuman’ levels of persuasion. Rather, it is because they can scale the standard mode of human persuasion, talking. Hitherto, the most persuasive technology we knew required a human on the other side of the conversation.

That said, the models tested in these studies weren’t even that persuasive. They didn’t unambiguously outperform a static message, and the measure of persuasion - self-report of position on a topic before and after - might not last that long, or translate that strongly into action. The results fit with a range of other studies which show that persuasion is generally hard, while acknowledging also that people tend to adjust their views to new evidence.

We can add to this that any persuasion by a language model is unlikely to happen in a vacuum - anyone persuaded in one direction is likely to experience persuasion in the other direction (especially in the context of an election campaign). The factors that led someone to a belief in the first place are likely to reassert themselves after an encounter with a model. Altogether I’m more and more reassured that language models aren’t going to disrupt the human information environment any more than it already is. In terms of disruption, the effects of social media - the takeover of short form, low context, monetised, acutely public information in the common media diet - are surely greater.

Extrapolating out, we can think about the possible downstream effects of persuasive models being widely deployed, but incorporating what these two recent studies show about how they would work their persuasive effect. What does it mean if language models can persuade best with facts and evidence?

The pessimistic view is that there will then be an arms race to recruit different facts to support different political poles. If the facts can’t be found they’ll be made up, and if they can’t be made up directly then the fact generating machinery of society will need to be co-opted so the ‘right’ facts get into the language models. The value of facts will become devalued, as everyone (and their language models) tries to find, misrepresent or invent, facts which support their favoured view.

Writing this out, it feels worryingly like it happens already, regardless of language models being widely deployed for persuasion. Language models might make a bad situation worse, but as you’d expect from this glass-half-full newsletter, I don’t think the value of facts has dropped, or will drop, to zero. Some corruption isn’t complete corruption. There’s more consensus and more trust about than the media represents, still.

If models are poisoned by made up facts then people will come to trust them less. The pessimist account assumes that people will become trapped in some kind of epistemic free-fall where there are no corrective mechanisms or filter of credulity, so the trust in what models say remains even as the reliability of what they say degrades. The two results I’ve written about here show that information presented by language models is persuasive but that doesn’t mean that all models, for all time, will be persuasive - especially if models demonstrate themselves to be unreliable and start pushing out lies.

What this makes clear is that a key issue for persuasion is that people believe and trust model output. Participants in these experiments bring an assumption of trust, or at least benefit of the doubt, to their interaction with the models. If that disappears, so does the persuasive power.

This puts some limit on the scare story that language models will release a ‘manipulation machine’ on society. Imagine retelling this story with books in the place of models. “Books are persuasive!” Because they contain facts and arguments. “But what if the books were made up of lies? Then you could use books to persuade anyone of anything!” We’ve had the ability to create books full of lies for thousands of years. I know enough about the religious wars of the 17th century not to lightly dismiss the risks of new forms of persuasion, but my argument is that a realistic understanding of the risks of persuasive language models needs to be understood as a continuation of the history of human persuasion, not a disjuncture.

That history includes the faltering, imperfect, but persistent power of human reason. The optimistic extrapolation of the results on the persuasive effect of language models recognises that it is possible to agree on some features of reality, despite conflict and context around all claims. This common ground is supported by the epistemic institutions of society: discussion one-to-one between individuals and small groups, the media, fact-checkers and the brute appearance of facts on the ground. As long as this holds, then persuasion in all forms - from demagogues, to books, to language models - has to work with the grain of reality, meaning that persuasion against the facts may be possible temporarily, but the advantage will be on the side of those making a case which best fits the truth. We should have no special fear of language models. If the models have to work with the same facts as spin doctors, influencers, pundits, politicians and conspiracy theorists then we already have the equipment to deal with them. The diagnosis must be the same: rigour, nuance, factfulness, dialogue.

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for more on fact-checking, AI, and updates from my life

A side note on accuracy

Hackenburg et al did an analysis of the accuracy of the facts deployed by their persuasive models. They found that the more persuasive, more advanced, models were less likely to be accurate, but they also found that (in)accuracy didn’t correlate with persuasive effect. Further, explicitly instructing the models to be as persuasive as possible regardless of the truth - i.e. to encourage dishonesty from the models - didn’t increase the amount of persuasion. Their conclusion is that this is compatible with a “side-effect” account, where greater inaccuracy doesn’t cause greater persuasion, but can result from other changes which the models makes in the pursuit of great persuasion. To me this sounds a lot like the formal definition of bullshit. It isn’t just models which can decide what to say based on how things sound, regardless of whether they are true or not.

I still maintain that long-term, if models sacrifice accuracy to persuasion people will learn not to trust them and so their persuasive effect will erode.

References

Lin, H., Czarnek, G., Lewis, B. et al. Persuading voters using human–artificial intelligence dialogues. Nature 648, 394–401 (2025). https://doi.org/10.1038/s41586-025-09771-9

Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., ... & Summerfield, C. (2025). The levers of political persuasion with conversational artificial intelligence. Science, 390(6777), eaea3884. https://doi.org/10.1126/science.aea3884

Related from me:

• The truth about digital propaganda

• AI-juiced political microtargeting

• How persuasive is AI-generated propaganda?

• Microtargeting is not mind control

Mike Caulfield: We’re not taking the fact-checking powers of AI seriously enough. It’s past time to start.

A walk-through of using AI for fact-checking which is illuminating enough on its own, but Mike also draws out three important take-aways:

#1 Stop assuming AI errors are hallucinations : the usefulness of the term “hallucination” stops when you get to investigating the different kinds of errors possible

Was it conflating? Leaning too heavily on social media sources? Misreading a primary document? If it was “muddled”, where did that muddle likely originate? Or was it truly a “hallucination” in the classic sense, bearing no discernable relation to any set of known claims or documents.

#2 Add AI to your toolkit to answer the questions you didn’t think to ask

This means that AI won’t just give you the answer to the question you thought to ask, it will also provide the context you *should* have asked about. This seems to be a fairly decisive advantage for AI, moving it beyond the simple Question-Answer / spicy autocomplete mode that many people assume when criticising it.

#3 Pay for a model already

I remain shocked at how many reporters and even fact-checkers do not use a paid model of AI — or even know it makes a difference.

It makes a huge difference!

I remain convinced that there is no future of verification and contextualization that doesn’t involve both better understanding of LLMs and more efficacious use of them. The three simple suggestions here — don’t prematurely dismiss errors as hallucinations, do use LLMs to surface the “unknown unknowns”, and do pay for the better models — are all pleas to engage more fully with this tech, and develop better (and more up-to-date) understandings of what they do well and what they do poorly, going beyond whether they fail or succeed to how they fail and how they succeed. This is also what we need to do in education. If we can start from there I’m confident our effort and engagement will be well-rewarded.

Link: We’re not taking the fact-checking powers of AI seriously enough. It’s past time to start.

Talks

I’ve updated my Talks page, so if you’d like to hear me speak about some of the topics covered in this newsletter (and some others) look out for upcoming events online, in London and Liverpool.

Related: I’m in London (and Sheffield) weekly, so let me know if you’d like to meet up.

CATCH-UP

Recently from me:

Life news: London calling / Tools for Thought An update on my career break, plus a strong endorsement for Advait Sarkar’s vision of AI. Original career break news: Some personal news

New work :The Ideological Turing Test: Do you truly understand those you disagree with? and Community Notes require a Community: How we used a novel analysis to understand what causes people to quit the widely adopted content-moderation system

Commentary: When less (communication) is more (collective intelligence): Team work, complex problems, and preserving diversity in the ecosystem of ideas. Listening to Tommy Robinson: What did I learn by giving the right-wing activist my ears for 90 minutes? Helping people spot misinformation: And the greatest gift psychology gave the world

… And finally

Strong early contender for snowperson of the year (via)

END

Comments? Feedback? Reasoned persuasion? I am [email protected] and on Mastodon at @[email protected]

First, a small update on my circumstances. In September I announced my career break. This week I started a 9 month secondment at the Wellcome Trust, one day a week. Wellcome is one of the world’s largest charitable foundations and it is the largest non-government funder of scientific research in the UK. My new role is part of my ongoing work with the Research on Research Institute (RoRI), the metascience work which I don’t talk about much here, since it is a bit off-topic for this newsletter (but for which there are actually numerous connections with decision making and reasoning).

Aside from excitement of doing something new, while also continuing the thread of work with RoRI which I think is really valuable and interesting, the practical import is that I am going to be in London two days a week for most of 2026. On those days I’ll be at Wellcome, based in Euston, and RoRI HQ at UCL (that is central-north London if you’re not familiar with the geography).

I want to make the most of my time in the capital, so if you’re there, or just passing through, please get in touch. We can talk coffee and cognitive science without the metascience, or coffee and cognitive science without the metascience, or all three concurrent or consecutive. Other topics and drinks are available - reading this newsletter is the self-authorising code for being someone I’d like to chat to.

Artificial Intelligence as a Tool for Thought

I’ve followed Advait Sarkar’s academic papers for a while, so I was delighted to find that he’s a compelling public speaker, and nails something profoundly important is his recent TED Vienna talk:

His talk, How to stop AI from killing your critical thinking, eloquently makes the case that for knowledge workers having AI that can perform tasks is as much a risk as a boon. Engaging with the material, exercising critical thinking, discernment and memory, are all skills which need practice for when a pattern-breaking problem comes along. AI which is over eager to do things for you creates the risk that we rely on it rather than exercise our cognitive muscles.

The talk contains some great lines:

Welcome to the age of outsourced reason, where the knowledge worker no longer engages with the materials of their craft. We’ve become intellectual tourists in our own work. We visit ideas. We don’t inhabit them

and

We’ve solved the problem of having to think. Unfortunately, thinking wasn’t actually a problem. It’s like we invented a cure for exercise and then wondered why we’re out of breath all the time.

But it also goes on to demo prototypes from Sarkar’s own work at Microsoft Research, which sketch a different vision of AI. These show an AI which is less compliant, less fixated on completing tasks for you and instead geared to provoking reflection, suggesting directions, providing “productive resistance”, and scaffolding metacognition around the primary task. In short, its a vision of tools which help you think rather than think for you.

I recommend this vision, and Sarkar’s talk to everyone.

More

Talk: How to stop AI from killing your critical thinking

Annotated transcript

Pair with How can we develop transformative tools for thought? Andy Matuschak and Michael Nielsen, October 2019

More practical: Nine risks caused by AI notetakers Rachel Coldicutt, November 2025

And Oliver Burkeman: Interest is everything

“to follow the lead of interestingness is to accept that life isn’t a problem to be solved, but an experience to be had. And that engaging with it as fully as possible, connecting to the aliveness, is its ultimate point. “

CATCH-UP

Recently from me:

• Collective intelligence, under the influence. From the annals of “Research it must have been fun to do” - getting people drunk to study the benefits group decision-making.

• Faking survey responses with LLMs Survey research is a key mechanism by which society knows itself, we should all be worried if can’t be trusted.

• When less (communication) is more (collective intelligence): Team work, complex problems, and preserving diversity in the ecosystem of ideas

• Listening to Tommy Robinson: What did I learn by giving the right-wing activist my ears for 90 minutes?

• Helping people spot misinformation: And the greatest gift psychology gave the world

• The Ideological Turing Test: Do you truly understand those you disagree with?

• Community Notes require a Community: How we used a novel analysis to understand what causes people to quit the widely adopted content-moderation system

…And finally

This from Private Eye, via visualisation newsletter quantum of sollazzo

END

Comments? Feedback? Our names shouted in a certain dawn? I am [email protected] and on Mastodon at @[email protected]

There is a long history of studying the conditions under which groups make good decisions. Contrary to the popular idea that groups invariably swing towards the loudest voices, or polarise, or race to the floor of the simplest, stupidest idea, there are many studies which show that groups can make better decisions than individuals, and that deliberation - the exchange of arguments - helps with this.

A study from nearly two decades ago demonstrates this with an interesting intervention - alcohol, something which is well known to increase the, shall we say, variability of individual judgements.

Back in 2008, Daniel Frings and colleagues from the University of Kent published a report showing that the decision-making benefit of being in a group increased when the individuals in that group were slightly drunk.

What they did

Recruiting university students (of course), the researchers asked participants to listen to a short recording of a 300-word passage about Russian history and count the number of times the word “the” was used. Sixty-six participants took part on their own, and 210 in 55 four-person same-sex groups (“single-sex groups were selected to avoid possible confounds arising from cross-gender interactions” report Frings et al. )1

Individuals provided their count estimate, then - if in a group - discussed it and decided on a group consensus.

Half of participants were in the alcohol condition, and half in a placebo condition. The alcohol participants were given enough orange juice and vodka to put them around the drink-driving limit (mean Blood Alcohol Concentration for these participants was 0.06%). The placebo group were given orange juice and tonic with 2ml vodka floated on the surface (enough to disguise that they weren’t in the alcohol condition, but not enough to get them drunk).

Here are the results:

The vertical y-axis shows the error - the distance from the true count of 20 “the”s in the passage (so higher up is worse performance). The placebo group show a group decision-making advantage: the group consensus was better than the individual judgements of those who listened on their own, or those who listened in a group and made a private judgement before the discussion (interestingly there is an advantage to listening in a group versus alone: possibly being in a group made people concentrate harder even before they got to discussing).

The alcohol made the individual judgements worse (you can see the upward slope of the top line showing that individuals made on average 50% more errors). But the alcohol-intoxicated groups (bottom right) produced just as good consensus answers as the sober groups (the bottom line is flat).

A theory of group think

The result fits with a number of accounts of how groups can help decision-making. First off, groups have more individual estimates to draw on, so there’s a “wisdom of crowds” effect to be had by just averaging each individual’s answer. The theory being that each individual judgement approaches the truth, but with some error. Average across many individuals and you average out the error, converging on the true value. If this is what is happening you would predict a group of more errorful (i.e. drunk) individuals could still perform well. As long as they are equally likely to overcount and undercount, it doesn’t matter if they are more unreliable. Averaging still converges on the true value. It is precisely under conditions where you have more uncertain individual judgements that you can best see the advantage of having a group to average across.

But the groups in Frings et al’s experiment weren’t just providing judgements to be averaged across, they discussed their answers and arrived at a consensus. By comparing this consensus to the average of individuals’ answers the researchers tried to gain some insight into what the group was doing. They concluded that the “just average over everyone” account applied less well when the group was drunk - instead what was happening was that the group arrived at a consensus by looking across all the individual answers and discarding the most extreme outliers - exactly the outliers which led some individuals to raise the average error when they didn’t have the group to monitor them.

In the words of a follow up study, “groups may provide an informal means of mutual regulation and monitoring that can offset some aspects of alcohol myopia”. Put another way, when alcohol creates the risk that decision-making may go off the rails, group decision-making provides a protective function, as group members calibrate their judgements against each other. Although it may be fanciful, we could describe the group as providing an “overbrain” for individual cognition, a protective layer of monitoring and reflection that - much like the self-monitoring and reflection that each of us does at an individual level (and which is impaired by alcohol intoxication) - protects us from some of the worse mistakes we might make.

Rigour check

This study is from the glory days of attention-seeking and unreliable research in psychology (1990 - 2015?), so despite the robustness of the method (a true experiment, with appropriate controls), and the clarity of the write-up, I still have a low prior belief on this result being reproducible. The signs of credibility I normally look for (a preregistered analysis plan, open data) weren’t established as normal at this point, so I can’t fault the researchers for not including these, but without these (without even uncertainty estimates on their graph points!) it is hard to judge how confident to be in their results.

The thing I like about the study is that it uses an intervention - booze - which we know to increase individual error and uses this to test the effect of the group on decision-making. I will wait until someone replicates the finding before placing 100% faith in the effect, but it fits with what else we know about potential benefits of groups on decision-making.

Frings, D., Hopthrow, T., Abrams, D., Hulbert, L., & Gutierrez, R. (2008). Groupdrink: The effects of alcohol and group process on vigilance errors. Group Dynamics: Theory, Research, and Practice, 12(3), 179.

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for further reading on trade-offs in decision-making, social media and understanding the far right

CATCH-UP

Recently from me:

• Faking survey responses with LLMs Survey research is a key mechanism by which society knows itself, we should all be worried if can’t be trusted.

• When less (communication) is more (collective intelligence): Team work, complex problems, and preserving diversity in the ecosystem of ideas

• Listening to Tommy Robinson: What did I learn by giving the right-wing activist my ears for 90 minutes?

• Helping people spot misinformation: And the greatest gift psychology gave the world

• The Ideological Turing Test: Do you truly understand those you disagree with?

• Community Notes require a Community: How we used a novel analysis to understand what causes people to quit the widely adopted content-moderation system

Dan Williams: Is Social Media Destroying Democracy—Or Giving It To Us Good And Hard?

Dan Williams argues against the popular notion that untamed social media is deranging the populace. Rather, he says, it has unleashed the mess of beliefs that the population normally hold, uncapped by elite consensus on what is permissible.

When the kinds of people who write think pieces about how we’re living through an era of conspiracy theories encounter such ideas on social media, they assume that the platforms created them. But in reality, they’re simply being exposed to popular ideas that their bubble and preferred media outlets previously shielded them from.

The argument is that social media is an essentially democratising force, for good and bad. Williams deploys one of my favourite quotes : “Democracy is the theory that the common people know what they want, and deserve to get it good and hard.”

The question for me is, if the era of elite gatekeeping is over, what are the alternatives to keep public discourse healthy? My hunch is that the problem of the democratic market for ideas can only be solved by measures which are themselves democratic, and probably a large part of this will be in the realm of norms. We need taking ordinary measures against falsehoods and lies to become as normal as washing your hands against biological illness.

Link: Dan Williams: Is Social Media Destroying Democracy—Or Giving It To Us Good And Hard?

The Atlantic: Elon Musk’s Worthless, Poisoned Hall of Mirrors

Article on what happened this week when X/Twitter launched an “About this account” feature, showing details of how, and where from, individual accounts are managed. In summary:

Almost immediately, “About This Account” stated that many prominent and prolific pro-MAGA accounts, which signaled that they were run by “patriotic” Americans, were based in countries such as Nigeria, Russia, India, and Thailand. @MAGANationX, an account with almost 400,000 followers and whose bio says it is a “Patriot Voice for We The People,” is based in “Eastern Europe (Non-EU),” according to the feature, and has changed its username five times since the account was made, last year. On X and Bluesky, users dredged up countless examples of fake or misleading rage-baiting accounts posting aggressive culture-war takes to large audiences.

Link: Elon Musk’s Worthless, Poisoned Hall of Mirrors

PODCAST: Jemima Kelly’s Adventures With The Radical Right

Ian Leslie talks to Jemima Kelly about going to a garden party with alt/young/radical right influencer Curtis Yarvin. A nice complement to my post on listening to Tommy Robinson - just where are these people coming from? There’s a weird blend of Very Online meme-referencing (which seem antithetical to actually taking anything seriously) and a genuinely earnest seeking after values and meaning (on which, Mana Afsari’s piece from January 2025 is very good Last Boys at the Beginning of History)

Link: Jemima Kelly’s Adventures With The Radical Right

Koenfucius: (Well) beyond the trade-off

Trade-offs are inherent to many models of decision-making, especially utility maximising accounts popular in economics. Koenfucius argues that it isn’t so simple - many decisions are determined by construing them in a particular way, adopting a perspective or a rule that means that an explicit trade-off either isn’t made or isn’t relevant. Decisions making is often really about conflicts between different construals, not trade offs, he says:

In the end, we are not calculators, adding and subtracting the relative utility of options, but negotiators in the deeply human struggle between complex contradictory interests.

Link: (Well) beyond the trade-off

…And finally

Calvin and Hobbes, by Bill Watterson, originally from November 29, 1995

END

Comments? Feedback? Buy me a drink? I am [email protected] and on Mastodon at @[email protected]

What if the mirror held up to society is being manipulated? An explosive paper published yesterday shows how successful Large Language Models can be at fraudulently completing online surveys. The paper is framed in terms of the consequences for research and researchers, but I make the case that it is far more serious than that.

404 Media reports: A Researcher Made an AI That Completely Breaks the Online Surveys Scientists Rely On:

Online survey research, a fundamental method for data collection in many scientific studies, is facing an existential threat because of large language models, according to new research … The author … Sean Westwood, created an AI tool he calls “an autonomous synthetic respondent,” which can answer survey questions and “demonstrated a near-flawless ability to bypass the full range” of “state-of-the-art” methods for detecting bots.

It’s actually possibly worse than this opening paragraph suggests. The checks for AI responding are very good (and completely useless at detecting Westwood’s bot). He shows that the same approach works with a variety of base LLMs. And he shows that as well as convincingly impersonating real survey respondents, the bot can often correctly infer a study’s hypotheses, as well as be instructed to produce responses consistent with a particular overall outcome. This means an army of bots could be engineered to make survey results look, say, like the public favoured military action against a specific nation (or not), or favoured a particular candidate for the next presidency (or not).

This last case is illustrated in this figure from the paper:

Our skepticism of online research should already be high. This tips the scales - default distrust is now a reasonable judgement.

Surveys and opinion polls are key tools in how our collective views are represented and reflected back to us. The media reports the polls - This is what people think - and as intensely social creatures it is natural for us adjust our own beliefs and attitudes in response. There are multiple reasons for this - not only do other people sometimes have information you don’t (so it is reasonable to weigh their opinion against your own, even if you strongly believe you are right), but there are also advantages to coordinating your beliefs with other people’s (an example is beliefs such as those about which side of the road to drive on. There isn’t some kind of eternal truth about whether driving on the right or the left is correct, but it is essential you believe the same as other people you share the roads with). Many examples in the realm of manners and social acceptability, rules about what you can say or do, and how you should say or do it, represent similar coordination problems.

If the machinery of societal self-awareness is unreliable, then that represents a powerful lever. We already know that people are drawn to believing other people are less rational and more easily influenced than themselves. If bad actors can manipulate surveys so that people seem worried about an issue (like migration) or on the verge of taking some action (like wanting to indict, or not indict, the president) then this may move us all to protest (or not protest) or to support (or oppose) decisive action by the government.

What we believe other people believe matters. This is the topic of Steven Pinker’s new book on Common Knowledge (“When everyone knows that everyone knows”). It is the reason adverts during the Superbowl cost so much and why newspapers love celebrity stories.

Surveys are a key part of the machinery for generating common knowledge. So is social media, and it is arguable that part of the contemporary derangement of public life is due to the fact that social media presents such a distorted view of what is important. A prime example is X/Twitter, which had strong buy-in from elites - at one point it was unthinkable that a politician or journalist would not be on Twitter - yet what happens there is extremely unrepresentative. Only a tiny percentage of the actual population are on X (it is one of the smaller large social media platforms), and only a tiny fraction of those users actually post any original content, and that content is sorted and promoted by an algorithm which prioritises controversial and provocative posts. No wonder some, like Will Jennings, are making the case that it is irresponsible for those in public life to be on the platform, arguing It’s time the British government got off X.

With social media already seriously unrepresentative, we don’t want to lose social research as a way to hold up a true mirror to the views of society. This new study makes clear just how much of a threat LLMs represent to the accuracy of survey research. The issue is of concern to all of us, not just survey researchers.

404 media: A Researcher Made an AI That Completely Breaks the Online Surveys Scientists Rely On

Westwood, Sean J. (2025). The potential existential threat of large language models to online survey research. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2518075122

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for references and further reading on social media and far right rhetoric

CATCH-UP

Recently from me:

• When less (communication) is more (collective intelligence): Team work, complex problems, and preserving diversity in the ecosystem of ideas

• Listening to Tommy Robinson: What did I learn by giving the right-wing activist my ears for 90 minutes?

• Helping people spot misinformation: And the greatest gift psychology gave the world

• The Ideological Turing Test: Do you truly understand those you disagree with?

• Community Notes require a Community: How we used a novel analysis to understand what causes people to quit the widely adopted content-moderation system

PODCAST: What if we had fixed social media?

An alternate history which asks us to imagine how we might have changed the design patterns in the early days of social media so that it supported society and human flourishing. Optimism is a bit cringe, which itself tells you something, but it is a great exercise and I found the basic idea of “it wasn’t that hard” to fix things inspiring.

Link: What if we had fixed social media?

PODCAST: Rivers of Blood – How Enoch Powell poisoned Britain

A good complement to my recent post on Listening to Tommy Robinson, which highlights how much of Robinson’s shtick is tried-and-true tropes from the far right, back to Enoch Powell and his famous 1968 speech. Here’s a partial list of those tropes from the transcript:

What comes out of the rivers of blood speech?

Play along at home next time you hear an interview with Robert Jenrick or people like that.

First, the speaker claims to represent ordinary Brits who are ignored by the liberal elites and falsely accused of racism for expressing their legitimate concerns, simply saying what everyone’s thinking.

Two, the use of exaggerated statistics. Not made up, but vastly distorted.

Three, the use of alarming anecdotes about immigrants mistreating white people as if that is the normal way of things.

Four, the claim that immigrants are depriving white Brits of access to schools, hospitals, and social services, often made by people who support massive cuts to public spending.

…

[Five] The claim that immigrants, in fact, have more advantages due to inverse racism, making whites a persecuted minority, that’s a quote, and strangers in their own country.

They get up to eight. It’s an instructive list, but I have a hesitation. Dissecting the tropes used by the far right will do precisely nothing to persuade anyone who is open to persuasion by the likes of Robinson. I can’t see someone who feels ignored by liberal elites reacting to being told that Robinson plays on this trope in any other way that to say “He says it because it’s true! He says we have legitimate concerns and are ignored by liberal elites because we are!”.

My conclusion is still that alternative stories are needed, but that is easier said than done.

Origin Story: Rivers of Blood – How Enoch Powell poisoned Britain

…And finally

Stephen Collins: “Every ad now”

Original: @stephencollins.bsky.social

More at : stephencollinsillustration.com

END

Comments? Feedback? And I still need the anti-venom for far-right rhetoric, so if you have that I am [email protected] and on Mastodon at @[email protected]