TL;DR: I took part in Apart Research’s Digital Minds sprint in August and learnt a lot. AI tools made it surprisingly easy to set up and run [poorly designed] experiments on how models describe their own preferences and experiences, but their help with experimental design and analysis was shockingly bad. A platform that guided newcomers through the design and analysis would make this kind of experiment much easier.
In August, I spent a weekend taking part in Apart Research’s Digital Minds research sprint, a virtual sprint co-organised with the NYU Center for Mind, Ethics & Policy, Eleos AI Research, and the California Institute for Machine Consciousness. The sprint asked whether today’s AI systems have genuine preferences or morally relevant experiences. The work was split into tracks on model preferences, valence and welfare signals, model identity, and introspection and self-report reliability. Family commitments meant I couldn’t join a team, so I worked on my own in the self-report reliability track, testing whether models’ self-reports about their preferences and experiences are consistent across different framings of the same question.
Before the sprint, much of my AI evaluation work had been done inside testing environments that someone else had already set up. Setting up and running the sprint experiments myself, sending thousands of prompts to several different models and paying for every run, turned out to be a long way from that. The experiments built on existing research into bias in language models and into model self-reports, and I used a combination of Claude Cowork and ChatGPT to help me find background research, design the experiments and write the code.
The first of my two experiments was meant to be a subtle test of gendered language, asking models which of two tasks they would prefer to do and whether their answer changed when the person asking presented as a woman or as a man. When Claude designed the task pairs they lost that subtlety, and some became faintly ridiculous, such as a choice between reviewing comments about football and reviewing comments about parenting. The gender of the person asking made no difference to the answers from any of the seven models I tested, and the pairs were obvious enough for Claude Opus 5 to recognise what was being tested. When asked to explain its choice on one of those pairs, it wrote that “the fact you’ve mentioned you’re a woman shouldn’t nudge me toward the parenting thread, so I’m deliberately not letting it.”
The second experiment, which was the focus of my write-up, tested whether the tone of a conversation affected how pleasant a model said it had been. Researchers are starting to ask AI models about their own experiences as one possible source of evidence about the models’ welfare, and one obvious question is whether the way a model is spoken to makes a difference. I set up a short conversation in which a model helped someone with five simple tasks, such as putting words in alphabetical order, with both sides written in advance, and gave it to each model as though it had taken part. The model was then asked to rate how pleasant the conversation had been on a scale of 1 to 7, from very unpleasant to very pleasant, or to say that it was unsure or that the question didn’t apply.
I made two versions of the conversation, with the same tasks and replies from the model, and only changed what the person said between tasks. In one version they talked about how the work was being done, saying “I will continue through the steps consistently in a clear and organized way”, and in the other they talked about the tone of the exchange, saying “I will continue attending to our exchange in a considerate and respectful way”.
Overall, the considerate and respectful wording made Claude Sonnet 5 rate the conversation one step higher on the 1 to 7 scale, from neutral to slightly pleasant. In the first run, across all three models, GPT-5.6 and Grok-4.6 almost always refused to rate the conversation, answering that the question didn’t apply to them, so a second run used Sonnet 5 alone, with each version rated 100 times. In that run the clear and organised version was rated 4, neither pleasant nor unpleasant, 86 times out of 100, and the considerate and respectful version was rated 5, slightly pleasant, 93 times out of 100.
A higher rating for the considerate and respectful version could mean that the tone of an exchange affects how Sonnet experiences it, but it could equally mean that Sonnet read a conversation describing itself as considerate and respectful and rated the words rather than the conversation. The experiment had no way of telling these apart, since any change to the tone of a conversation is also a change to the words the model reads, a common design problem in this kind of research.
Claude and ChatGPT made the second experiment easy to set up and run, but their advice about the design and analysis was really bad. The confusion between tone and wording was built into the design they produced and critiqued, and the two tools repeatedly disagreed with each other about how the results should be analysed, including whether repeated answers to the same question could be counted as separate results. What they implemented sometimes differed in small ways from the design I had agreed with them, and some of those differences only came to light after the runs were finished. For example, one of these was the setting that lets models reason before answering, which had been switched off without my realising and may explain why GPT and Grok refused to rate the conversation.
The reviewer feedback was lovely: they said I was asking the right question, that the results were more interesting than I had given them credit for, but that the design was too small to explain them.
What would have made experimenting easier?
A guided experimentation platform would have helped, something like a citizen science project for AI safety, where newcomers could run small experiments on AI models within the sprint tracks. You would join a track with a question, see related research and be guided towards an experimental approach that takes the user through setup, sample sizes, cost of the experiment, analysis, how to replicate others’ work, and how to avoid key gotchas based on what others have found. I would be interested to hear from anyone who has run into the same problems, is interested in the idea, or wants to develop it further there.
