The Science of Belief: a deep dive

Looking at the weird ways that we form, store, and update our beliefs

The quality of the lives we lead is almost certainly a function of the beliefs we hold. That means if we want to understand how to live a good life – however you want to define ‘good’ – we probably need to understand how belief works. This piece is a deep dive on how we form, store, and update our beliefs, based on the best available science I could find.

But first, I want to raise the stakes a bit:

Beliefs impact our lives in all kinds of ways. Some of them are obvious and superficial; some of them are unobvious and profound.

On the more obvious side of things, it’s clear that beliefs impact the way we organise our lives. If I believe that smoking cigarettes, eating junk food, and living in isolation are good for me, I’m probably going to structure my life in ways that are detrimental to my wellbeing.¹

It also seems pretty clear that in conjunction with confirmation bias, beliefs influence the way we sift incoming information. If I believe that everybody hates me, I’m probably going to interpret ambiguous social signals as proof of my universal odiousness. Again, this can’t be good for my general wellbeing.

Examples like this are easy to come by and obvious – but there’s also lots of good evidence to suggest that our beliefs influence the way we experience reality at a much deeper level. Here are three particularly striking examples:

1) Remifentanil

There’s a synthetic analgesic opioid called Remifentanil. It’s administered intravenously and is used in operations for deep sedation. It’s two hundred times as potent as morphine and twice as potent as fentanyl. But here’s the catch:

If you give someone Remifentanil and tell them that they’ve been given an analgesic, i.e. something that will reduce their sensitivity to pain, its efficacy as an analgesic doubles. If you give someone Remifentanil and tell them that they’ve been given a hyperalgesic, i.e. something that will increase their sensitivity to pain, its efficacy as an analgesic completely disappears.

So, in other words:

If the receiver holds certain beliefs while Remifentanil is being administered, this powerful analgesic – two hundred times as potent as morphine – completely stops working.²

As far as I know, this is the only discovered case where belief has the power to completely decimate the impact of a potent drug, but it nonetheless points to the fact that beliefs can affect deep parts of our neural machinery in big ways.

2) Beliefs impact our reward circuitry

Another pretty wild finding that points to a similar conclusion comes from an experiment run by Gu et al³:

They took a group of smokers and measured brain activity while they – the smokers – smoked cigarettes. Here’s the thing, though: some of these cigarettes contained nicotine and some of them didn’t. By manipulating participant’s beliefs about which kind of cigarette they were smoking, these researchers could measure how belief impacted neural activity.

Here’s what they found:

First, when smokers’ beliefs matched up with the reality, their brains reacted as you might expect: in the nicotine condition, they showed elevated activity in their bilateral ventral striatum, a key player in the dopaminergic reward circuitry; when in the no-nicotine condition, they showed suppressed activity in the bilateral ventral striatum.

So far so good. No major surprises. But here’s where it gets interesting:

When the smokers believed they were smoking a cigarette that contained nicotine, but they were actually smoking a cigarette that didn’t contain nicotine, their reward circuitry responded in the same way as it had done when they were actually receiving cigarettes with nicotine in them.

And when smokers believed that they were smoking a cigarette that didn’t contain nicotine, but they were actually smoking a cigarette that did contain nicotine, their reward circuitry responded in the same way as it had done when they were actually smoking cigarettes with no nicotine in them.

So, basically:

Whether or not their reward circuitry registered the experience as rewarding or aversive was determined by their belief about whether or not they were receiving a rewarding substance – rather than whether or not they were actually receiving a rewarding substance. Also, this phenomenon isn’t isolated to nicotine; similar effects have been found with cocaine⁴ and alcohol⁵ too.

3) Beliefs structure our experiences

Here’s something you may not know:

We have more connections running from deep within our brains out to our sensory organs than we have running from our sensory organs in to our brains⁶. In some places, four times as many. Under the common sense view of perception, this doesn’t make any sense. The senses, so this view goes, are there to deliver information to our brains – not to receive it. What’s going on?

As recently as 2012, this strange information flow was enough to lead the AI pioneer Patrick Winston to state that our brains possess ‘a strange architecture about which we are nearly clueless.’ Since then, an increasingly influential framework known as Predictive Processing has come onto the scene that appears to shed light on this recalcitrant datapoint – and that also explains a variety of other previously unexplained findings too.

I’m not going to give a complete presentation of the theory now – this is a great overview – but for a quick tl;dr, predictive processing basically says that perception is made up of four key parts:

1. Generative models - our brains possess generative models that are trained on and constantly refined by our experiences.

2. Predictions - these generative models make predictions about what we are experiencing at any given time. Our experiences are constructed from these predictions. Notice: under this model, we are not passively receiving experiences from our senses; instead, these generative models are actively constructing them – hence why there are so many connections going from the brain out to the senses.

3. Prediction errors - predictions generated by our generative models are compared to the incoming sensory signal, e.g. from our eyes, ears, etc. If the difference between the prediction and the sensory signal is large enough, this generates a prediction error. Prediction errors are fed back up the processing hierarchy, where they produce 1) changes to the prediction, and 2) changes to the generative model itself.

4. Precision weightings - at any given time, our brain is able to adjust the precision-weighting it assigns to either top-down predictions and bottom-up signals. In some situations, our predictions will be weighted more strongly; in others, the bottom-up sensory signal will be.

To make this a bit less abstract, consider the Mooney image below.

When most people first see this, they perceive a meaningless collection of blobs. But if you now look at the image below and then return to this one, your perception of these blobs should hopefully have transformed.

…

…

…

…

…

…

…

…

…

…

…

…

According to predictive processing theory, two things have happened here: first, your generative models have been tweaked to incorporate this new information. As a result, they are now delivering recalibrated predictions. Second, your precision-weightings have changed: your experience is now being constructed from your top-down predictions, i.e. the dalmatian, rather than the bottom-up sensory information, i.e. the blobs. This is why, try as you may, you can’t make the incoherent blobs return – your generative models are predicting a Dalmatian, and there’s now no way for you not to see it.

Alongside things like past experiences and hyperpriors, beliefs are an important component of the generative models that predict our experiences. If you hold certain beliefs, your brain will make certain predictions that align with those beliefs – and you will perceive reality in certain ways.

This is likely what’s happening in the Remifentanil and smoking experiments discussed above; and it also seems to be why the placebo effect, the white christmas effect, chronic illnesses, and all kinds of other seemingly unrelated phenomena happen.

So, yeah, in sum:

Beliefs are hella important.

They inform the way we structure our lives, the way we interpret incoming data, and the way we experience reality. If we can understand how beliefs work, we might be able to start manipulating them more intentionally, in ways that make our lives better.

To wit:

The rest of this piece.

Note: I wrote a section about what beliefs are, but given the final length of this piece, I decided to relegate it to the appendix. If you’re interested – or if you’re struggling with how I use words like belief and credence – you’ll find some more info there.

How we form beliefs

As far as I can see, the conventional wisdom about belief formation looks something like this:

1. Before accepting a proposition as either true or false, we have the ability to impartially entertain that proposition and weigh it based on the evidence.

2. Once we’ve weighed the evidence, we then either actively accept or reject the proposition.

3. If we accept the proposition, it becomes a belief; if we reject it, it becomes a, umm, disbelief.

In the literature, this common sense view is very close to what is known as the Cartesian theory of belief formation. Only difference is that the Cartesian theory spells out two additional corollaries that help make the theory more testable:

4. Accepting and rejecting a proposition uses roughly the same mental processes and so should be affected by interferences in similar ways.

5. Forming and rejecting beliefs is an active endeavour, i.e. it does not happen automatically or unconsciously.

On its face, the Cartesian⁷ view seems perfectly sensible, but as it turns out, there’s pretty good evidence to suggest that it’s probably wrong. Before explaining why this is the case, I’m going to introduce a rival theory of belief formation that looks – at least as things currently stand – like it might be right (or, at least, better able to account for existing evidence). This view is known as the Spinozan⁸ theory of belief formation, and it goes a little something like this:

1. We do not have the ability to impartially entertain propositions before accepting or rejecting them.

2. Instead, due to our mental architecture, we automatically accept propositions. It is only once they have already been accepted that we can then either actively accept or reject them.

3. The initial process of acceptance is passive, automatic, and effortless. The process of rejection is active and effortful.

4. To negate a proposition is to reject it (I’ll explain this bit more in a minute).

So, to summarise and concretise:

Let’s say that a friend comes up to me one day and tells me that ‘God’s real name is Fred.’ The Cartesian view states that I impartially entertain this proposition, weigh the evidence, and then either choose to reject or accept it.

(The bible, as far as I know, makes no reference to God as Fred; this friend has a history of making outlandish, unfounded claims; ergo, I’m going to reject this proposition.)

The Spinozan view, on the other hand, thinks that in order to so much as entertain the proposition, I first need to accept it. It’s only once the proposition has been accepted that I am then in a position to reject it. However, if something happens that interferes with this rejection process, I may well find myself uttering things like ‘Oh Fred, who art in heaven, hallowed be thy name.’

Jokes aside, this is actually (more or less) a prediction of the Spinozan view: we accept by default and if the process of rejection is perturbed in some way, we revert to acceptance.

(This whole debate may seem quite academic, but I think the ramifications are potentially pretty far-reaching. I cover some of the practical implications of this – and everything else I talk about in this piece – in the final section, ‘Takeaways: what you should make of all of this’.)

Now, onto the evidence.

The strongest line of evidence in support of the Spinozan view and against the Cartesian view comes from research on memory asymmetries between truths and falsehoods. In one very typical experiment⁹, participants were asked to memorise nonsense word meanings like, e.g. ‘suffa is a cloud’. Right after each word meaning was presented, the word true or false was flashed on the screen to indicate whether the previous meaning was to be taken as true or false. While all of this was happening, participants were asked to listen out for a tone. When the tone sounded, they were instructed to press a button as quickly as possible. The idea here was that by asking people to respond to the sound of a tone while memorising the word meanings, researchers could intentionally induce cognitive load and thereby see if true and false beliefs were differentially impacted by it.

During the course of the experiment, participants were shown 6 ‘true’ word meanings and 6 ‘false’ word meanings. Meanwhile, the tone was played 4 times – twice during ‘true’ meanings and twice during ‘false meanings. Given that the Cartesian view states that accepted and rejected beliefs are processed the same way and that they should be similarly affected by interference, we’d expect to see both true and false beliefs similarly impacted by the tone.

Alas, this is not what we see at all.

For the most part, false meanings that have been interfered with are remembered as being true, while true meanings that have been interfered with are also remembered as being true. This is exactly what the Spinozan view would predict – interference leads to acceptance by default – and it is consistently what we see in these kinds of experiments.

(It’s worth mentioning here that there’s been some recent pushback¹⁰ against this particular line of research: many of the memory asymmetry experiments (like the one mentioned above) use meaningless propositions, but there’s some initial evidence that the relevance and prima facie plausibility of a proposition may mediate these effects. Using the same paradigm as mentioned above, if a proposition is perceived as being implausible – e.g. in England, Mince pies cannot be eaten on christmas day – people often misclassify true statements as false rather than false statements as true. This suggests we might have something like an intuitive plausibility filter. If we have no intuition about the plausibility of a proposition, the default is to classify it as true; if we do have intuitions - which we often do in real-life contexts - then plausibility influences default credence allocation. This isn’t settled science yet, but it does raise some interesting questions for the model.)

Another fascinating line of evidence in support of the Spinozan theory and against the Cartesian theory relates to belief perseverance, i.e. the tendency of beliefs to persist in the face of disconfirming evidence.

Consider this experiment from Ross et al¹¹: Participants were given twenty five pairs of suicide notes and asked to sort the real ones from the fake ones. Immediately afterwards, they – the participants – were given feedback about how well they’d done on the task, i.e. how accurate they had been in identifying the real vs. fake suicide notes.

Here’s the catch, though: the feedback they received was completely fabricated and bore no reflection on their actual performance on the task. Shortly after the fake feedback had been delivered, participants were informed that this was the case, i.e. that the feedback was made up. They were then asked to guess how many of the twenty five real suicide notes they had identified correctly.

Turns out that people who had initially been given high scores via the fake feedback rated themselves as much more accurate, and that people who had initially been given low scores rated themselves as much less so. So, in other words, even though participants accepted the initial feedback as being meaningless, it still continued to exert a strong influence on their assessments of their own performance.

This is pretty wild in and of itself, but it’s not necessarily watertight evidence in support of the Spinozan theory. After all, it could simply prove that beliefs are sticky – that once we acquire them, it’s hard to shake them off. Thankfully for us, though, a followup session was conducted in which everything was kept the same except that the researchers moved the debriefing session to the start of the experiment rather than the end. So, in other words, participants were told at the outset that any feedback they would receive later on was going to be complete baloney.

Guess what happened?

The feedback continued to exert a significant influence on people’s assessments of their own performance. That is: even though people knew the feedback they were receiving was nonsense, it still impacted their self-assessments.

The Cartesian theory has a good deal of trouble explaining this result. It predicts that when the participants receive the feedback, they will entertain and then quickly reject it on the basis that it’s false.

The Spinozan theory, on the other hand, has much less trouble. It states that regardless of whether or not participants believe the feedback is legitimate, they will accept it by default. Once they have accepted it, it then has the power to influence their self-assessed performance.

(For the record, I think the Spinozan theory taken on its own only partially accounts for this result. After all, the Spinozan view would presumably predict that these participants would reject the feedback as soon as it came in. To fill in the gaps, I think we need an adequate theory of belief storage, which we will encounter in the next section).

One final piece of evidence that I wanted to cover before moving on (because it bears quite heavily on some stuff I’m going to talk about in the final section of this piece) relates to negation.

Recall from the section above that one of the core assumptions of the Spinozan model is that to negate a proposition is to reject it. To explain a little further, the negation of a proposition P is essentially just the proposition not-P. So, for example, the negation of the proposition ‘God is real’ would be ‘God is not real.’ The Spinozan theory states that in order to accept the proposition ‘God is not real’ we must first accept the proposition ‘God is real’ and then reject it.

As it turns out, we’ve got some pretty smart ways of validating this prediction.

Take this study from Hasson and Glucksberg¹²:

Participants were shown a number of affirmative and negative assertions – for example, ‘The kindergarten is a zoo’ or ‘Lawyers are not sharks.’¹³

After reading each of these statements, participants were then asked to perform a lexical decision task, which basically involves showing them a string of letters on a screen and asking them to say, as quickly as possible, whether the letters spell an english word or not.

Before I can explain the result of this experiment, I just need to drop in a few lines about priming (apologies if you know all of this already!). Priming is a phenomenon whereby exposure to one stimulus impacts a person’s response to a subsequent stimulus.

So, for example, if I show you the word ‘dog’ and then subject you to a lexical decision task (like above), you will be quicker to respond to the word ‘dog’ than you would be to the word ‘car’. This is an example of direct priming. Likewise, if I show you the word ‘dog’, you would also be quicker to respond to semantically related words like ‘wolf’, ‘collar’, ‘bone’, etc. This is an example of semantic priming.

There are a number of different types of priming. Some of the more baroque forms don’t replicate, but many of the more commonplace ones – like direct and semantic priming – do.

The idea behind priming – so we think, anyway – is that the initial stimulus – the prime – activates a mental representation in the mind. This representation and its associated network of representations are still partially active when the second stimulus – the target – is presented, which means less additional activation is needed for the subject to become consciously aware of the second stimulus – hence the reduced response time.

Anyhow, returning to the experiment at hand: it turns out that if you show someone an affirmative statement, e.g. ‘Surgeons are butchers’ they become primed for, i.e. respond more quickly to, the negative-related word ‘clumsy’ but not for the positive-related word ‘precise’.

If, on the other hand, you expose them to a negation, e.g. ‘Surgeons are not butchers’, they become primed for BOTH the negative-related word ‘clumsy’ and for the positive-related word ‘precise’.

This finding gestures to the fact that in the case of negations, both the affirmation and the negation are being processed – hence why both negative- and positive-related words are being primed for.

So, yeah, basically, another dub for the Spinozan theory.

This section is already pretty long, so I’m going to cut it off here, but it’s worth mentioning that the Spinozan theory also offers up pretty convincing explanations for the ‘mere possibilities’ version of confirmation bias and of anchoring too. For those interested, I’ve included a few paragraphs about this in the appendix.

How we store beliefs

Now that we’ve talked through how beliefs are initially formed, we’re going to move on to how they’re stored. Much like belief formation, this turns out to be a good deal less intuitive than you might expect.

One view of belief storage, famously defended by Willard von Quine, is sometimes called the Web of Belief view. It goes like this:

1. Beliefs are stored in a single database;

2. Beliefs within the database are interconnected (like a web) and are consistent with one another;

3. When a belief changes, all other beliefs adjust to remain consistent with the modification;

4. Each belief is only represented within the database once.

What’s more, beliefs that are least likely to need changing – e.g. necessary beliefs like 2+2=4 – are at the centre of the web, while beliefs that are most easily falsified – e.g. sensory beliefs like ‘that boat is red’ – are at the outside of the web.

Speaking for myself, much of this lines up quite closely with my own intuitions about how I would expect beliefs to be stored in my mind – but there are some pretty glaring problems with it. For one, it assumes that all of our beliefs are consistent with each other, which is obviously just not the case. Anecdotally, we’ve all encountered people whose beliefs on a specific subject fall well short of perfect logical consistency. I’m sure most of us have even encountered this kind of thing within ourselves too. A paradigm example comes from the philosopher David Lewis, who famously believed that:

• Nassau Street ran roughly east-west;

• The railroad nearby ran roughly north-south;

• The two were roughly parallel.

Looking beyond anecdotal accounts, the experimental literature is rife with examples of garish inconsistencies in beliefs too. One of the most interesting studies I came across while doing my research – which has been replicated¹⁴ and which I think bears out the Spinozan theory of belief too – was done by Johnson and Seifert¹⁵.

To begin, they gave participants a series of thirteen messages, presented one by one, about the cause of a warehouse fire. In the fifth message, participants were told that ‘cans of oil paint and gas cylinders’ were stored in the room where the fire had started. This information was then immediately retracted – they were told that there had in actual fact been no oil paint or gas cylinders in this room. After the retraction, the remaining eight messages were delivered without any further interference. Once all messages had been received, participants were given a questionnaire designed to ascertain their conclusions vis-a-vis the cause of the fire.

Turns out that 90% of participants made reference to ‘volatile materials’ in their responses, and that on average they made 4.9 (out of 10) inferences consistent with the retracted information (as opposed to the corrected information). Also, you might think that this result came about because participants had simply missed the retraction information, but this wasn’t the case: 90% of participants explicitly made reference to the retraction in their responses, and when those who didn’t were removed from the analysis, the results remained the same.

So, in other words, participants were making inferences about the cause of the fire using both the retracted information – which they acknowledged as being retracted! – and the corrected information. If we define belief as something like ‘the point at which our credence level for a proposition is high enough to make that belief available for inferential reasoning’, then it seems that these participants believed both P and not-P, i.e. that the oil & gas had been in the room where the fire started and also that it hadn’t been there.

So, given that the Web of Belief view is untenable to some extent or another, how are our beliefs stored?

The strongest theory I’ve come across – the one that best accounts for the experimental data – is known as the Fragmentation view of belief storage. It’s an improvement on the Web of Belief view in a number of ways, not least of which that it’s able to comfortably account for the kinds of inconsistencies just discussed.

It goes like this:

• Beliefs are stored in unique data structures, called fragments;

• Beliefs within fragments are consistent – when new contradictory beliefs are added, we work to maintain consistency within the fragment – but beliefs across fragments need not be consistent with one another;

• The same belief can be simultaneously represented within different fragments;

• Belief updating usually takes place within a single fragment at a time.

If we circle back to the Johnson and Seiffert experiment, then: the fragmentation view explains this result by suggesting that the retracted information and the corrected information were stored in different fragments. The retracted information may have been updated in some fragments but not in others – hence its perseverance later on when participants made inferences about the cause of the fire.

Looking beyond this particular finding, the fragmentation theory of belief storage is also able to account for some other phenomena that leave the Web of Belief theory scratching its head. One such phenomenon is known as the wisdom of crowds within effect. I’m sure you’ve come across the idea of the wisdom of crowds: if you ask 100 people to estimate how many sweets are in a jar, the average of their scores will generally be better than any individual estimate taken alone. But here’s something you may not know: the wisdom of crowds effect also seems to show up within individuals too. Consider:

Vul and Pashler¹⁶ found that if you ask people the same question several times (e.g. ‘how many airports are there in the US?’), the mean of their answers tends to be closer to the truth than any single answer taken alone. They also found that the longer you wait between each response, the closer the average gets to the truth.

This is one of those findings that seems too weird to be true – but the fragmentation view does an admirable job of explaining how it might come about. Essentially: different fragments are active while each answer is being given. Each of these fragments constitutes only a limited sample of the overall information in the mind. By leaving time between answers, we’re able to access different fragments each time we provide an answer, which means our average response is reflective of a larger body of information than any individual response taken alone.

Another line of evidence often used to support the fragmentation view (which I’m a little bit more dubious of¹⁷) comes from reinforcement learning:

Lets say, as with the standard classical conditioning paradigm, you teach a dog to associate the sound of a bell with food (such that the sound of the bell results in the dog salivating). Now lets say that you take the dog out of the context in which it learned this association, i.e. context A, and you move it into a new context, i.e. context B, where you extinguish the association (such that the bell sound no longer elicits salivation). If you then return the dog to context A, you will find that the association immediately returns, so that once again bell = salivation.

This is known as ABA renewal, and it turns out that this kind of renewal shows up in every conditioning paradigm in which it’s been investigated. (Also, if you leave enough time after extinction, the renewal can occur spontaneously, without any kind of intervention).

Again, fragmentation is able to explain these kinds of results much better than the Web of Belief view, since it proposes that returning to a previous context would reactivate the initial fragment within which the association was first encoded.

I’m sure the fragmentation view of belief storage is at least partially right, and it does a great job of explaining lots refractory experimental data that otherwise resist explanation. The only issue I have with it, I guess, is that it feels like a bit of an overcorrection: I can readily accept that many of my beliefs are inconsistent with one another, and I think the fragmentation view does an admirable job of explaining this fact. But it also doesn’t seem reasonable to say that large bodies of my beliefs are completely cut off from one another, and that only one fragment can be active at any given time. If this were the case then wouldn’t I basically just become a different person, with a completely different set of beliefs, from one moment to the next depending on whatever fragment is active at each time?¹⁸

So, yeah, basically, while the fragmentation view is definitely compelling, it feels like it needs a bit of fine-tuning (which I think the theorists behind it would accept?).

One proposal is that there are certain basic beliefs that, for some reason or another, exist within multiple fragments at once. That’s not to say that they are represented multiple times, in different fragments; rather, that they are represented only once, but in a way that shows up in multiple fragments. This would allow you to activate different fragments while certain basic beliefs, such as your name, your age, your political and religious beliefs, etc., remain available and consistent at all times. I’m imagining this almost a bit like a mountain range, where individual mountains share a common base but taper off and become disconnected from one another above a certain level.

Another idea is that maybe there are multiple fragments that are active at any given time, which allows them to communicate with one another and share beliefs. One or two of these fragments may even be perpetually active.

Of course, this is just blind speculation from someone who has only racked up maybe thirty hours with his head in this literature, so take it with a pinch of salt, but maybe there’s something there.

How we update our beliefs

One of the most popular accounts of belief change comes from Bayesianism. This view states that when we receive new evidence for or against one of our existing beliefs, we update our credence level in line with this new evidence.

Up to a point, it’s hard to argue against this:

If I believe that a new friend supports Arsenal but this friend then turns up to my house wearing a Man United shirt and tells me that he’s a lifelong United fan, I’m going to update my belief in line with this evidence.

Another example: I catch a glimpse of a car out of the corner of my eye. I’m sure it’s orange, but when I take a closer look, I realise it is in fact yellow – so I update my belief in line with the new evidence.

This is all pretty straightforward and obvious, and the claim of the bayesian view is that this is the way we update all of our beliefs. Unfortunately, as has been a recurring theme throughout this piece, there are plenty of experimental findings that this theory does a horrible job of explaining. Take, for example, belief perseverance:

In one study¹⁹, researchers asked participants to form a theory about the relationship between firefighter performance and risk aversion vs. risk seeking. In essence: are the best firefighters risk averse or not? Some participants were asked to form their theory using things like fictitious case histories and fake data. Others were merely assigned a given relationship and asked to contemplate why it was true. In the next part of the study, participants were given counter-attitudinal evidence, i.e. evidence against the view that they had formed in the first part of the experiment.

The Bayesian view predicts that participants will adapt their position in line with the new evidence, but this is not what we see at all: more or less all participants showed a tendency to stick with their original position in the face of the counter-attitudinal evidence – even if the position was arbitrarily assigned to them and regardless of how this counter-attitudinal evidence was presented. This is the opposite of what the bayesian account would predict.²⁰

Another well-replicated finding that runs in the complete opposite direction to the bayesian view comes from research on belief polarisation. Consider:

In one study²¹, researchers started out with two groups of participants: one group believed that Jesus was the son of God, the other didn’t. All participants were then presented with an article that they were told had been suppressed by the world council of churches because it had definitively proven, through various esoteric historiographical techniques, that the writings of the new testament are fraudulent. Once they’d read the article, participants were asked to 1) say if they believed the article was legitimate or not; 2) complete a test that tracked how their attitudes had changed since reading the article.

The tests revealed that:

• Those who didn’t believe that Jesus was the son of God but did believe that the article was legit, generally increased the strength of their initial belief. No major surprise there.

• Those who did believe that Jesus was the son of God but who didn’t believe that the article was legitimate simply kept the strength of their initial belief constant. Again, kind of what you’d expect.

• Here’s the wild bit, though: those who did believe that Jesus was the son of God and who also believed that the article was legitimate tended to INCREASE the strength of their initial belief, i.e. that Jesus is the son of God.

So, just to repeat:

In this final group, participants accepted the evidence as legitimate and then updated their beliefs in the opposite direction of said evidence! This is the reverse of what the bayesian view would predict. Moreover, this finding isn’t just a one off; it shows up over and over, e.g. in relation to beliefs about:

• JFK’s assassination²²

• Affirmative action and gun control²³

• Nuclear power²⁴

• Birth control²⁵

• & more

What are we supposed to make of this!?

Well, again, many smart people have put a lot of thought into how best to understand these results, and the strongest explanation is known as the Psychological Immune System.

From decades of dissonance research, we know that people are put into an aversive state when they receive evidence that disconfirms their existing beliefs. This may, incidentally, be why confirmation bias is a thing: because we learn, via operant conditioning, that disconfirming evidence = unpleasant feelings, so we go out of our way to avoid or reinterpret said evidence in ways that allow us to escape this aversive state.

The idea of a psychological immune system pushes this insight further by claiming that when we encounter evidence that disconfirms beliefs we self-identify with, this puts us in an extremely aversive, dissonant state. In order to get out of this state, we choose to reaffirm our antecedent beliefs rather than entertain new beliefs that have the potential to compromise our sense of self.

The self-identification part of this is important: like the biological immune system, which isn’t set off by every minor infection, the psychological immune system is only set off by disconfirming evidence that challenges strongly held beliefs that are subjectively important in some way. The more we self-identify with a belief, the more subjectively important it is, and therefore the more likely the psychological immune system is to be activated in relation to it. In such cases, belief updating isn’t about tracking the truth as faithfully as possible; it’s about avoiding dissonance.

So, returning to the study above:

People who believe in Jesus qua son of God and who also accept the evidence as legitimate are put into an extremely aversive state. When this happens, the psychological immune system becomes active, which results in the antecedent belief being reaffirmed.

Bringing this all together, then:

As far as I can work out, belief updating is basically governed by three main processes.

The first, which applies to beliefs that we hold relatively dispassionately, proceeds roughly along bayesian lines. The second, which applies to beliefs that we self-identify with, is governed by the psychological immune system.

And in between these two extremes, it seems like we update our beliefs via a combination of bayesian updating and confirmation bias. There’s some evidence²⁶ that the more confident we are in a belief, the more potent confirmation bias is likely to be – so it may well be that bayesian updating grades into confirmation bias as credence levels increase.²⁷

What should you make of all of this? Some discussion

In a previous post of mine about dopamine, I presented tons of information but I never actually spelt out what that information meant, i.e. how people can use it in their own lives.

My writing generally is about how we can all live good – or at least, better – lives, so here’s an attempt to draw out some (slightly speculative) takeaways from all of the information presented thus far.

If you think I’ve missed anything – or if you think my read on the research is wrong – drop a comment below and if I think it’s fair, I’ll do my best to update accordingly (assuming you’ve not hit upon anything that I self-identify with, in which case I’ll probably just double down!).

So – what should you make of all the information that has been presented thus far?

1. The way we form, store, and update beliefs is a good deal less ‘rational’ than you might expect – but maybe let’s not get too carried away just yet

I guess the main takeaway from all of this is that the way we form, store, and update our beliefs falls pretty far short of optimal rationality:

We can’t help but accept newly encountered propositions, regardless of their plausibility – and the process by which we reject them is vulnerable to all kinds of interference.

Once beliefs are held, confirmation bias and the selective exposure effect²⁸ ensure that we interpret new data in biased ways and that we actively avoid disconfirming evidence where we can.

The fragmentation view of belief suggests that it’s perfectly possible – nay, common – to hold contradictory beliefs at the same time and for those contradictory beliefs to feature in inferential reasoning without our even realising it.

And as if that weren’t bad enough, the idea of a psychological immune system suggests that when our beliefs are closely tied to our identities, we may actually update them in the opposite direction to that of disconfirming evidence.

What a shitshow.

One thing I would say, however, is that maybe we shouldn’t get too carried away with all of this stuff. It seems like much of psychology for the past couple of decades has been about debunking the idea that human beings are rational. Part of this has come in the form of the cognitive bias research, and part of it has come in the form of the ‘situations matter’ research, i.e. subtle things in our environments that influence our minds in BIG ways, e.g. things like age priming and power poses.

But as we now know, much of the research in this second category is bunk. This means that while we humans may be less rational than initially thought, we’re also probably not completely irrational automatons either. I wouldn’t be surprised if a similar, though less extreme, arc unfolds here:

Much of the picture presented above may well hold up – it certainly feels less breathless than the ‘situations matter’ research – but I also wouldn’t be surprised if we find that there are limitations and qualifications that need to be introduced. The plausibility objection to the Spinozan model suggests one place where this might need to happen. Likewise the fragmentation view of belief storage (at least as i understand it) feels like it needs a bit of work.

2. Minimise cognitive load

The section on belief formation indicates that we acquire beliefs effortlessly, but that the process by which we reject them is easily compromised when we’re under cognitive load. We don’t know exactly how much cognitive load is too much, i.e. enough to interfere with our belief rejection processes, but I think we can probably draw out some fairly sensible speculation:

It seems likely that we as a species are under more cognitive load today than we have ever been before: phones are constantly providing us with notifications demanding our immediate attention; we are plugged into the internet during every waking moment of our lives; none of us can go anywhere without a podcast or some kind of music ringing in our ears; cars, buses, trains, and planes are constantly rushing by, etc.

As far as belief hygiene goes, this can’t be good – and it might explain why more people than ever before subscribe to wacky ideas and conspiracy theories. Maybe we’re under so much cognitive load that the processes by which we would ordinarily reject beliefs have been more or less taken offline?

This may be a stretch – although it might not – but I think aiming to minimise cognitive load could nonetheless be a good epistemic best practice if we’re looking to defend against goofy propositions worming their way into our psyches without our say-so.

(If you’re anything like me, you’ll find this happening all the time now that you’re on the lookout for it!)

3. Keep your identity small

Paul Graham wrote an essay a while back where he asked the question ‘why do some debates devolve into acrimony while others seem to be genuinely aimed at finding truth?’

His conclusion – which, in light of the research discussed above about the psychological immune system, seems to be right – is that some debates relate to subjects that people tend to self-identify with (e.g. politics and religion), while others don’t (e.g. the best lasagne recipe).

As he puts it:

Which topics engage people's identity depends on the people, not the topic. For example, a discussion about a battle that included citizens of one or more of the countries involved would probably degenerate into a political argument. But a discussion today about a battle that took place in the Bronze Age probably wouldn't. No one would know what side to be on. So it's not politics that's the source of the trouble, but identity. When people say a discussion has degenerated into a religious war, what they really mean is that it has started to be driven mostly by people's identities.

His takeaway is that we ought to try to keep our identities as small as possible, and again, in light of this research, this seems like sensible advice. If we’re looking to arrive at truth as reliably as possible, we need to make sure our identities don’t get in the way.

Saying that, I have two potential qualms with this:

First, how do we go about shrinking our identities? I don’t really know. Presumably we have to identify with something, so maybe we should just identify as people who don’t identify with their beliefs!?

Second, there are certain types of beliefs that seem to lend themselves more to self-identification than others. For example, how would I stop myself identifying with a belief like ‘God exists’?

A belief like this would be pretty central to the way I live my life, the values I adopt, my model of the world, etc. If a belief runs through every area of my life, I’m probably going to self-identify with it to some extent or another. Consistency probably demands that I do.

It might be the case that the reason we tend to self-identify with specific religious and political beliefs is that they ramify widely and necessarily cut across many areas of our lives in profound ways. If this is true, then the advice about keeping our identities small may be a bit pat without concrete guidance on how exactly to do this. Hmm.

4. Affirmations might be a thing – also, watch out for maladaptive beliefs

This point might sound a bit wooey, but bear with me a minute and I’ll try to make my case. This is probably the most speculative thing I’m going to write, but I think it could also be the most interesting too.

The Spinozan model of belief suggests that we assent to beliefs effortlessly, but there’s another well-documented finding in this literature, known as the illusory truth effect, which suggests that the more we’re exposed to a proposition, the higher our credence level for that proposition goes.

For example:

• Seeing a sentence twice makes people more likely to judge it as true.²⁹

• This effect holds up even for sentences that are clearly labelled as false or that are obviously false.³⁰

• This effect holds up for beliefs that we antecedently know to be false, i.e. if you repeatedly expose people to the sentence ‘the atlantic ocean is the largest ocean in the world,’ their credence level for this proposition will increase even if they came into the experiment knowing this proposition to be false.³¹

• Cognitive ability and style do not appear to exert any moderating impact on the illusory truth effect.³²

• One study even showed that credence levels continue to rise with up to 27 repetitions of a proposition, although the credence function was logarithmic, i.e. the first repetition produced the largest increase in credence, and credence increases got progressively smaller from there.³³

On the face of it, these findings are quite concerning – particularly as they relate to things like fake news and propaganda – but I also think they open up some space for us to tinker with our beliefs in ways that work in our favour. Think back to the example from the intro. If I believe that everyone hates me, I’m probably going to have a pretty miserable life.

But if I believe that everyone likes me, even if this isn’t strictly true, my life is probably going to be a good deal better. I’ll probably interpret ambiguous social cues in my favour; I’ll probably be more relaxed and less defensive around others; I’ll probably make friendships more easily. In fact, I’ll probably be genuinely more likeable, since I’ll be less up tight, less suspicious, etc.

Previously, you might have felt justified in responding: ‘well, yeah, sure, that’s all good and well, but I don’t choose what I believe.’ But in light of the research on belief formation and the illusory truth effect, it actually looks like we can exert some control over what we believe. By repeatedly affirming certain salutary beliefs, e.g. that everyone likes me, I may actually be able to push my credence level high enough to activate confirmation bias and begin using this belief in automatic inferential reasoning (as in the warehouse fire example).

Assuming this speculation is right – and, of course, it might not be – the next question we’ll need to ask ourselves is ‘what propositions should I believe?’ I have a few thoughts on this subject, but they’re not really based on anything other than hunches, so I’ll leave this one with you to decide for yourself.

5. Watch out for what you believe

Since first diving into this literature a while back, I’ve been on the lookout within my own mind for the phenomena laid out in this piece. While doing so, the thing I’ve found most striking – and most concerning – is this:

We are constantly absorbing beliefs – propositions are constantly being added to our databases – and many of these beliefs, when examined, turn out to be absolutely baseless. When I first encountered the Spinozan theory of belief formation, it felt intuitively wrong – but now that I’m on the look out for it, I see its effects all the time.³⁴

To give one of many examples:

The other day I went out for a Chinese with some friends. We’d had a few drinks, conversation was lively (read: potentially load-inducing), and at the end of the meal, I received a fortune cookie that said something to the effect of:

‘Many surprises are just around the corner.’

Now, I’m not a believer in fate. I know that this restaurant has a big old pile of fortune cookies in the back room; that they were were probably unlovingly mass produced in a UK-based fortune cookie factory; that they were not sent down by an all-knowing power to sign-post the future for me.

I know all of that.

And yet the next day, I found myself sitting at my desk, feeling kind of excited and expectant about the future. When I tried to trace this feeling back to its source, I realised that the belief at its root was the one that had been presented to me in the fortune cookie:

‘Many surprises are just around the corner.’

The belief had wormed its way into me, it had completely bypassed my rejection machinery, I was unconsciously using it to make inferences – and it was influencing my mood!

I’ve observed this kind of thing happening multiple times, in various different ways, over the last month or so. In light of some of the research presented in this piece, I wouldn’t be surprised if it’s happening much more often than I’m even aware of. So, with this little anecdote in mind, here’s my parting advice:

Question your beliefs. Ask where they came from. Some of them will be well-founded, backed by evidence, arguments, probabilities, etc.

And others will be absolutely, laughably baseless.

This is fine and probably unavoidable – not every belief can be well-supported – but remember: the beliefs you hold literally structure your reality. If you have to hold baseless beliefs, let them be baseless beliefs that make that reality better.

Appendix

1. What is a belief?

In everyday speech, we seem to treat belief as a binary phenomenon. Either I believe in God or I don’t. Either I believe you are telling the truth or I don’t.

It’s all or nothing. On or off. Yes or no.

This kind of definition serves the purposes of everyday conversation, but I think it fails to capture the more considered intuitions that many of us have about how our own beliefs work.

An increasingly popular view – which has apparently been around for as long as people have been thinking about these issues – is that belief is a graded phenomenon. It is possible to believe in something more or less strongly. It is possible to believe in one thing more strongly than you believe in another. And along the same lines, it’s possible to be agnostic, with a slight leaning in the direction of belief or disbelief.

Under this view, we can think of degrees of belief – or credence levels – in terms of numerical values and probabilities. I believe that God exists with 80% confidence. I believe that you are telling the truth with 65% confidence.

Coming up with specific numbers may seem a little bit unscientific, but it at least gives us a means of quantifying our intuitions and comparing the strength of different beliefs.

But one of the challenges with this graded view is that it forces us to define, or at least to question, the point at which an attitude towards a proposition hardens into belief. Is it at 75% credence? 80%? 95%?

There are a number of different responses that we can go with here, and I’m not sure how we would adjudicate between them, really.

One approach is to say that for a proposition to achieve belief status, it has to reach a credence level at which it becomes available for inferential reasoning. To give an example: let’s say I’ve been fed lots of conflicting information about the cause of a warehouse fire. Maybe I’ve been told that

• a) there were highly flammable ingredients stored in the room that the fire started in.

• b) there were actually no highly flammable ingredients stored in the building at all.

Now, if you ask me a question like ‘why did the fire burn so quickly’ and I say things like ‘oil fires are hard to put out,’ then clearly I’m making inferences based on a).

Under this view, then, I could be said to believe in a) and to not believe in b).

This seems like quite a sensible approach, but as it turns out, its possible – maybe even common – to simultaneously (and unwittingly) make inferences based on mutually exclusive propositions like a) and b). This warehouse fire example was borrowed from an experiment I talk about later in the Belief Storage section of this piece – head there if you’d like to learn more.

Another approach to defining belief – and the one we’re going to roll with here, since it’s the preferred definition for a number of papers I’ve been drawing upon³⁵ – is that a proposition becomes a belief when it reaches the level of credence at which it is able to produce behaviour.

So, to give an example:

Let’s say I’m sat on my couch and I want a beer.

Now consider the proposition: there is a beer in the fridge. If my credence level is at 0.01%, I’m probably not going to get up and search the fridge. If, on the other hand, my credence level is at 80%, I’ll probably beeline for the fridge and begin rifling around.

This is kind of an interesting – and maybe problematic – definition, because by its lights, belief is influenced by other factors beyond credence alone. For one, how much do I want the beer?

For another, how easy is it to falsify my belief? My confidence level may be 20% for the proposition ‘there is a beer in the fridge’, but if it only takes me 3 seconds to falsify this hypothesis, I’m probably just going to check the fridge before I resign myself to a trip to the shops.

If, on the other hand, my confidence level is 20% that God exists, I’m probably not going to start praying or heading to church on a Sunday.

So, in one instance, 20% credence = belief and in the other, 20% credence = not belief.

Anyhow, while interesting, this is one rabbit hole I don’t intend to go down right now. All you need to know here is that throughout this piece – unless stated otherwise – the word belief means ‘the point at which credence is high enough to produce behaviour.’

Capisce?

2. Spinozan theory - the mere possibilities version of confirmation bias and anchoring

Mere possibilities version of confirmation bias - If you ask people if they are happy, they will generally say that they are. If you ask people if they are unhappy, they will likewise generally say that they are. The Spinozan theory suggests that when people are asked if they are happy, they entertain the proposition ‘I am happy’, which automatically results in the proposition being accepted. Once the proposition is accepted, confirmation bias becomes active, meaning they start sifting and searching for evidence in ways that support the proposition in question. This results in an affirmative answer being produced.

Anchoring – let’s say you ask someone how old Ghandi was when he died, but before they give their answer, you spin a wheel that generates a completely random number. The higher the number the wheel generates, the higher the participants’ estimates will be. The lower the number, the lower their estimates will be. This is pretty weird. The Spinozan theory explains this by suggesting that the randomly generated number is automatically accepted as the answer to the question. From here, it is then able to exert an influence on participants’ responses (perhaps by acting as one of the many datapoints on which they base their estimate).

1. You can the make the case, a la hume, that beliefs alone are not enough to generate motivation & behaviour – that we need desires + beliefs to do this – but this doesn’t change the fact that beliefs are a key part of this equation.

2. Bingel et al (2011) - The effect of treatment expectation on drug efficacy: Imaging the analgesic benefit of the opioid remifentanil.

3. Gu et al (2015) - Belief about nicotine selectively modulates value and reward prediction error signals in smokers

4. Kufahl et al (2008) - Expectation modulates human brain responses to acute cocaine: A functional magnetic resonance imaging study.

5. Gundersen et al (2008) - Separating the effects of alcohol and expectancy on brain activation: An fMRI working memory study.

6. Muckli et al (2015) - Contextual Feedback to Superficial Layers of V1 Report.

7. Named after the philosopher Renes Descartes, who seems to have held a view that roughly resembles something like this.

8. Named after the philosopher Baruch Spinoza, who seems to have held a similar view.

9. GIlbert et al (1990) - Unbelieving the Unbelievable: Some Problems in the Rejection of False Information

10. Vorms et al (2022) - Plausibility matters: A challenge to Gilbert's “Spinozan” account of belief formation

11. Ross et al (1975) - Perseverance in Self-Perception and Social Perception: Biased Attributional Processes in the Debriefing Paradigm

12. Hasson and Glucksberg (2006) - Does Negation Entail Affirmation? The Case of Negated Metaphors

13. All of the statements were metaphors because this apparently stops regular semantic priming effects from interfering with the data.

14. Laurent et al (2023) - I know It's false, but I keep thinking as if it were true: A replication study of Johnson and Seifert's (1994) continued influence effect

15. Johnson, H., and Seifert, C. (1994), ‘Sources of the Continued Influence Effect: When Misinformation in Memory Affects Later Inferences.’

16. Vul, E., & Pashler, H. (2008). Measuring the crowd within: Probabilistic representations within individuals.

17. I suppose you could make the case that basic beliefs about, e.g. my name, my identity, etc., are represented in every single fragment, but this doesn’t feel like the most economical way of structuring things. It would basically mean that these basic beliefs are tokened literally thousands or even millions of times across different fragments.

18. I suppose you could make the case that basic beliefs about, e.g. my name, my identity, etc., are represented in every single fragment, but this doesn’t feel like the most economical way of structuring things. It would basically mean that these basic beliefs are tokened literally thousands or even millions of times across different fragments.

19. Anderson, C. A. (1983). Abstract and concrete data in the perseverance of social theories: When weak data lead to unshakeable beliefs

20. I think we can explain this quite easily through recourse to the Spinozan view of belief formation + confirmation bias. In essence: by entertaining a given position – even if it was arbitrarily assigned to us – we come to accept it. Once we’ve accepted it, confirmation bias becomes active, at which point iwe begin interpreting incoming information in a way that supports our initial position.

21. Batson, C. D. (1975). Rational processing or rationalization? The effect of disconfirming information on a stated religious belief.

22. McHoskey (1995) - Case closed? On the John F. Kennedy assassination: Biased assimilation of evidence and attitude polarization.

23. Taber & Lodge (2006) - Motivated skepticism in the evaluation of political beliefs.

24. Plous (1991) - Biases in the assimilation of technological breakdowns: Do accidents make us safer?

25. Kiesler (1971) - The psychology of commitment: Experiments linking behavior to belief.

26. Rollwage et al (2020) - Confidence drives neural confirmation bias

27. Saying this, i’m not sure how well this sketch accounts for belief perseverance as it shows up in the firefighter/risk aversion experiment, so we may need a bit more here if we want to develop a high-fidelity picture of how this stuff works.

28. Another manifestation of confirmation bias, the selective exposure effect is basically where we go out of our way to avoid disconfirming evidence.

29. Hasher et al (1977) - Frequency and the conference of referential validity.

30. Fazio et al (2019) - Repetition increases perceived truth equally for plausible and implausible statements.

31. Fazio et al (2015) - Knowledge does not protect against illusory truth

32. Keersmacker et al (2020) - Investigating the robustness of the illusory truth effect across individual differences in cognitive ability, need for cognitive closure, and cognitive style

33. Hassan and Barber (2021) - The effects of repetition frequency on the illusory truth effect

34. This, of course, doesn’t mean that it’s true; i may well be bending reality to my ideas.

35. Bendana and Mandelbaum (2021) - The fragmentation of belief

Mandelbaum (2014) - Thinking is believing

Mandelbaum (2021) - The science of belief

Mandelbaum (2018) - Troubles with bayesianism: an introduction to the psychological immune system