These
types of error are typically investigated by asking test persons to form judgments
about various ficitious situations. It seems to me that in some cases one may question
whether the tests actually reveal what they are taken to reveal. They suffer
from what might be called the distance between class-room dialogues and real conversations.
(On this compare my earlier blog on experimental philosophy.)
The
following example comes from the work of Amos Tversky and Daniel Kahneman, two
leading proponents of this branch of research (quoted in the Wikipedia article).
Test persons were given the following character sketch of a woman called Linda:
"31 years old, single, outspoken, and very bright. She majored in
philosophy. As a student, she was deeply concerned with issues of
discrimination and social justice, and also participated in anti-nuclear demonstrations".
They were then asked to rank the probability of certain facts about Linda,
among others "Linda is a bank teller" and, "Linda is a bank
teller and is active in the feminist movement".
Test
persons tended to consider the latter description more probable. The answer is
wrong, we are told, because a single assertion of the form “L is X” is always
more probable than a conjunction of the form “L is X and Y”. The error supposedly
shows a cognitive bias (of the kind called the “conjunction fallacy”). What
trips people up, apparently, is the fact that the description of Linda is more
likely to fit someone who is an active feminist than someone who is a bank
teller.
However,
it seems to me that that is not the relevant comparison. We are told that Linda is a bank teller, so
the likelihood of her being one plays no role in the context. There seems to me
to be a more plausible explanation: if we are given the alternatives (1) “Linda
is a bank teller FULL STOP” and (2) “Linda is a bank teller and active in the
feminist movement”, the natural way to read (1) is as implying that Linda is not active in the feminist movement. So
what we will actually be comparing, in all likelihood, are the descriptions “Linda
is a bank teller but she is not active in the feminist movement” and “Linda is
a bank teller and she is active in the feminist movement”. In other words, her
being a bank teller drops out of consideration altogether: we are simply comparing
the likelihoods that she might or might not be an active feminist; and, given
her background, the latter comes to sound a fairly plausible alternative. (One
might even think that, given that she is a bank teller, she will need other
outlets for her civic involvement; maybe she even chose her trade in order to to
have time for political activities.)
Undoubtedly
there are such things as cognitive biases, and some of the tests carried out in
this field might accordingly be revealing. But often a kind of smart-alecky attitude
shines through in the manner ordinary forms of reasoning are being shown up, as
though the only way of judging a claim is in accordance with the principles of propositional
logic or probability calculus. This makes for a deafness to the endless subtleties
of actual human conversation. That is a serious drawback, for it is, after all,
the reasoning of people in everyday contexts that one purports to be studying.
This was very interesting for me. I have been increasingly interested in judgemental heuristics in recent years, and I have also had exactly the same misgivings as you express (without these misgivings threatening my interest in any way, however).
ReplyDeleteWhat frustrates me about experimental philosophy more generally is that its experiments are so often superfluous. By this I mean that the facts the experiments are designed to find out (or relevantly similar facts) are often already known, only as a by-product of empirical research into something completely different. On most phenomena of social psychology, we already have good empirical data that were originally collected by someone with a different knowledge-interest (to use the Habermasian word) than that of researchers like Kahneman and Tversky. And this gives rise to a suspicion that the reason why they are designing their own experiments exactly to their personal taste is because they then get to sound as "smart-alecky" as possible. (I am using Kahneman and Tversky as a metonymy for students of judgemental heuristics more generally.)
In other words, I have nothing against the motivational background of experimental philosophy. For instance, I would not accuse it in a Wittgensteinian tone of voice of suffering from conceptual confusion. But we simply already have accumulated enough empirical knowledge about social psychology to enable us to do our experimental philosophy on its basis, and even better, the purposes for which these data were originally collected are different enough from Kahneman and Tversky's to prevent the data from being "contaminated" by their need to be smart-alecky.
That’s interesting. As I was arguing there need be nothing wrong about the idea of studying human thought from an empirical point of view, as long as one is clear what one’s questionnaires are actually testing. Are you writing about this?
DeleteWhat I find confused about experimental philosophy is the idea that it can be used to clear up one’s own philosophical confusions. On the other hand, when it is devoted to the study of human habits of thought, from a third person perspective as it were, it can be illuminating (say, about the way those habits differ between different ethnic or social groups) – provided it is well done. But I would suggest the term “experimental philosophy” is a misnomer for this type of activity.
This comment has been removed by the author.
ReplyDeleteSo, I agree with your Wittgensteinian criticism of Kahneman and Tversky. But there is a related thing about Wittgenstein himself that I also have some misgivings about – and interestingly enough, I just thought of it earlier today after a long interval.
ReplyDeleteAs we all know (especially you as the author of a paper of the same name!), Wittgenstein repeatedly appeals to "very general facts of nature": that figures written with ink on paper do not alter erratically when one is making mathematical calculations (Remarks on the Foundations of Mathematics, VII, §61); that "the eye-piece of even the hugest telescope cannot be bigger than our eye" (Culture and Value, 1998 ed., p. 25); that human individuals' personality traits cannot suddenly leave their physical body and travel to another physical body (Blue Book, p. 61); and so on. But, as far as I know, the "very general facts of nature", while very often psychological, are never social-psychological.
Wittgenstein writes: "A curious analogy could be based on the fact that the eye-piece of even the hugest telescope cannot be bigger than our eye." But we cannot imagine him writing: "A curious analogy could be based on the fact that [insert one of Kahneman and Tversky's research results]". Not even when the research result is mere experimental confirmation of something we already know more vaguely from our everyday experience – which would make it suitable for serving as a Wittgensteinian "reminder" of something that has "escaped remark only because [it is] always before our eyes" (Philosophical Investigations §415).
I see no distinctively Wittgensteinian reason for declining to include social-psychological facts, as one category of psychological facts among others, in the "very general facts of nature" which Wittgensteinian philosophy invokes. But Wittgenstein himself seems to have seen one. In your above mentioned paper, you write that "the importance, for our way of thinking about language, of recognizing the ways in which the language we speak is contingent on the circumstances of our lives" is a Wittgensteinian "dimension that has largely been bypassed or underplayed (perhaps even [...] by Wittgenstein himself)". I see this as one way in which he did underplay it.
I guess this depends on how you define the notion of social-psychological facts. Wittgenstein does appeal to *interactive* facts, e.g. the fact that a child who hurts herself will respond in ways that others will recognize as expressions of pain. Or (in *On Certainty*) that the child will believe what adults tell her.
DeleteWhat I meant by a social-psychological fact, I guess, was a fact about those beliefs or attitudes that are themselves beliefs about society or attitudes to society (e.g. the bank teller example). Practically none are mentioned by Wittgenstein.
DeleteBut in fact I later remembered one place where Wittgenstein at least comes close. It is in the lectures on the freedom of the will, where he discusses the fact that the same people may express both determinist and voluntarist attitudes on moral responsibility, depending on the rhetorical framing of the issue at hand. This is an example of a so-called framing effect, which is well documented empirically and which even Kahneman and Tversky themselves have researched. But I can't think of any other examples.
I'm not writing on this thematic myself, although I'd like to read about it. Somehow I have been waiting for somebody more qualified (Don S. Levi?) to come along and do it.
You suggest that in response to Kahneman and Tversky’s test, the subjects compared the likelihood of two cases: 1) Linda is a bank teller and a feminist activist, and 2) Linda is a bank teller and not a feminist activist. So, part of what you say is that Kahneman and Tversky are misdescribing what their subjects are doing.
ReplyDeleteNow, how important is the further claim to you: that the alternative description you give of what the subject were doing is true? How important to you is it that other descriptions are not true? (Other re-descriptions of what the subjects did in response to the test may be suggested: e.g. that they were trying to find a claim that would tell a good rounded story about Linda.)
I mean, I think your claim about the misdescription is very strong, and interesting in itself, and that it deserves the focus of attention. The other claim--the particular re-description you offer--seem to me less important. Because it does seem to me right that psychologists and other social scientists really don’t spend enough time asking themselves questions about the proper ways to describe what they are testing, the different variables. And they don’t take seriously enough how difficult it is to describe what people are doing, and thinking, and feeling. (Even the assumption that all the subjects did the same thing seems very questionable.)
I often get the sense, listening to the results of experiments in psychology that language is being abused. I seem to be unable to see what they claim to be studying (e.g. cognitive biases) in the actual studies--as if the studies give me no sense of the reality of what they are supposed to be studying. But if language is being abused in those studies, it is probably being abused in more than one way.
That’s an important point. When I wrote the blog it didn’t occur to me that there might be several ways of describing the responses of the test persons. But of course there are, and it doesn’t really matter which description is correct, or whether different descriptions may apply to different individuals, as long as there are alternative readings that are at least as plausible as the one offered by Tversky and Kahneman. (E.g. the idea that the test persons wanted to make up a well-rounded story about Linda.) This is in accordance with a central principle of empirical research: in order for an experimental result to confirm a hypothesis you need to be able (within reason) to eliminate alternative explanations of the result. This, it appears, is what Tversky and Kahneman have failed to do.
DeleteI sense that there is something further in what you say against Kahneman and Tversky. Namely, that their description of what the subjects did is implausible: that it is not one of the possible descriptions of what the subjects did. Kahneman and Tversky, if I am right about what you say, make their subjects into confused silly people who don’t know what they are doing. And that, if I understand, you think is wrong: It is not just a wrong description; the misdescription is revelatory of a wrong attitude. Would you make this distinction?
DeleteThis matter of attitude is also an ethical matter. At any rate, it is a question of will: an attitude that Kahneman and Tversky towards their subjects, an attitude of unwillingness to learn from the subjects, or limiting what can be learnt from them, imposing on what the subjects did categories that are foreign to them, not willing to learn (or to think from within) the categories, the language, in which they actually think and act. Unwillingness to look.
Also, to the extent that Kahneman and Tversky understand themselves as studying something general about human thinking, they are also bound to take the study to be about themselves. But if I’m right about what you are saying, then the failure to attend to the categories in which their subjects are thinking is also a failure to attend to themselves. And that’s a further failure.
Quite right! I think you've managed to deepen the point I was trying to make.
DeleteThe thinking of Kahneman and Tversky has interesting analogies with that of Soviet psychologist Aleksandr Luria, who studied the reasoning abilities of illiterate peasants in Uzbekistan, discussed by Don Levi, "Why do illiterates do so badly in logic?" *Philosophical Investigations* 19 (1996), 34-54.
Criticisms similar to yours were made more comprehensively (in relation to the whole heuristics-and-biases program) by L. Jonathan Cohen in "Can human irrationality be experimentally demonstrated?", Behavioral and Brain Sciences 4 (1981): 317-370. Cambridge Journals will sell you an electronic copy for a picayune US$45.00 or UK£30.00, or, if you Google it, you can (at the moment of writing) find a scan of an extensively marked-up copy in a PDF file. See also Gerd Gigerenzer, "On Narrow Norms and Vague Heuristics," Psychological Review 103 (1996): 592-596, of which a PDF can also be found on line.
ReplyDeleteThank you for the tips. They were very instructive. Evidently this debate has been going on for quite some time.
ReplyDeleteI’ll take this opportunity to quote a line from Terry Eagleton (*Across the Pond: An Englishman’s View of America*): “People with very small feet tend to know less about ancient history than people with larger feet, since toddlers are less knowledgeable about the topic than adults.”
The consequent would seem to express what on a certain view is considered excellent reasoning.
Excuse me, but where exactly are we told that Linda is a bank teller? Am I missing something?
ReplyDeleteBoth the alternatives in the passage quoted above state that Linda is a bank teller.
ReplyDelete(There may have been other alternatives in which she was not a bank teller, but in the two cases being compared here, she evidently is one.) It does look as if you missed something.