May 07, 2012

Fake questions and experimental philosophy

A few years ago an apparently new branch of philosophy was launched to a certain degree of media attention: experimental philosophy, or "x-phi " as it is called by some of its adherents. Alternatively, it might be called philosophy by questionnaire. (The movement has its own website.)

To what extent experimental philosophy is still thriving I cannot judge. The idea was not entirely new: it was presaged by Arne Næss's work in the 1950's. (See Siobhan Chapman's article "Arne Naess and Empirical Semantics".)

Experimental philosophers suggested that one might approach philosophical issues about various themes such as knowledge or moral responsibility by asking people how they would apply the relevant words in particular situations.

On the face of it, this may seem like a sound idea. Non-philosophers are often puzzled by the armchair nature of philosophical inquiry. Why do philosophers think they can simply pull the solutions to the problems of philosophy out of their own heads? On what authority do they claim to be able to decide what our words mean without checking with the rest of us?

These qualms, I would argue, are based on a misunderstanding of the nature of philosophy, of the way it differs, say, from linguistics. (In using the word "philosophy" I am referring to the type of activity commonly carried on in Western universities under that heading.) Philosophers are not concerned with making assertions about language use, but with clarifying the use we share. They are not making knowledge claims, but trying to achieve a clearer understanding about the matters that puzzle us. What makes this difficult is that we often do not have an overview of our own use. The point that philosophy is concerned with clarification rather than knowledge is not always fully appreciated by those who practise the subject. Experimental philosophy, I believe, is one expression of the failure to understand this point.

I shall try to illustrate what I mean. But first some background.

One issue that has occupied experimental philosophers are the so-called Gettier examples. A definition of knowledge that used to be largely accepted by philosophers is the following: if I am to be said to know something, what I believe must, for one thing, be true and, for another thing, I must have grounds for believing it to be true. Of course, I may have any number of correct beliefs that are just happy guesses, and hence wouldn't qualify as knowledge. That's how the thinking went. However, in 1963 the American philosopher Edmund Gettier thought up some counter-examples to the definition, by constructing cases in which one could have a correct belief, have grounds for the belief, and would still not be said to know. (Read his article.)

Here is one of his examples:
Suppose that Smith and Jones have applied for a certain job. And suppose that Smith has strong evidence for the following conjunctive proposition:

d. Jones is the man who will get the job, and Jones has ten coins in his pocket. Smith's evidence for (d) might be that the president of the company assured him that Jones would in the end be selected, and that he, Smith, had counted the coins in Jones's pocket ten minutes ago. Proposition (d) entails:

e. The man who will get the job has ten coins in his pocket.

Let us suppose that Smith sees the entailment from (d) to (e), and accepts (e) on the grounds of (d), for which he has strong evidence. In this case, Smith is clearly justified in believing that (e) is true.

But imagine, further, that unknown to Smith, he himself, not Jones, will get the job. And, also, unknown to Smith, he himself has ten coins in his pocket. Proposition (e) is then true, though proposition (d), from which Smith inferred (e), is false. In our example, then, all of the following are true: (i) (e) is true, (ii) Smith believes that (e) is true, and (iii) Smith is justified in believing that (e) is true. But it is equally clear that Smith does not know that (e) is true; for (e) is true in virtue of the number of coins in Smith's pocket, while Smith does not know how many coins are in Smith's pocket, and bases his belief in (e) on a count of the coins in Jones's pocket, whom he falsely believes to be the man who will get the job.
The example, no doubt, is trite as well as farfetched (more life-like examples could presumably be thought of). Anyway, the Gettier examples have been widely discussed. Is Gettier right in claiming that these are not cases of knowledge, or that in these cases the person is really justified in his beliefs? This is where x-phi comes in. In fact, experimental philosophers may approach such an issue in two different ways. (See Antti Kauppinen, "The Rise and Fall of Experimental Philosophy.") One may either try to settle the issue by establishing that all or a majority of English speakers would not (or would) apply the word "know" in this case, or one may try to show that there is no general agreement on how to describe the case.

The article ”Gender and Philosophical Intuition” by Wesley Buckwalter and Stephen Stich is an instance of the latter approach. Among other things, they discuss the following experiment: test subjects were told the below story, and then were asked, ”Does Peter really know that there is a watch on the table, or does he only believe it?”:
Peter is in his locked apartment, and is reading. He decides to have a shower. He puts his book down on the coffee table. Then he takes off his watch, and also puts it on the coffee table. Then he goes into the bathroom. As Peter’s shower begins, a burglar silently breaks into Peter’s apartment. The burglar takes Peter’s watch, puts a cheap plastic watch in its place, and then leaves. Peter has only been in the shower for two minutes, and he did not hear anything. They discuss various questionnaire studies, searching for gender bias; thus, in a Gettier-type case, they found that women were more willing than men to attribute knowledge.
In this particular case, it was found that women were more willing than men to attribute knowledge to Peter. However, I shall not specifically be concerned with the writers' conclusions. What I do want to question is the relevance of this type of study (or the consensus-seeking kind) to issues in philosophy.

Why do I find this type of thought experiment problematic? For one thing, again, the story is farfetched. It is difficult to imagine ever being in a similar situation. The stage-setting seems designed to provide a richness of context, but what is missing is precisely that aspect of the context that would be relevant: the background that would show why it would matter whether Peter is said to know or not to know. What would one be doing in attributing knowledge to him? Is he somehow responsible for noticing that there had been a burglary? Should he have been on his guard?

In being asked to give a verdict, the test subjects are supposed to accept the request for an unambiguous verdict without question. Thus they are encouraged to think of questions of knowledge as concerned with the correct description of a highly complex situation, and at the same time to disregard the fact that in attributing knowledge to a person we are doing something, the significance of which depends on the actual context in which we are speaking, on the relation between the speaker, the listener, the person being spoken about and the situation.

The test just described, I would argue, constitutes a fake experiment. The writers have failed to ask a genuine question. They are guilty of what Frank Ebersole has dubbed "the classroom and blackboard fallacy", the "assumption that the special conditions for asking a question can miraculously be produced by writing an interrogative sentence on the blackboard". (Quoted by Don S. Levi, in "Ebersole’s Philosophical Treasure Hunt ", Philosophy 79 (2004), p. 305). Also, see Levi's article, "The Gettier Problem and the Parable of the Ten Coins", Philosophy 70 (1995), 5-25.)

Thus, rather than countering the problem of aprioism, an "experiment" like this one embodies the direct-insight prejudice that they were ostensibly designed to counteract, simply extending it to a larger population. More important, however, it is subservient to the idea of philosophy as concerned with the establishing of truths – which, it is assumed, the questionnaires may either confirm or disconfirm.

On the other hand, such a study fails to acknowledge the sense in which philosophical reflection is self-oriented. If we are puzzled, say, by the question how knowledge is possible or what it is to act intentionally, the puzzle arises from our attempt to understand our own use of these words (on the model of Augustine’s dictum: “What is time? If you don’t ask me, I know, if you ask me, I don’t know”). Hence whatever facts may be discovered through a questionnaire are external to our problem.

Thinking of philosophical discussion as aiming at the establishing of truths gives rise to a false dichotomy, in which it is either thought that speakers have direct access to their own use of words, or that they have to defer to the results of empirical inquiry. If we think about the aim of discussion as not being truth but clarification, on the other hand, there is no need nor room for an authoritative judgment: neither my own, nor that of the majority or of a consensus.

Rather than the traditional question: "what is knowledge?", I would contend, the route to clarity goes through asking the question "what do we do when we attribute knowledge to someone?" - the answer to this question presumably varying depending on the context of speaking. The experimental philosophers' questionnaires, however, steer our attention towards the traditional question.

When we carry on a philosophical discussion with others, everything is in fact open to view, we are as it were involved in a joint, open-ended thought experiment in which we may or may not succeed in reaching clarity concerning the problems that beset us (and, where clarity is obtained, it may turn out to be provisonal). Rather than trying to establish what does and does not make sense, we are engaged in a continuing reflection on possibilities of making sense. (This seems to be the gist of Wittgenstein’s exhortation, “Don’t think, but look!”)

In sum, then, an investigation like that described above seems to retain the problematic aspect of the idea of philosophy as an a priori form of inquiry (direct insight), while failing to acknowledge the sense in which philosophy is not empirical but reflective.


  1. I'm completely sympathetic with all of this. But let's suppose, for the sake of argument, that an x-phi proponent says that what they mean to do is to follow the dictum, "Don't think, but look!" Now, I think your point that doing an "experiment" of the above sort does pass the problem to a larger population. But then I suppose we can ask: what is or is not to be gained by that? A "gain" might be that we have more "data-points" on which to base our reflections about the issue, and seeing patterns--either agreement or disagreement--on responses to these cases might be relevant to what we (philosophers) can say about how we "normally" use words like "know," etc. But then x-phi looks to be not too much different than something like applied empirical moral psychology, and what we're doing is informing our own philosophical reflections by making ourselves aware of how people other than philosophers use various words/apply various concepts. So, surveying only takes you so far. As you point out, it can't be a substitute for reflection; it supplies something further upon which to reflect.

    This, as you say, is not new. And we can go further than Naess. Look at Aristotle's Nicomachean Ethics, in which he often appeals to what people commonly say about, e.g., what happiness consists in. I think this is all good and healthy, but then I'm not sure if it's what self-branded experimental philosophers think they are doing. At any rate, if we're going to give "intuitions" any weight, then the commonness (or not) of an intuition does not itself prove anything. Something else that seems worthy of attention is the work some x-phi folks have done to raise worries about the stability of philosophers' "intuitions," by showing how responses to thought experiments (think of them what you will) are prone to framing effects and other factors that make those intuitions seem unstable, even in the case of philosophers. Again, you might say this is applied psychology (and giving it a different name is not to say it is insignificant). I think this research is important as a way of demystifying "intuitions" and revealing the potential fallacies of doing philosophy via "intuition pumping." And so in that way--whether you call it x-phi or applied psychology--would be an important way of looking rather than thinking, as it were. However, I also don't think that all "armchair philosophy" involves Parfit-style intuition pumping. And I suspect you're right that the "fake cases" used in these studies may be of limited value (but this goes for psychologists, too, who study moral thinking by giving undergraduates twenty different versions of the trolley problem in order to understand the nature of moral judgment!).

    I don't particularly like disciplinary boundary policing, and so I'm hesitant to say that x-phi is psychology rather than philosophy. I'm not so sure someone like Hume, for example, could make much sense of that distinction.

  2. Don Levi asked me to enter this comment:

    An obvious issue with experimental philosophy is how results obtained in the laboratory or philosopher’s study have any validity outside of such an artificial setting.
    You say that what is missing from the “far-fetched” shower example is something to show what turns on whether or not the man in the shower, Peter, knew. And your diagnosis of where the experiment goes wrong puts the emphasis on how the attribution of knowledge is something that the attributor is doing. By doing so, your critique seems vulnerable to the question of whether attributions are doings.
    The Peter example has to do with the supposed logical truism that if Fa then for some x, Fx. An experiment involving subjects who are asked whether this implication is valid might get unexpected results, but why should we trust any of them? However, I don’t think that the antidote for ‘logic’ is for ‘self-orientation,’ for an understanding of our own (uses of) words. The example is set up by telling us that Peter left his watch on the coffee table when he went to take a shower. It does not explain why he did so, or what the significance of his putting it there is. Then it tells us that the watch was switched by a burglar (in 2 minutes!) with a cheap plastic watch. We are not told why the burglar did this. As a practical joke? Or, what? Instead, we are supposed to agree that there is a watch on the coffee table (because there is a cheap plastic watch there). That there is a watch on the table is different from there being a cheap plastic watch there. That there is a watch there is to be understood as the implication both of the fact that Peter’s put his watch there and that the burglar put a cheap watch there. Can it be understood that way?
    The lesson here is that this is not the kind of question that subjects in cog sci experiments can be expected to know how to think about. (The fact is that few philosophers do either).
    The issue is not really confined to talk of what Peter knows, or so I am suggesting. And so the fact that we are not told what someone is doing in claiming that he knows is not critical. Moreover, that the idea that it is important for us to be told it raises the issue of whether claims to know are doings!

    1. I take this to be your point: if Peter tells someone "There's a watch on the table" because he has left his watch there, then he won't feel what he said had been confirmed if someone tells him, "Yes, there's a cheap plastic watch on the table." Not everything that could, given some suitable context, be expressed by the words "There's a watch on the table" would be taken by him (or by others) to show that he had been right in what he originally said.
      I believe your point is well taken: we don't really get a Gettier case, for what Peter believed wasn't true (in spite of what might be called the verbal coincidence). The central issue concerns WHAT he believes, not whether it is to be called knowledge. It's quite a subtle point, and I believe cuts deeper to the core than the one I was making.
      Your other point concerns whether to attribute knowledge to someone is to DO something. I suppose "A attributes knowledge of ... to B" is ambiguous. It may mean that A judges that B knows ... or that B says that A knows ... You are right that the former should probably not be thought of as something A DOES: she may reckon with the fact, or it may be obvious to her, or it may suddenly strike her, say, that B knows they're planning a surprise party for him. But this is not a thing A does: she doesn't try to realize that B knows, or go in for seeing that he does, or do her best to be aware that he does.
      On the other hand, for A to SAY that B knows ... IS for her to do something. What it will mean for her to say that will depend on the circumstances in which she is speaking, on how the question arises. "Do you suppose B knows about the surprise party (being planned for him)?" - "Well, he saw Helen buying those balloons, so he may suspect it, even though they weren't for his party but for Patty's." We might decide to try to diffuse his suspicions, maybe drop it into a conversation that Helen went out to buy balloons for Patty's party (without making too much of a fuss about it). Things that we would find it pointless to do if on the other hand we decided B really does know.

  3. I’m not sure how relevant this is, but I’m reminded of L.W.’s remark “if you want to know what is proved, look at the proof.” (PG p.369) Thus, we should also ask, what does the experiment (e.g.) with the watch show? Well, say, that 30% of women said he knew as opposed to 20% of men. And that might indeed be an interesting fact, but does it tell us anything of philosophical import? Not really. Interpret it whatever way you want – so long as this interpretation is taken to have any bearing on the philosophical question ‘did he know’? it will, I should like to say, belong to the domain of prose.

    In relation to this, at the last ALWS symposium, there were a lot of epistemologists there talking about ‘degrees of belief’. I had never heard of such a thing and asked what that phrase meant. The reply I got was that if you ask someone how much they would wager on X out of 100$ or whatever, what they tell you is their degree of belief (or something along these lines). But how incredibly misleading to describe it in these terms! For, what you have actually discovered by asking such a question is simply that someone is willing to bet x amount of money in such and such a context. But this is disguised by the bit of prose, ‘degree of belief’.