Developing the RQ Test

The following interview was conducted with Dr. Keith Stanovich.  Dr. Stanovich is the author of What Intelligence Tests Miss: the psychology of rational thought, the recipient of many prestigious awards, and recognized as one of the most important cognitive scientists ever.  Visit Dr. Stanovich’s site at http://www.keithstanovich.com/Site/Home.html

Congratulations on the three-year grant that you and a Richard West received from the John Templeton Foundation to develop a comprehensive test of rational thinking.  Do you think the test will be completed in three years?   Will we see it being put to use in three years?

Thanks very much. We were very happy and flattered to receive the grant. We are fairly certain that we will have completed a prototype of a comprehensive test that could be used in scientific work by the end of the three-year grant.  There of course will still be substantial work to do after that to make it useful in applied settings such as education, business, and industry. For example, subsequent to us producing the research prototype, there will be much more standardization work to be done to make it useful in applied settings.  However, I think that it is totally realistic to think that we will have a comprehensive instrument ready for scientific use in just two and a half years–we are about six months into the grant now.

Your work shows that individuals can rate high in intelligence, and at the same time, rate low in rationality.  Is it likely that an individual will rate low in intelligence but high in rationality?

Yes, that is a very good question. It is important to realize that those two outlier states will not occur with equal frequency.  By outlier states I mean people who are low in rationality and high in intelligence, and then also the converse state, people who are high in rationality and low in intelligence.  The former will be much more frequent than the latter.  For many types of rational thinking subcomponents intelligence is necessary but not sufficient.  Also, with respect to many different rational thinking components, there are at least mild to moderate correlations with intelligence.  Only on a few rational thinking components–myside bias for example–is it the case that the rational thinking component is totally disassociated from intelligence.  On those few tasks there will indeed be as many individuals high in rationality and low in intelligence as there are low in rationality and high in intelligence.  But that will be the minority of cases.

Another way to put it is to say that we already know from the past research that led up to the grant that there is a profile of associations between intelligence and rational thinking subcomponents that is quite varied.  A few rational thinking tasks such as belief bias in syllogistic reasoning are quite highly correlated with intelligence.  Most rational thinking skills are modestly correlated with intelligence.  The use of base rates in probabilistic reasoning would be an example.  And finally there are those like myside bias that are quite dissociated.  Those differing profiles will lead to somewhat different outlier groups.  Rational thinking is quite multifarious, much more so than intelligence, so any given statement about individual differences may vary quite a bit across the subcomponents.  The answer to your question here will probably vary quite a bit across the different subcomponents.

In your excellent book What Intelligence Tests Miss: the psychology of rational thought you point out the irrational thinking habits of George Bush.  Why did you choose George Bush as the example?  Have you received any negative comments concerning the discussion of Bush's irrational thinking tendencies?

I chose Bush because he was such a surprising example dysrationalia:  the failure to think rationally despite adequate intelligence. He was a surprising example because most people would not grant his intelligence. But as I point out in the book, this is because they are confused about what intelligence is.  And that is equally true of his supporters and his detractors.

Bush’s detractors described him as taking disastrously irrational actions, and they seemed to believe that the type of poor thinking that led to those disastrous actions would be picked up by the standard tests of intelligence.  Otherwise, they would not have been surprised when his scores were high rather than low.  Thus, the Bush detractors must have assumed that a mental quality (rational thinking tendencies) could be detected by the tests that in fact the tests do not detect at all. 

In contrast, Bush’s supporters like his actions but admit that he has “street smarts,” or common sense, rather than “school smarts.”  Assuming his “school smarts” to be low, and further assuming that IQ tests pick up only “school smarts,” his supporters were likewise surprised by the high pro-rated IQ scores that were indicated.  Thus, his supporters missed the fact that Bush would excel on something that was assessed by the tests.  The supporters assumed the tests measured only “school smarts” in the trivial pursuit sense (“who wrote Hamlet?”) that is easily mocked and dismissed as having nothing to do with “real life.” That the tests would actually measure a quality that cast Bush in a favorable light was something his supporters never anticipated.

In the talks that I give on these topics, when I use the Bush example I tried to head off questions and negative comments by pointing out that there is an absolute consensus that there was something wrong with his thinking style and that this fact is not in dispute–that even his supporters acknowledge this fact.  For example, in a generally positive portrait of the President, David Frum nonetheless notes, “he is impatient and quick to anger; sometimes glib, even dogmatic; often uncurious and as a result ill-informed”.  Conservative commentator George Will agrees, when he states that in making Supreme Court appointments, the President “has neither the inclination nor the ability to make sophisticated judgments about competing approaches to construing the Constitution” (p. 23, 2005).  In short, there is considerable agreement that President Bush’s thinking has several problematic aspects: lack of intellectual engagement, cognitive inflexibility, need for closure, belief perseverance, confirmation bias, overconfidence, and insensitivity to inconsistency.

The full interview can be read with Dr. Stanovich in my new book – In Evidence We Trust: The Need for Science, Rationality, and Statistics.