Scientific Cognition: Implications for Learning Science

Scientific cognition (thinking) involves complex cognitive mechanisms.  Scientific Cognition involves much  more than: gen. scientific knowledge, procedural skills to conduct research, attaching "science says" to your statements, a science degree, perpetuating views of popularizers of science, identifying yourself as evidence based, asking for evidence, being skeptical, etc.  Scientific thinking involves an array of components and can be used in everyday out of the lab thinking as well as when evaluating research and examining science texts.

Deanna Kuhn asserts that the essence of scientific thinking is coordinating belief with evidence (2001).  At the very least scientific cognition involves philosophy of science, scientific methodology, quantitative reasoning, probabilistic reasoning and elements of logic. Scientific cognition requires specific cognitive abilities and cognitive style (thinking disposition).

In a recent study we investigated whether or not scientific cognition and scientific literacy (general scientific knowledge) scores were associated, and whether or not there were gender differences for total scores for each scale (Hale, Sloss, & Lawson, Paper Forthcoming).  The scientific literacy and scientific cognition assessment consisted of mostly questions  derived from measuring devices used in the past. The assessments were administered as part of an online survey. The participants were 202 university students. The study was approved by the university's Institutional Review Board. The results indicate a positive association between scientific literacy and scientific cognition, and no gender differences for total scores from the scales. Additional analyses indicate there was gender differences for some of the items. There was gender differences for one item from the scientific literacy assessment and for two items from the scientific cognition assessment. One of the important findings that was found in the study was that students confused science with pseudo- science. The overwhelming majority of students (79%) in the study report that astrology is scientific, or is at least partly scientific. Only twenty one percent of participants in the study answered the following question correctly: "Which of the following statements are true? A) Astrology is not at all scientific B) Astrology is partly scientific C) Astrology is a legitimate field of scientific study."  The correct answer is A. The astrology question is an item from the scientific literacy assessment.  Another important finding was, consistent with finding in past studies, students didn't do well on a covariation task.  Knowledge in research methodology should assist students in providing the correct answer for this item. The question most often answered incorrectly, from the scientific cognition assessment,  was a question involving identifying a relationship between treatment and effects, and making use of comparison groups. These skills are taught in research methods courses. The question was presented as "A new medical treatment was designed to treat a serious health problem. Using the information provided below decide whether the treatment was effective: 200 people were given the treatment and improved 75 people were given the treatment and did not improve 50 people were not given the treatment and improved 15 people were not given the treatment and did not improve A) Treatment was effective B) Treatment was not effective." The probability that the treatment is effective is (200/275) .727. The probability that the treatment is not effective is (50/65) .769.  The answer is B.  Approximately 53% of the students answered the question incorrectly.

 

The cognitive processes underpinning scientific cognition are important and can be extended to various conditions. To reiterate, scientific cognition is about much more that remembering scientific theories, laws and principles.  Scientific cognition is essentially analytical thinking that can be used, and should be used in a wide range of conditions. At the very least in an effort to develop better scientific cognition students should be educated in the areas of the philosophy of science, research methodology, quantitative reasoning (probabilistic reasoning) and logic. These components are involved with scientific thinking. Science educators and the media do a disservice when they promote science and its wide range of relevant concepts as "just" being able to remember scientifically derived information, or promoting science as if it is all about a just having a sense of "wonder."  Being able to recollect scientific facts, being skeptical and having a sense of wonder is important regarding science, but those qualities alone do not ensure high levels of scientific thinking. Myself and colleagues would like to see future research indicating a strong positive association between scientific cognition and scientific literacy.

References available upon request

Don't Forget About The Stats: Quantitative Research

In the context of everyday language statistics (numbers, quantitative representations) are used to represent basketball player's free throw average, death rates, life spans and so on. In science statistics are tools used in describing, organizing, summarizing and analyzing data. Learning about stats will help you think in terms of probabilities, and allow you to gain a better understanding of research data. Statistics are not easy, but with some effort the basics can be learned by most people. Research methods and statistics are often taught together in college courses. Quantitative research uses stats, and these stats are essential in an effort to represent the data; stats are required for making sense of the research.          

Descriptive statistics are numerical measures that describe a population by providing information on the central tendency of the distribution, the width of distribution (dispersion, or variability), the shape of distribution (Jackson, 2009). Inferential statistics are procedures that allow us to make an inference from a sample to the population. That is, we are able to make generalizations about a population based on the information derived from the sample.

A key reason we need statistics is to be able to effectively interpret research. Without statistics it would be very difficult to analyze the collected data and make decisions based on the data. Statistics give us an overview of the data and allow us to make sense of what is going on. Without statistics, in many cases, it would be extremely difficult to find meaning in the data. Statistics provides us with a tool to make an educated inference.

Most scientific and technical journals contain some form of statistics. Without an understanding of statistics, the statistical information contained in the journal will be meaningless. An understanding of basic statistics will provide you with the fundamental skills necessary to read and evaluate most results sections. The ability to extract meaning from journal articles, and the ability to evaluate research from a statistical perspective are basic skills that will increase your knowledge and understanding of the article of interest. To reiterate, quantitative research uses stats, and to assess statistical validity, at least a basic understanding of stats is essential.

When researchers question a study’s statistical validity they are questioning issues relevant to how well the conclusions coincide with the results, represented as statistics. Interrogating statistical validity may include some of the following questions: If the study found a difference what is the probability that the conclusion was a false alarm?  If the study’s finding found no difference what is the probability that a real relationship went unnoticed?  What is the effect size?  Is the difference between groups statistically significant? Are the finding practically significant? What type of inferential stats were used to assess predictions? Could different statistical procedures have been used?

Gaining knowledge in the area of statistics will help you become a better-informed consumer. Statistics are difficult for many people. Students often cringe when they hear the word - statistics. Learning about statistics requires the same strategies as learning about other topics (strategies to improve learning and memory). Once an individual learns theoretical aspects and calculations used for basic statistical procedures the learning of more complex statistics become much easier. Everyone benefits from learning the basics of statistics. Statistics is not an easy subject compared to many other subjects, but the subject is much easier when one doesn't have negative expectations and realizes that with the appropriate cognitive effort and understanding of some rather basic mathematical principles the subject is learnable.  Being knowledgeable in the area of statistics will be beneficial across domains of scholarly and everyday life. 

Recently I asked Dr. Jonathan Gore (from Eastern Kentucky University) the following question- Why is a basic understanding of stats important for the public? He gave the following answer:

"My answer to why stats is important is that pretty much everything operates based on probability. Even some of the "hard" sciences are starting to realize that phenomena that used to only require a basic equation are now having to factor in probability to account for all that they observe."

If the objective is to thoroughly analyze the study, don't skip over the "Results" section when reading the paper. A key guideline for the Results section is a presentation of numerical findings that should be stated clearly, concisely and accurately. The methodology provides detailed information regarding processes used in the collection of data, while statistical procedures provide information on detecting meaningful signals among the noise: making sense of the data collected.    

To learn more about stats and research methods  read- In Evidence We Trust: The need for science, rationality and statistics (Hale,2013).

 The book contains 76 questions and answers regarding scientific research methods and stats. It also contains practice problems involving statistical procedures. 

    

References are available upon request

Better Study Strategies

Studying should be cognitively challenging, persistent and structured. Strong memory is not built easily or overnight. This article consists of links to articles relevant to learning / memory and key points from my seminar- Strategies To Maximize Learning.

How To Study

The effort required to form strong memory is often intense for students. Students often spend hours trying to master new information. Of course, methods to enhance memory are important for everyone, not just students. For example, when a friend recommends a new shoe store we want to remember the name of of it, or when going to the grocery it is important to remember the items we need to pick up. What are some strategies that can be used to strengthen memory?

Trying To Remember

In one study researchers investigated the role of intentional-encoding instructions and task relevance at study on visual memory performance (Varakin & Hale, 2014). Task relevance was manipulated by having participants keep a running tally of either the objects they were attempting to remember or an irrelevant category of objects during study. Half of the participants within each level of task relevance were further instructed to remember one category of objects for a subsequent recognition memory test (intentional memory group) , and the other half of the participants were not informed of a memory test (incidental memory group). Intentional-encoding instructions improved recognition discrimination only when participants were not already keeping a tally of the to-be-remembered objects. This result suggests that intentional-encoding instructions may improve visual memory due

Building a Better Memory

Are learning and memory completely distinct?  No; both are experienced based.  “[M]emory is the consequence of learning from an experience- that is, the consequence of acquiring new information” , asserts James McGaugh (memory researcher, author of Memory and Emotion).  Learning is a process of memory formation.  There are 2 general categories of memory: explicit and implicit.

Key Points from- Strategies to Maximize Learning (Hale, 2014):

•      Memory is the product of learning

•      Memory formation = brain change

•      All cognition, emotion, feeling, perception  and learning emanate from the brain

•      Healthy brain is imperative to maximize learning / memory

•      Mind- body is a unit- not separate

•      All cognition, emotion, feeling, perception  and learning emanate from the brain Healthy brain is imperative to maximize learning / memory   Mind- body is a unit- not separate

•      Foundations of memory include: brain health, focused attention, elaborative encoding, spaced rehearsal and testing

•      Understanding is imperative for strong memory

•      Studying should be structured: progressive, organized, spaced over multiple sessions and involve accurate evaluation

 

 

 

 

Rationality Quotient: Comprehensive Assessment of Rational Thinking

Stanovich and colleagues recently developed a prototype of the first comprehensive assessment of rational thinking. The test is discussed, and presented in detail in the new book,  titled- The Rationality Quotient.

Up until publication of- The Rationality Quotient - components of rational thinking had been tested using various tasks, but a comprehensive test was not available. I first discussed the development of such a test with Stanovich, in 2013- interview here.

In the following interview (conducted in November, 2016) Stanovich provides detailed answers to important questions about the test. 

 

What are some of the initial reactions, regarding the RQ, from academics?

Uniformly positive so far, and I believe that is because we were careful in the book to be explicit about two things.  First, we were clear about what our goals were and the goals were circumscribed.  Secondly, we included an entire chapter contextualizing our test (the Comprehensive Assessment of Rational Thinking, CART) and discussing caveats regarding its use as a research instrument or otherwise. In fact, I think we have already entirely achieved our aims.  We have a prototype test that is a pretty comprehensive measure of the rational thinking construct and that is grounded in extant work in cognitive science.  Now, this is not to deny that there is still much work to be done in turning the CART into a standardized instrument that could be used for practical purposes.  But of course a finished test was not our goal in this book.  Our goal was to show a demonstration of concept, and we have done that.  We have definitively shown that a comprehensive test of rational thinking was possible given existing work in cognitive science.  This is something that I have claimed in previous books but had not empirically demonstrated with the comprehensiveness that we have here by introducing the CART.  As I said, there are more steps left in turning the CART into an “in the box” standardized measure, but that is a larger goal than we had for this book.

I think that, at least so far, most academics have understood our goals and the feedback has been good.  We wrote a summary article on the CART in a 2016 issue of the journal Educational Psychologist (51, 23-34) and the feedback from that community has been good.

Are there components of the RQ that can be expected to show a strong positive correlation with intelligence?

The CART has 20 subtests and four thinking dispositions scales (the latter are not part of the total score). Collectively they tap both instrumental rationality and epistemic rationality.  In cognitive science, instrumental rationality means behaving in the world so that you get exactly what you most want, given the resources (physical and mental) available to you.  Epistemic rationality concerns how well beliefs map onto the actual structure of the world.  The two types of rationality are related.  In order to take actions that fulfill our goals, we need to base those actions on beliefs that are properly calibrated to the world.

The CART assesses epistemic thinking errors such as: the tendency to show incoherent probability assessments; the tendency toward overconfidence in knowledge judgments; the tendency to ignore base rates; the tendency not to seek falsification of hypotheses; the tendency to try to explain chance events; the tendency to evaluate evidence with a myside bias; and the tendency to ignore the alternative hypothesis.

Additionally, CART assesses instrumental thinking errors such as:  the inability to display disjunctive reasoning in decision making; the tendency to show inconsistent preferences because of framing effects; the tendency to substitute affect for difficult evaluations; the tendency to over-weight short-term rewards at the expense of long-term well-being; the tendency to have choices affected by vivid stimuli; and the tendency for decisions to be affected by irrelevant context. 

 

Importantly, the test also taps what we call contaminated mindware.  This category of thinking problem arises because suboptimal thinking is potentially caused by two different types of mindware problems.  Missing mindware, or mindware gaps, reflect the most common type—where Type 2 processing does not have access to adequately compiled declarative knowledge from which to synthesize a normative response to use in the override of Type 1 processing.  However, in the book, we discuss how not all mindware is helpful or useful in fostering rationality.  Indeed, the presence of certain kinds of mindware is often precisely the problem.  We coined the category label contaminated mindware for the presence of declarative knowledge bases that foster irrational rather than rational thinking.  Four of the 20 subtests assess contaminated mindware.

My purpose in digressing here to describe the CART is to point out that given the number and complexity of rational thinking skills, it is likely that the subtests will have correlations with intelligence that are quite variable.  The four subtests with the highest correlations are: the Probabilistic Reasoning Subtest; the Scientific Reasoning Subtest; the Reflection Versus Intuition Subtest; and the Financial Literacy Subtest.  Correlations with these subtests tend to .50or higher.  Most of the subtests of the CART correlate with intelligence in the range of .25 to .50 (a few have even lower correlations). Some very important components of rational thinking do show considerable dissociation from intelligence.  Overconfidence (measured by the Knowledge Calibration Subtest of the CART) shows only a .38 correlation with intelligence.  This represents a substantial amount of dissociation for a key component of rational thinking.  Kahneman, for example, devoted substantial portions of his best-selling book to this component of rational thinking.  Myside bias (measured by our Argument Evaluation Subtest) likewise shows a correlation of .38, indicating a substantial dissociation.  This thinking bias is at the center of many discussions of what it means to be rational.  Some of the subtests that most directly measure the components of the axiomatic approach to utility maximization show relatively mild correlations with intelligence.  For example, the Framing Subtest shows a fairly low .28 correlation.  Framing measures a foundational aspect of rational thinking according to the axiomatic approach.

Finally, some subtests of immense practical importance show very low correlations with intelligence in the CART.  The skill of assessing numerical expected value shows a correlation of only .21, and the ability to delay for greater monetary reward shows a correlation of only .06.  The tendency to believe in conspiracies shows a modest correlation of .34.

Do you think rationality will acquire the same high level status as intelligence in the near future?

Not in the near future, no.  Our goal with the book was more modest—to simply raise awareness of the importance of rational thinking and the ability of modern cognitive psychology to measure it.  The result of our efforts will, we hope, redress the imbalance between our tendency to value intelligence versus rationality.  In our society, what gets measured gets valued.  Our aim in developing the CART was to draw attention to the skills of rational thought by measuring them systematically.  In the book, we are careful to point out that we operationalized the construct of rational thinking without making reference to any other construct in psychology, most notably intelligence.  Thus, we are not trying to make a better intelligence test.  Nor are we trying to make a test with incremental validity over and above IQ tests.  Instead, we are trying to show how one would go about measuring rational thinking as a psychological construct in its own right.  We wish to accentuate the importance of a domain of thinking that has been obscured because of the prominence of intelligence tests and their proxies.  It is long overdue that we had more systematic ways of measuring these components of cognition, that are important in their own right, but that are missing from IQ tests.  Rational thinking has a unique history grounded in philosophy and psychology, and several of its subcomponents are firmly identified with well-studied paradigms.  The story we tell in the book is of how we have turned this literature into the first comprehensive device for the assessment of rational thinking (the CART).

Why does society need a comprehensive assessment of rational thinking?

To be globally rational in our modern society you must have the behavioral tendencies and knowledge bases that are assessed on the CART to a sufficient degree.  Our society is sometimes benign, and maximal rationality is not always necessary, but sometimes, in important situations, our society is hostile.  In such hostile situations, to achieve adequate degrees of instrumental rationality in our present society the skills assessed by the CART are essential.  In Chapter 15 of The Rationality Quotient we include a table showing that rational thinking tendencies are linked to real life decision making.  In that table, for each of the paradigms and subtests of the CART, an association with a real-life outcome is indicated.  The associations are of two types.  Some studies represent investigations where a laboratory measure of a bias was used as a predictor of a real-world outcome.  Others are reports of real-world analogues of biases that were originally discovered in the lab.  Clearly more work remains to be done on tracing the exact nature of the connections—that is, whether they are causal.  The sheer number of real-world connections, however, serves to highlight the importance of the rational thinking skills in our framework.  Now that we have the CART, we could, in theory, begin to assess rationality as systematically as we do IQ.  If not for professional inertia and psychologists’ investment in the IQ concept, we could choose tomorrow to more formally assess rational thinking skills, focus more on teaching them, and redesign our environment so that irrational thinking is not so costly.  Whereas just thirty years ago we knew vastly more about intelligence than we knew about rational thinking, this imbalance has been redressed in the last few decades because of some remarkable work in behavioral decision theory, cognitive science, and related areas of psychology.  In the past two decades cognitive scientists have developed laboratory tasks and real-life performance indicators to measure rational thinking tendencies such as sensible goal prioritization, reflectivity, and the proper calibration of evidence.   People have been found to differ from each other on these indicators.  These indicators are structured differently from the items used on intelligence tests.  We have brought this work together by producing here the first comprehensive assessment measure for rational thinking, the CART.