Association Between Scientific Cognition and Scientific Literacy: Implications for Learning Science (Hale, Sloss, & Lawson, 2017)
In the current research scientific literacy is synonymous with general scientific knowledge. This form of literacy is sometimes referred to as a form of derived scientific literacy. Scientific cognition is not the same thing as scientific literacy; scientific cognition involves multiple components and sub-components. At the very least scientific cognition involves philosophy of science, scientific methodology, quantitative reasoning and logic. The primary interests in the study were whether or not scientific cognition and scientific literacy scores were associated, and whether or not there would be gender differences for total scores for each scale. The scientific literacy and scientific cognition assessment consisted of mostly questions derived from measuring devices used in the past. The assessments were administered as part of an online survey. The participants were 202 university students. The study was approved by the university's Institutional Review Board. The results indicate a positive association between scientific literacy and scientific cognition, and no gender differences for total scores from the scales. Additional analyses indicate there was gender differences for some of the items. There was gender differences for one item from the scientific literacy assessment and for two items from the scientific cognition assessment. The research report includes a discussion regarding future directions for relevant research, implications of learning science and limitations of the study.
Full Research Report Available Upon Request (PDF File)
The results show a positive association between scientific cognition and scientific literacy. The association was moderate in strength. The differences between men and women for total scores on scientific cognition and scientific literacy were not significant. The results indicate an association between gender (men vs. women) and responses (correct vs. incorrect) for three items from the online survey; one of the items from the scientific literacy assessment and two of the items from the scientific cognition assessment.
Scientific cognition and scientific literacy are measured on a continuum. Results from a study conducted by Drummond and Fischhoff (2015) show a positive association between the Scientific Reasoning Scale (SRS) and two widely used measures of scientific literacy, the Trend Factual Knowledge of Science Scale (TFKSS) and the Understanding of Scientific Inquiry Scale (USIS). The strength of the association was moderate, similar to our findings regarding the association between scientific cognition and scientific literacy. The SRS assesses skills needed to evaluate scientific evidence; the scale consists of items related to research methodology. The TFKSS assesses knowledge of scientific concepts. The USIS assesses knowledge or research methodology and probability. The scientific literacy assessment used in this study is similar to the TFKSS, as it is an assessment of knowledge of general scientific concepts. These scientific literacy scales have been used often in the field of public understanding of science (Alum, Sturgis, Tabourazi, & Brunton-Smith, 2008). The scientific cognition assessment is similar to the SRS and the USIS, as it involves questions regarding research methodology. Similar to the USIS it also involves questions regarding probability (quantitative reasoning). In addition, the scientific cognition assessment involves items that require knowledge in the philosophy of science.
In contrast to the finding that total scores, for men, on a general scientific knowledge test were better than for women (Sloss & Hale, Paper Forthcoming) we found no significant differences. Also, there were no differences between men and women for total scores on the scientific cognition assessment. There were significantly different scores between men and women for one item from the scientific literacy assessment and two items from the scientific cognition assessment. Men scored better on one item from the scientific literacy assessment and one item from the scientific cognition assessment. The item, question no.9, for which men scored better from the scientific literacy assessment involved a chemistry question. Question no. 9 was "[w]hich of the following are smaller than atoms a) proteins b) electrons c) amino acids." The correct answer is b. Past research indicates a difference in scores for some chemistry related items. A study comparing the performance of boys and girls in the Australian National Chemistry Quiz found no differences on some of the questions, but on some of the questions boys performed better than girls (Walding, Fogliani, Over, & Bain, 1994). There were other chemistry items on the scientific literacy assessment, but there were no gender differences for chemistry items other than question 9. Men scored better on an item (question no.9), involving quantitative reasoning, from the scientific cognition assessment. Question no. 9 was "[i]n the universal lottery, the chances of winning a prize are 1%. How many people do you think would win a prize if 1000 people buy a single ticket?" The correct is answer is 10. A gender difference on a task involving quantitative reasoning is in agreement with the scientific literature that demonstrates better performance of males regarding quantitative reasoning (Friedman, 1989; Leahey, & Guo, 2001). The only gender difference on quantitative reasoning occurred for question no. 9; there were no differences for other items involving quantitative reasoning. Women scored better on an item (question no.3) involving the philosophy of science. Question no. 3 was "[t]he falsification criteria in the context of science suggests a) If a scientific claim is proven then it is not false b) False claims are not accepted c) In order for a claim to be scientific it must be testable." The correct answer is c. The concept of falsification is one of the most discussed concepts in the philosophy of science. The concept is taught in low level research methods courses and philosophy of science courses. We weren't able to locate studies that investigated differences between genders regarding philosophy of science. It is unclear why the gender difference occurred on this task. There were other philosophy of science questions on the scale, but there were no gender differences on those tasks. Gender differences are often found when comparing scoring for individual items. A study investigating gender differences, for Hong Kong students, didn't find significant differences for total score in scientific literacy, but differences were found for components of scientific literacy (Yan Yip, D., Ming Chiu, M., & Chu Ho, E., 2004). Scientific literacy as conceptualized in that study was different than the conceptualization used in the current study. Scientific literacy ,in the study of Hong Kong students, consisted of five components: "understanding concepts, recognizing questions, identifying evidence, drawing conclusions, communicating conclusions." Females scored significantly higher in "recognizing questions" and "identifying evidence" while boys scored higher in "understanding concepts." These components demonstrate various elements involved with scientific thinking. To reiterate, our conceptualization of scientific literacy, is that scientific literacy demonstrates general scientific knowledge. Scientific literacy has a much broader definition in the Hong-Kong study than the definition we used.
Another important finding in the current study was that students confused science with pseudo- science. The overwhelming majority of students (79%) in the current study report that astrology is scientific, or is at least partly scientific. Only twenty one percent of participants in the study answered the following question correctly: "Which of the following statements are true? A) Astrology is not at all scientific B) Astrology is partly scientific C) Astrology is a legitimate field of scientific study." The correct answer is A. The astrology question is an item from the scientific literacy assessment. The results from a study conducted by Sugarman and colleagues (2011) found that majority of students (78%) considered astrology at least sort or scientific. Only 52% of science majors indicated that astrology was “not at all” scientific. Those finding are similar to what we found. Astrology has no scientific validity, although at one time it was considered a science by some. Newspapers and magazines dedicate sections to horoscopes, and belief in astrology is prevalent in western society. This exposure to astrology as a legit domain probably has a strong influence regarding belief in the scientific validity of astrology. Cognitive priming is often powerful, and may modulate beliefs, even when priming is used to promote pseudo-science. Some people may confuse astrology with astronomy; astrology has origins associated with positional astronomy. This confusion may lead to an incorrect response regarding the scientific validity of astrology. A high level of scientific literacy and scientific cognition may serve as safeguards against these sort of pseudo-scientific beliefs.
The question most often answered incorrectly, from the scientific cognition assessment, was a question involving a covariation task. The question was presented as "A new medical treatment was designed to treat a serious health problem. Using the information provided below decide whether the treatment was effective: 200 people were given the treatment and improved 75 people were given the treatment and did not improve 50 people were not given the treatment and improved 15 people were not given the treatment and did not improve A) Treatment was effective B) Treatment was not effective." The probability that the treatment is effective is (200/275) .727. The probability that the treatment is not effective is (50/65) .769. The answer is B. Approximately 53% of the students answered the question incorrectly. The incorrect response given to this question stems from at least two key cognitive errors: too much focus on the large number of people for which improvement occurred following treatment and a focus on the fact that more people who received treatment showed improvement than showed no improvement (Stanovich, 2009).
The survey used in this study consisted of items derived from other assessment tools, as well as questions designed by the researchers, similar to those from past studies. Some of the questions on the survey were designed by researchers involved with this study. Thus, it follows that the constructs of scientific literacy and scientific cognition were validly measured. A more comprehensive measure may require assessments consisting of more items. It is also possible that measures of these concepts may yield different results inside and outside the laboratory. Different conceptualizations of scientific literacy and scientific cognition require different measuring devices.
The study involved non-probability sampling. The participants in the study were college students, who have a great variability in scientific knowledge and variability in the number of science courses completed. Some of the students had taken higher level courses in research methods and stats, and it is reasonable to suggest that some of the students had probably taken philosophy of science courses and other courses that may have had an impact on performance. The external validity of this study is limited. Non-student samples and samples of other students may provide different results.
Future research could include using the scientific literacy assessment and scientific cognition assessment in a variety of contexts. The assessments could be revised and expanded in an effort to increase sensitivity and make them more comprehensive. Further investigation of gender differences as related to specific items from the assessment may be beneficial. A key focus for extensive investigation is the development of a model that allows, at least a basic framework, that can be used in teaching students and the general public. This type of investigation requires a multidisciplinary approach and a line of studies involving different science related areas. The cognitive processes underpinning scientific cognition are important and can be extended to various situations. To reiterate, scientific cognition is about much more that remembering scientific theories, laws and principles. Scientific cognition is essentially analytical thinking that can be used, and should be used in a wide range of conditions. At the very least in an effort to develop better scientific cognition students should be educated in the areas of the philosophy of science, research methodology, quantitative reasoning (probabilistic reasoning) and logic. These components are involved with scientific thinking. Science educators and the media do a disservice when they promote science and its wide range of relevant concepts as "just" being able to remember scientifically derived information, or promoting science as if it is all about a just having a sense of "wonder." Being able to recollect scientific facts and having a sense of wonder is important regarding science, but those qualities alone do not ensure high levels of scientific thinking. Assessment tools may help predict scientific eminence and be used as screening tools when hiring or considering admissions to college programs. More research needs to be done regarding scientific literacy and scientific cognition. Both of these concepts involve related cognitive mechanisms, and being knowledgeable in these areas will have positive consequences. Society is heavily dependent on science and technology, and these complex endeavors require complex thinking. We would like to see future research indicating a high positive association between scientific cognition and scientific literacy. A moderate association is not satisfactory.