The Illusive Scientific Method

Science writers, educators and sometimes researchers have a tendency to refer to the scientific method. This method is the systematic process, methodology, used in science to acquire knowledge and test claims. One huge problem with the referral to a singular methodology, it doesn't exist. Science makes use of multiple methods, models and inferential strategies. Why so much discussion about and illusory method? Maybe, it demonstrates and appeal to tradition, an appeal to popularity, a misunderstanding of the sometimes complex nature of science and its relevant components and interacting factors, or mentioning a singular scientific method is easier than the alternatives (demonstrating cognitive miserliness- which is normal for most).

Science is broad; it consists of many components and sub-components. Discussions regarding science are sometimes short-circuited by discussing a single component. These types of discussions oversimplify the wide range of science, its development, and implications. A full appreciation of science requires much more than a focus on a singular element. Skepticism is an element of a scientific attitude and is important, but a skeptical attitude alone—without other cognitive skills and knowledge—doesn’t make one a scientific thinker. Science is all about skepticism, so say the popularizers of science. Skepticism is important, but without the knowledge and appropriate skills, this characteristic will not make one a scientific thinker. Science is hard. In the words of Albert Einstein, “Things should be made as simple as possible, but not any simpler.”  Science: The Vast Enterprise

Scientific literacy, in the context of this article and my research, is synonymous with general scientific knowledge. Scientific literacy in this form involves remembering scientific facts, theories, principles, and so on—products of scientific inquiry. This form of literacy is sometimes referred to as a form of derived scientific literacy. Scientific literacy is important, however other science related concepts are just as important. Scientific cognition is not the same as scientific literacy.

Scientific cognition (thinking) involves complex cognitive mechanisms. Scientific cognition involves much more than general scientific knowledge, procedural skills to conduct research, attaching “science says” to your statements, a science degree, perpetuating views of popular science figures, identifying yourself as evidence based, asking for evidence, being skeptical, etc. Scientific thinking involves thinking that can also be used out of the lab. At the very least scientific cognition involves philosophy of science, scientific methodology, quantitative reasoning, probabilistic reasoning, and logic (deductive and inductive). To learn more about scientific cognition and scientific literacy refer to- ScientificCognition and Scientific Literacy at Kentucky Academy of Science 

A better understanding and promotion of science is needed. Science does not involve a singular method. Rethinking science education involves at the most basic level acknowledging the broad nature of science, promoting the message that science is hard but learnable, identifying and accepting the limitations of science, and understanding science is about more than just retrieving scientific facts from memory. Science education is valuable for everyone. Recommendations for future science education: Rethinking Science Education   

Precision in language is essential for science communication. The misuse of the term "scientific method" is one of many examples of unclear language. In my book- In Evidence We Trust- I point to examples, often seen in textbooks and used by instructors, to the lack of precision in scientific language.  That is a problem.  Clear communication requires precise, consistent use of terminology.  Students often accuse me of being a semantic stickler.  I accept the label.  When talking and writing about science, practice precision and consistency.  

2019 edition- In Evidence We Trust

Scientific evidence depends on converging evidence. That is, the convergence of different strategies, making use of the preponderance of evidence, that converge as a tentative finding.  Avoid the notion of the gold standard, or referring to a single methodology as superior.  Different methods have strengths and weaknesses and each study, review or meta-analysis should be evaluated accordingly.  Reliability and validity across methodologies and contexts are important.     In the following pages science, rationality/ critical thinking (in cognitive science terms) and statistics (frequentist type of stats) are discussed.

The content in chapter one includes short-articles (old, new & revised), a science discussion roundtable (featuring individuals from various fields), a full research report, concise overview of a study involving a teaching strategy regarding research methodologies and a nonsense detection kit. Some of the short articles presented in chapter one have been published on various internet sites, and some of the same or similar information may be discussed across different articles. There are at least two key benefits that can occur when presenting similar information across different articles (in different contexts): strengthening of memory connections, and each article can be read as a stand-alone article. In the science discussion roundtable participants are asked two questions. One) Do you have any tips for people that are interested in enhancing their ability to read scientific research? Two) What is the biggest (or at least one of the biggest misconceptions) misconception about science? The Nonsense Detection Kit is presented in chapter one. The impetus for designing the Nonsense Detection Kit was similar kits devised by Sagan, Shermer, and Lilienfeld.

Chapter two features short articles on rationality. That is rationality, as defined by cognitive science. Some of the same or similar information is contained across different articles. There are at least a couple of advantages to presenting information in this manner (refer to previously mentioned advantages in chapter one). Some of the articles focus on the rationality intelligence dichotomy. Also included in this chapter are interviews with Keith Stanovich and the Stanovich Research Lab (Keith Stanovich, Richard West and Maggie Toplak). In the interviews with Stanovich, he discusses the development of an RQ Test. In the interview with the Stanovich lab, rationality and intelligence are discussed.

Chapter three features frequently asked questions about research methods and statistics. Many of the questions are questions I have received in the past from my students. Some of the questions address basic research and statistics problems, while other questions are more complex. To reiterate, the research methods and stats discussed in this book mention a small sample relative to the wide range of methods and stats used across areas of science. Hard sciences and soft sciences often use different models, methodologies, stats, inferential processes, manipulations and assessments. Even within the same area of science different strategies are used. Not all science involves null hypothesis statistical testing and Popperian principles of falsification; science and the methodologies used are much broader. Science is a vast enterprise.

This edition consists of a lot of the same material that was in the first edition. The 2nd edition offers a revision to some parts of the 1st edition, and it consists of additional information including a full research report (examining association between scientific cognition and scientific literacy), concise overview of a research report on teaching research methodology, development of the rationality quotient and a discussion on different topics involving statistics (valuable sources provided if further study is an interest).

Precision in Language

I attempted to be precise with language use.  Precision in language is essential for science communication.  I point to examples, often seen in textbooks and used by instructors, to the lack of precision in scientific language.  That is a problem.  Clear communication requires precise, consistent use of terminology.  Students often accuse me of being a semantic stickler.  I accept the label.  When talking and writing about science, practice precision and consistency.

In Evidence We Trust: The need for science, rationality and statistics 2nd edition