Scientific evidence depends on converging evidence. That is, the convergence of different strategies, making use of the preponderance of evidence, that converge as a tentative finding. Avoid the notion of the gold standard, or referring to a single methodology as superior. Different methods have strengths and weaknesses and each study, review or meta-analysis should be evaluated accordingly. Reliability and validity across methodologies and contexts are important. In the following pages science, rationality/ critical thinking (in cognitive science terms) and statistics (frequentist type of stats) are discussed.
The content in chapter one includes short-articles (old, new & revised), a science discussion roundtable (featuring individuals from various fields), a full research report, concise overview of a study involving a teaching strategy regarding research methodologies and a nonsense detection kit. Some of the short articles presented in chapter one have been published on various internet sites, and some of the same or similar information may be discussed across different articles. There are at least two key benefits that can occur when presenting similar information across different articles (in different contexts): strengthening of memory connections, and each article can be read as a stand-alone article. In the science discussion roundtable participants are asked two questions. One) Do you have any tips for people that are interested in enhancing their ability to read scientific research? Two) What is the biggest (or at least one of the biggest misconceptions) misconception about science? The Nonsense Detection Kit is presented in chapter one. The impetus for designing the Nonsense Detection Kit was similar kits devised by Sagan, Shermer, and Lilienfeld.
Chapter two features short articles on rationality. That is rationality, as defined by cognitive science. Some of the same or similar information is contained across different articles. There are at least a couple of advantages to presenting information in this manner (refer to previously mentioned advantages in chapter one). Some of the articles focus on the rationality intelligence dichotomy. Also included in this chapter are interviews with Keith Stanovich and the Stanovich Research Lab (Keith Stanovich, Richard West and Maggie Toplak). In the interviews with Stanovich, he discusses the development of an RQ Test. In the interview with the Stanovich lab, rationality and intelligence are discussed.
Chapter three features frequently asked questions about research methods and statistics. Many of the questions are questions I have received in the past from my students. Some of the questions address basic research and statistics problems, while other questions are more complex. To reiterate, the research methods and stats discussed in this book mention a small sample relative to the wide range of methods and stats used across areas of science. Hard sciences and soft sciences often use different models, methodologies, stats, inferential processes, manipulations and assessments. Even within the same area of science different strategies are used. Not all science involves null hypothesis statistical testing and Popperian principles of falsification; science and the methodologies used are much broader. Science is a vast enterprise.
This edition consists of a lot of the same material that was in the first edition. The 2nd edition offers a revision to some parts of the 1st edition, and it consists of additional information including a full research report (examining association between scientific cognition and scientific literacy), concise overview of a research report on teaching research methodology, development of the rationality quotient and a discussion on different topics involving statistics (valuable sources provided if further study is an interest).
Precision in Language
I attempted to be precise with language use. Precision in language is essential for science communication. I point to examples, often seen in textbooks and used by instructors, to the lack of precision in scientific language. That is a problem. Clear communication requires precise, consistent use of terminology. Students often accuse me of being a semantic stickler. I accept the label. When talking and writing about science, practice precision and consistency.