Jamie Hale

Jamie Hale

Monday, May 20, 2019

2019 edition- In Evidence We Trust

Scientific evidence depends on converging evidence. That is, the convergence of different strategies, making use of the preponderance of evidence, that converge as a tentative finding.  Avoid the notion of the gold standard, or referring to a single methodology as superior.  Different methods have strengths and weaknesses and each study, review or meta-analysis should be evaluated accordingly.  Reliability and validity across methodologies and contexts are important.     In the following pages science, rationality/ critical thinking (in cognitive science terms) and statistics (frequentist type of stats) are discussed.

The content in chapter one includes short-articles (old, new & revised), a science discussion roundtable (featuring individuals from various fields), a full research report, concise overview of a study involving a teaching strategy regarding research methodologies and a nonsense detection kit. Some of the short articles presented in chapter one have been published on various internet sites, and some of the same or similar information may be discussed across different articles. There are at least two key benefits that can occur when presenting similar information across different articles (in different contexts): strengthening of memory connections, and each article can be read as a stand-alone article. In the science discussion roundtable participants are asked two questions. One) Do you have any tips for people that are interested in enhancing their ability to read scientific research? Two) What is the biggest (or at least one of the biggest misconceptions) misconception about science? The Nonsense Detection Kit is presented in chapter one. The impetus for designing the Nonsense Detection Kit was similar kits devised by Sagan, Shermer, and Lilienfeld.

Chapter two features short articles on rationality. That is rationality, as defined by cognitive science. Some of the same or similar information is contained across different articles. There are at least a couple of advantages to presenting information in this manner (refer to previously mentioned advantages in chapter one). Some of the articles focus on the rationality intelligence dichotomy. Also included in this chapter are interviews with Keith Stanovich and the Stanovich Research Lab (Keith Stanovich, Richard West and Maggie Toplak). In the interviews with Stanovich, he discusses the development of an RQ Test. In the interview with the Stanovich lab, rationality and intelligence are discussed.

Chapter three features frequently asked questions about research methods and statistics. Many of the questions are questions I have received in the past from my students. Some of the questions address basic research and statistics problems, while other questions are more complex. To reiterate, the research methods and stats discussed in this book mention a small sample relative to the wide range of methods and stats used across areas of science. Hard sciences and soft sciences often use different models, methodologies, stats, inferential processes, manipulations and assessments. Even within the same area of science different strategies are used. Not all science involves null hypothesis statistical testing and Popperian principles of falsification; science and the methodologies used are much broader. Science is a vast enterprise.

This edition consists of a lot of the same material that was in the first edition. The 2nd edition offers a revision to some parts of the 1st edition, and it consists of additional information including a full research report (examining association between scientific cognition and scientific literacy), concise overview of a research report on teaching research methodology, development of the rationality quotient and a discussion on different topics involving statistics (valuable sources provided if further study is an interest).

Precision in Language
I attempted to be precise with language use.  Precision in language is essential for science communication.  I point to examples, often seen in textbooks and used by instructors, to the lack of precision in scientific language.  That is a problem.  Clear communication requires precise, consistent use of terminology.  Students often accuse me of being a semantic stickler.  I accept the label.  When talking and writing about science, practice precision and consistency.


Monday, January 28, 2019

The So-Called Sports Illustrated Jinx

The Sports Illustrated Jinx is the notion that being featured on the cover of SI leads to bad-luck that negatively affects future performance.

Examples of the Sports Illustrated Jinx (Wikipedia excerpts):
“May 26, 1958: Race car driver Pat O’Connor appears on the cover of the magazine. He dies four days later on the first lap of the Indianapolis 500.

August 7, 1978: Pete Rose appears on the cover the same week that his 44-game hitting streak ended.

May 8, 1989: Jon Peters, of Brenham High School in Texas, sets the national high school record for games won by a pitcher, with a 51-0 record. The next game after his cover appearance, he loses for the first (and only) time of his high school career.

In November 2007, Kerry Meier of the Kansas Jayhawks appeared on the cover with the caption “Dream Season (So Far)” at the time when the Jayhawks’s record was 11-0. In their next game they lost to their arch rivals, the Missouri Tigers, 36-28, ending the Jayhawks’s perfect season.

November 9, 2009: Iowa’s Derell Johnson Koulianos appears on the front cover with the words “Still Perfect.” The Hawkeyes lost to Northwestern two days before the issue date, ending the longest winning streak in school history.”

Can the SI Jinx be explained by a statistical concept? Statistical regression to the mean indicates poor performance is typically followed by better performance, while exceptionally good performance is typically followed by decreased performance. How does this apply to the SI Jinx?

"It does not take much statistical sophistication to see how regression effects may be responsible for the belief in the Sports Illustrated jinx. Athletes’ performances at different times are imperfectly correlated. Thus, due to regression alone, we can expect an extraordinary good performance to be followed, on the average, by a somewhat less extraordinary performance. Athletes appear on the cover of Sports Illustrated when they are newsworthy- i.e., when their performance is extraordinary. Thus, an athlete’s superior performance in the weeks preceding a cover story is very likely to be followed by somewhat poorer performance in the weeks after. Those who believe in the jinx, like those who believe in the hot hand, are mistaken, not in what they observe, but in how they interpret what they see. Many athletes do suffer deterioration in their performance after being pictured on the cover of Sports Illustrated, and the mistake lies in citing a jinx, rather than citing regression as the proper interpretation of this phenomenon." Gilovich, 1991, p.26 

The term regression to the mean was discovered and coined by Francis Galton in the late nineteenth century (Kahneman, 2011).  The term might be hard to understand, as causal reasoning often overrides this type of statistical thinking. It might be a hard concept to understand for those without training in stats. Kahneman reports this sort of thinking doesn't come naturally and often leads to confusing outcomes. "The statistician David Freedman used to say that if the topic of regression comes up in a criminal or civil trial, the side that musty explain regression to the jury will lose the case. Why is it so hard? The main reason for the difficulty is a recurrent theme of this book: our mind is strongly biased toward causal explanations and does not deal well with 'mere statistics" (Kahneman, 2011, p.182).

References available upon request.