Defending- In Evidence We Trust

In 2005 I had an idea to write a book about scientific and rational thinking. I started writing the book that year, but progress was slow. In 2006 and 2007 I wrote a few books on different subjects; however I continued to work on the 2005 idea.  In 2010 I completed another book; this one was on exercise and nutrition myths.  Over the next few years I was occupied with various projects, and my ideas about the book contents changed often.  January 1, 2014, after changing the contents many times, my idea- In Evidence We Trust- came to fruition. 

The majority of books sold have been to college students (undergraduate & graduate).  Surprisingly, the book hasn't done as good as I thought it would with science writers and others that perpetuate science to the public.  Before publication of the book I thought those promoting science to the general public would be my main audience.  In accordance with reviews and feedback from readers there are 2 major criticisms aimed at the book- it is redundant and too complex.

Firstly, I would like to respond to the redundancy criticism.  To reiterate, the book was written over a period of 9 years, so overlap can be expected with contents regarding similar topics. Some of the short articles presented in chapter one and two have been published on various internet sites, and some of the same or similar information may be discussed in different articles. In chapter three some of the questions overlap. Many of those questions were questions asked by students from courses on research methods and statistics.  There are at least two key benefits that for presenting similar information across different articles (in different contexts): strengthening of memory connections, and each article can be read as a stand-alone article.  Elaborating on the information and repeating consistently over time will assist in enhancing memory (2 of foundations of memory).  I suspect some of this information will be new to readers, thus redundancy will be beneficial to an even higher level.  Novel information is processed differently than routine information.  With the appropriate strategies novel information moves from being cognitively difficult to much easier (requires few cognitive resources and is retrieved with less effort) to process, once it becomes learned (discussed in detail in my seminars - Strategies To Maximize Learning and Exploring Memory). Stand-alone articles have the benefit of allowing the reader to read that section only; no need to read entire chapter if only interested in specific topic.

In addressing the second major criticism- complexity- it is important to point out that science, rationality and statistics are difficult.  A comprehensive understanding of science and its applications requires general knowledge in the meta-sciences (philosophy of science, history of science, sociology of science, psychology of science) research methodology and statistics.  In Evidence We Trust- is not a complete guide to the meta-sciences (consists of phil. of science and psy of science information- concise), but it is focused on research methodology and statistics.  It is important to recognize different areas of science use the same words differently, and they often use different types of methodologies. The research methods and statistics discussed in the book may not apply to some areas of science.  This topic is mentioned in the book.  Is the book too complex?  The content in this book may be difficult for some to comprehend. However, with some effort and patience the content is learnable for most people. In the words of Albert Einstein “Things should be made as simple as possible, but not any simpler.” Science, rationality and statistics can be simplified to a degree, but relative to most other topics these topics are difficult.  This book is not written for cognitive misers (the cognitively lazy).  This book is written for individuals that are interested in separating knowledge and nonsense, and are willing to put forth at least a moderate level of cognitive effort.  If you sale science (science writer, science educator, science based -often synonymous with evidence based provider) you need to  understand research methods and statistics. If you are not willing to learn at least the basics you should stop selling science. 

Mervine, in a review of In Evidence We Trust published in Skeptic Briefs, points out that "Hale states, 'after learning the information provided in this chapter [chapter 3], you should have the skills necessary to read scientific reports.'  I come from a non-science background, and I do not feel confident that I could tackle a scientific report after reading this chapter alone."  A few others have pointed out that they agree with Mervine on this point.  A more appropriate statement is, after learning the information provided in this chapter and thoroughly reviewing the recommended sources (at end of chapter 3), references and appendices, you should have the skills necessary to read scientific reports. 

Overall, the response to the book has been positive.  Almost all of the conversations I have had, regarding he book, with students have been good.  A few students have said the book helped with their school work and assisted them in distinguishing between science and pseudo-science.  In addition, some top intellectuals have had some good things to say about the book.

“A great introduction to scientific thinking, useful for the student and the general reader alike”  …Keith E. Stanovich, Emeritus Professor, University of Toronto, author of How To Think Straight About Psychology 

“A useful, informative, and engaging compendium of critical thinking tools.  Should come in handy for novices and experts alike.  I recommend it!”  …Scott O. Lilienfeld, Ph.D., Professor, Department of Psychology, Emory University, Atlanta, Georgia, co-author of 50 Great Myths of Popular Psychology.

"Jamie Hale has written an engaging introduction to the importance of scientific and critical thinking and how useful and needed it is in everyday life. On top of that, he throws in a nice primer on basic statistical topics and offers useful and insightful answers. This book would be useful to any student of science and rationality as well as any person interested in these topics."

...Gregory Feist, Ph,D., Associate Professor of Psychology, San Jose State University, San Jose, CA, author of The Psychology of Science and the Origins of the Scientific Mind.

In the context of In Evidence We Trust evidence is synonymous with scientific evidence. Testimonials, anecdotes, they-says, wishful thinking and so on do not count for evidence.  If we consider these types of claims and feelings as evidence then any discussion of evidence is vacuous.  Testimonials exist for almost any claim you can imagine.  That does not mean that claims of this sort have no value.  However, experiences are confounded (confused by alternative explanations).  Experiences may be important in some contexts, and they may serve as meaningful research questions.  However, a meaningful question or a possible future finding is not synonymous with evidence; although, in the future they could become evidence.  

Recommended Readings:

In Evidence We Trust- Blog Post (contents)

Recommended Resources: In Evidence We Trust

Review Article: In Evidence We Trust

Mervine, B. (2014).  Trusting Evidence.  Skeptic Briefs, 24(2), p.7. 

Coert Visser- on In Evidence We Trust

Purchase- In Evidence We Trust  

 

Project: thinking about science

The word science is derived from a latin word scientia meaning an organized body of theoretical knowledge.  Giving a precise definition of science is difficult as there is little consensus in the scientific community as to what that definition should be.   When discussing science what is often discussed are products of science or scientific methodology.   That is, making use of scientific findings (scientific products).  Even though this description provides part of the picture it is an incomplete description regarding the enormous enterprise of science. Science in various contexts has huge implications and a comprehensive study should be approached using a multi-disciplinary approach.   A comprehensive understanding of the implications of science requires more than the procedural skills to conduct research. Acquiring general or domain specific knowledge or citing studies is important, but doesn't indicate scientific cognition, or a comprehensive understanding of science.

    

Project: thinking about science involves pilot studies, assessments, a wide range of samples, full studies, scale design, univariate, bivariate and multivariate stats.  The project is a long term commitment with an emphasis on examining the association among various measures of science and finding predictors that may be useful regarding scientific achievement in an array of domains. Science is hard work.  The project will involve a number of people from various disciplines. 

 

Scientific literacy vs. Scientific cognition (thinking, reasoning)

Scientific literacy has been conceptualized  in various ways (see; Norris & Phillips, 2002).  Examples include understanding science and its applications, knowledge of what counts as science, knowledge of risks and benefits of science, etc.  In the current research scientific literacy is synonymous with general scientific knowledge that involves various domains.  This form of literacy is sometimes referred to as derived scientific literacy.  Scientific cognition is not the same thing as scientific literacy; it involves multiple components and sub-components (Feist, 2006).  Deanna Kuhn asserts that the essence of scientific thinking is coordinating belief with evidence (2001).  At the very least scientific cognition involves philosophy of science, scientific methodology, quantitative reasoning, probabilistic reasoning and elements of deductive and inductive logic.

Various scales have been developed to measure scientific thinking / reasoning / cognition.  Kahan developed a scale called the Ordinary Science Intelligence Scale (OSI_2.0, Kahan, 2014).  Drummond and Fischhoff (2015) developed the Scientific Reasoning Scale.  Drummond and Fischhoff found that measures of scientific reasoning were distinct from measures of scientific literacy.  Kevin Dunbar (2000) and Zimmerman (2005) have also conducted research on scientific thinking.  Dunbar's research mostly involves examining cognitive processes underpinning thinking during the research process, while Zimmerman's research is broader, examining various scales, and development patterns of scientific thought.  Fugelsang et al. (2004) have examined strategies that scientists and non-scientists use to evaluate data that is consistent or non-consistent with expectations. 

Stage 1: Current research

Sam Sloss and I are working on a paper that involves a large number of outcomes, including measures of general scientific knowledge, attitudes towards science and confidence in science.  The paper also involves an array of stats; univariate and multi-variate statistical procedures are used.

The development of a new scientific literacy scale and scientific cognition scale are also in the process.  Soon, the scales will be administered, in a pilot study, to University students.  Cronbach's alpha will be utilized once data is collected.  The scale might need to be modified  before using in studies. 

    

We are in the early stages of the project and hope to advance at moderate pace.  I plan on making progress reports consistently.  In the end my hope is that a large amount of data will be collected and can be utilized to further advance science and provide everyone interested in science useful information.  

References available upon request

Trying to Remember

In one study researchers investigated the role of intentional-encoding instructions and task relevance at study on visual memory performance (Varakin & Hale, 2014). Task relevance was manipulated by having participants keep a running tally of either the objects they were attempting to remember or an irrelevant category of objects during study. Half of the participants within each level of task relevance were further instructed to remember one category of objects for a subsequent recognition memory test (intentional memory group) , and the other half of the participants were not informed of a memory test (incidental memory group). Intentional-encoding instructions improved recognition discrimination only when participants were not already keeping a tally of the to-be-remembered objects. This result suggests that intentional-encoding instructions may improve visual memory due to generic attentional modulation, not encoding-specific processes.

Full study:  Varakin,D.A., & Hale, J. (2014).  Intentionalmemory instructions direct attention, but do not enhance visual memory. SageOpen, 1-8.  doi:10.1177/2158244014553588  

In another study, we conducted at Eastern Kentucky University, we examined whether intentional encoding instructions improve long-term recognition memory for visual appearance (Varakin, Frye, & Mayfield, 2012). The effect of memory instructions was examined using a factorial design, so that attention to/task relevance of objects could be manipulated independently of memory instructions. The sample size was large enough to achieve power equal to .80 for medium effect sizes (f = .25). There was no effect of intentional memory instructions. These results suggest that observers cannot easily enhance encoding and storage of visual information in long-term memory.  Intent to remember, per-se, may not enhance memory.   

Full study:  Varakin,D.A., Frye, KM., & Mayfield, B. (2012). Intentional Memory Instructions do not Improve Visual Memory.  International Journal of Brain andCognitive Sciences, 1(3), 18-25.    

Trying to remember or reading material over and over does not necessarily lead to better memory. The appropriate behaviors are required, even when one is trying to remember. The foundations of memory (declarative memory) include: brain health, focused attention, elaborative encoding, spaced rehearsal (distributed practice) and testing. A key underlying factor supporting memory is understanding.

•      The foundations of memory support understanding

•      Understanding implies strong organization of memory connections

•      ALL MEMORY RESIDES IN THE BRAIN!!

In my seminars Exploring Memory and Strategies To Maximize Learning a comprehensive overview of memory is provided.

Confused About Critical Thinking?

Educators often pay lip service to the idea of teaching “critical thinking”.  But, when asked to define “critical thinking” answers are often weak and sometimes so ambiguous they are virtually worthless.  Common responses to the defining  critical thinking questions include, “teaching them how to think”, “teaching them formal logic”, or “teaching them how to solve problems.”   They already know how to think, logic is only a portion of what is needed to increase critical thinking, and teaching them how to solve problems is an ambiguous answer that is context specific.  Stanovich argues, “that the super-ordinate goal we are actually trying to foster is that of rationality” (Stanovich, 2010, p.198). Ultimately, educators are concerned with rational thought in both the epistemic sense and the practical sense.  Certain thinking dispositions are valued because they help us base our beliefs on available evidence and assist us in achieving our goals. Many educators express to students and administrators the importance of critical thinking, yet, many of those expressing the importance of critical thinking don't know what critical thinking encompasses.  In fact, many educators are simply in the business of repeating what others say-Critical thinking is important.  Critical thinking, as promoted by educators, is often and appendage used to increase the value of intellectual status, or used to indicate this course is different.   

Understanding Rationality

Rationality is concerned with two key things: what is true and what to do (Manktelow, 2004).  In order for our beliefs to be rational they must be in agreement with evidence.  In order for our actions to be rational they must be conducive to obtaining our goals. 

Cognitive scientists generally identify two types of rationality: instrumental and epistemic (Stanovich, 2009). Instrumental rationality can be defined as adopting appropriate goals, and behaving in a manner that optimizes one's ability to achieve goals. Epistemic rationality can be defined as holding beliefs that are commensurate with available evidence. This type of rationality is concerned with how well our beliefs map onto the structure of the world. Epistemic rationality is sometimes called evidential rationality or theoretical rationality. Instrumental and epistemic rationality are related.  In order to optimize rationality one needs adequate knowledge in the domains of logic, scientific thinking, and probabilistic thinking.   A wide variety of cognitive skills fall within these broad domains of knowledge.  Components of critical thinking have been operationalized in a wide range of studies.  

Critical thinking can and has been measured.  CT is something much more than the over conceptualized ambiguous definitions often provided by educators and others (evidence based practitioners, science writers, skeptics, and so on...)  perpetuating the importance of critical thinking. 

Tasks on a critical thinking test include (Hale, 2012):

Answer the following:

John is looking at Cindy but Cindy is looking at James. John is married but James is not.

Is a married person looking at an unmarried person?

A) Yes   B) No   C) Cannot be determined

Does a conclusion follow logically from the two premises?

Premise 1: All living things need food

Premise 2: Animals need food

Conclusion: Animals are living things

A) Yes B) No

Read and answer the following:

A suit and tie cost $120 in total. The suit costs $100 more than the tie.

How much does the tie cost?     

In order for educators to successfully teach critical thinking / rational thinking it is imperative that they understand what critical thinking actually is and why it matters.  What are the goals of critical thinking?  How can critical thinking be assessed?  Does my curriculum contain information regarding scientific reasoning, logic, heuristic processing and probabilistic thinking? 

Critical thinking is about what is true (epistemic rationality) and what to do (instrumental rationality).  The best tip I can provide regarding critical thinking is to educate yourself on the works of the most influential people in the field of critical thinking.  A few of those people  include: Keith Stanovich, Daniel Kahneman, Richard West, Shane Frederick and Jonathan Baron. 

Rationality vs. Intelligence

 Rational thinking skills are as important as intelligence.  Intelligence and rationality are often dissociated. Research demonstrates that intelligence is often a weak predictor of rationality.  This has been shown over a wide range of studies.  Intelligence is important, but there is more to good thinking than intelligence.  Intelligence reflects reasoning abilities across a wide variety of domains particularly novel ones.  In addition, intelligence reflects general declarative knowledge acquired through acculturated learning.  Rationality reflects appropriate goal setting, goal optimization, and holding evidence-based beliefs.

Chapter 2 in my book - In Evidence We Trust- provides information on the science of critical thinking / rational thinking.  

References are available upon request. 

Exercise Does The Brain Good!

Exercise may lead to a wide range of benefits- increased cardiovascular health, stronger bones and muscles, stronger connective tissue, and increased overall fitness and athleticism.  There is a plethora of evidence that shows exercise is beneficial to the brain (Fernandez et al., 2013).  Research using various methods from a variety of domains supports the finding.  

A recent study, conducted in the Netherlands, found evidence that people who utilized an exercise bike for 6 months experienced an increased connectivity and density in their brain’s white matter. This was seen in people with schizophrenia and people with no clinical diagnosis (Svatkova et al., 2015)  Abstract 

Another study, conducted in Taiwan, found that people with diabetes or metabolic syndrome who utilized a stationary bike for 12 weeks showed an increase in brain-derived neurotrophic factor (BDNF), a growth factor involved in supporting neural plasticity processes- growth and differentiation of new neurons and neuron connections (Tsai et al., 2015) Abstract 


Why do fitness professionals fail to mention that exercise benefits the brain? There are probably three primary reasons for this. First, they are not familiar with the research, which is usually conducted in the field of brain science, as opposed to exercise science. Second, the subject matter can be intimidating – the brain is arguably the most complex structure in existence. Third, they have minimal knowledge of the brain and brain processes, thus they would rather not discuss the topic (brain is often the domain of cognitive, behavioral and neuroscientists).

Some of the key mechanisms mediating the effects of exercise on the brain:

Neurogenesis
Synaptic plasticity Angiogenesis & vascular growth factors
Neurotransmitters & growth factors.
Synaptic plasticity
Spine density

In my seminar -Your Brain & Exercise- these mechanisms are discussed in detail.  Other topics discussed: why health pros fail to mention brain benefits of exercise, physical activity and school curriculum, exercise and Parkinson’s, depression, stroke, neuroplasticity demystified, what type of exercise (aerobic vs. anaerobic), exercise recommendations for brain health, future research directions, etc. 


In conclusion, exercise offers an array of benefits.  Brain health is imperative to overall health.  The brain is part of the body, and should be referred as so.  Discard use of the phrases “brain and body” and “mental and physical.” The brain is part of the body, and all mental processes emanate from a physical structure: the brain.  

Further Reading: 

Exercise and The Brain   

 Your Brain on Exercise


Exercise Benefits Individuals with Parkinson’s Disease 


Online Store