Jamie Hale

Jamie Hale

Thursday, January 11, 2018

Peer Review is not the Antidote



 In the Peer Review Process a paper is submitted to a journal and evaluated by several reviewers (often reviewers are individuals with an impressive history of work in the area of interest-that is, the specific area that the article addresses).  After critiquing the paper the reviewers submit their thoughts to the editor.  Then, based on the commentaries from the reviewers, the editor decides
whether to publish the paper, make suggestions for additional changes that could lead to publication, or to reject the paper. 

Single Blind and Double Blind Reviews

In Single Blind Reviews authors do not know who the reviewers are.  In Double Blind Reviews authors do not know who the reviewers are, nor do reviewers know the identity of the authors.  In many fields Single Blind Reviews are the norm, while in others Double Blind Reviews are preferred.
“Peer review is one way (replication is another) science institutionalizes the attitudes of objectivity and public criticism.  Ideas and experimentation undergo a honing process in which they are submitted to other critical minds for evaluation.  Ideas that survive this critical process have begun to meet the criterion of public verifiability” (Stanovich, 2007, p. 12).

Criticisms Peer Review Process

Reviewers find it hard to remain Purely Objective due to their own education, experience and biases

The process is slow Critics point out, and this may deter submission of quality papers  

There are many examples of poor research published in peer-reviewed journals, which indicates the peer review process is often unsuccessful in preventing the publication of  bad science

Sometimes good research is not published, especially when findings are not statistically significant.  Publication bias is problematic and demonstrates confirmation bias

Reviewers are not always knowledgeable regarding contents they are reviewing (I have been asked to review papers that consist of contents that was outside my area of knowledge)

Lack of agreement and communication among reviewers

Reviewers tend to be highly critical of articles that contradict their own views, while being less critical of articles that support their personal views (example of myside bias). Well-known, established scientists are more likely to be recruited as reviewers

Lacks standardized criteria, and criteria for publication demonstrates huge variability among scholarly publications.

Final word-Peer Review Process
The Peer Review Process is not perfect, but some researchers suggest  it is one of the best safeguards we have against junk science (Stanovich, 2007). When evaluating the worth of scientific data, in addition to whether it is published in a peer reviewed journal, it is important to take into consideration:  funding sources, study replication, study design, sample size, conflicting interest, sampling error, different measures of reliability and validity, reporting limitations and other possible criticisms of study (special concern with *statistical validity- often not acknowledged or understood). 

There are good studies that never get published in peer review publications, and low quality studies are published by peer review publishers.  It is erroneous to label a study, review, commentary, meta-analysis or any other scholarly papers as high quality based solely on peer review status. This over glorification of peer review pervades academia and pop science.      

* When researchers question a study’s statistical validity they are questioning issues relevant to how well the conclusions coincide with the results, represented as statistics. Interrogating statistical validity may include some of the following questions: If the study found a difference what is the probability that the conclusion was a false alarm?  If the study’s finding found no difference what is the probability that a real relationship went unnoticed?  What is the effect size?  Is the difference between groups statistically significant? Are the finding practically significant? What type of inferential stats were used to assess predictions? Could different statistical procedures have been used? How would different samples influence statistical findings?  Stats make use of samples; inferences about the population are derived from data collected in the study.  It is important to avoid an exaggeration of the findings, consider sampling error, consider external validity and consider the need for converging evidence, and what the finding indicate regarding a representation of the population.


Addendum- 2-9-18



from When Does Peer Review Make No Damn Sense:


"What sort of errors can we expect peer review to catch? ...I’m well placed to answer this question as I’ve published hundreds of peer-reviewed papers and written thousands of referee reports for journals. And of course I’ve also done a bit of post-publication review in recent years.

To jump to the punch line: the problem with peer review is with the peers.


In short, if an entire group of peers has a misconception, peer review can simply perpetuate error. We’ve seen this a lot in recent years, for example that paper on ovulation and voting was reviewed by peers who didn’t realize the implausibility of 20-percentage-point vote swings during the campaign, peers who also didn’t know about the garden of forking paths. That paper on beauty and sex ratio was reviewed by peers who didn’t know much about the determinants of sex ratio and didn’t know much about the difficulties of estimating tiny effects from small sample sizes.

OK, let’s step back for a minute. What is peer review good for? Peer reviewers can catch typos, they can catch certain logical flaws in an argument, they can notice the absence of references to the relevant literature—that is, the literature that the peers are familiar with. That’s how the peer reviewers for that psychology paper on ovulation and voting didn’t catch the error of claiming that days 6-14 were the most fertile days of the cycle: these reviewers were peers of the people who made the mistake in the first place!

Peer review has its place. But peer reviewers have blind spots. If you want to really review a paper, you need peer reviewers who can tell you if you’re missing something within the literature—and you need outside reviewers who can rescue you from groupthink. If you’re writing a paper on himmicanes and hurricanes, you want a peer reviewer who can connect you to other literature on psychological biases, and you also want an outside reviewer—someone without a personal and intellectual stake in you being right—who can point out all the flaws in your analysis and can maybe talk you out of trying to publish it.

Peer review is subject to groupthink, and peer review is subject to incentives to publishing things that the reviewers are already working on."


From: Evidence on peer review—scientific quality control or smokescreen?

"Summary points


  • Blinding reviewers to the author’s identity does not usefully improve the quality of reviews
  • Passing reviewers’ comments to their co-reviewers has no effect on quality of review
  • Reviewers aged under 40 and those trained in epidemiology or statistics wrote reviews of slightly better quality
  • Appreciable bias and parochialism have been found in the peer review system
  • Developing an instrument to measure manuscript quality is the greatest challenge" 


Even though not labeled as peer review, popular pubs often use peer review. That is, articles are reviewed by individuals that can be considered peers, suggestions are made regarding changes to article, and author responds and revises relevant to suggestions. So, the claim that only scholarly like "peer review" pubs involve reviewing and revising in accordance to others suggestions i incorrect. But, peer review journals, as a whole, publish higher quality information correct? Yes. To reiterate, the peer review process has strengths, but it has limitations, and it shouldn't be used "alone" as a marker of quality.

Read these:

Effects of  Editorial Peer Review

Effectsof Training on Quality of Peer Review 

 

 
   



Wednesday, December 13, 2017

The Sugar Brain is not Unique



In the context of this article the Sugar-Brain refers to neural correlates regarding sugar consumption. Firstly, information is given on the classification of sugar, a brief mention of different categories of sugar, and the properties underpinning different sugars.  Secondly, a concise overview of brain reward mechanisms is appropriate.

The term sugar, in everyday discussion, generally refers to sweets or highly processed sweet tasting food. The term is excessively ambiguous, so when discussing in everyday conversation it is not always clear what one is referring to.  I usually assume the  referent is highly processed, calorie dense, sweet food, in most cases.
  
Sugar, in the context of science, is defined precisely and is dependent on various properties. Sugar (of all types ) is a carbohydrate.  The term carbohydrate was originally used to describe compounds that were hydrates of carbon; they have an empirical formula of CH2O. In my book- The Carbohydrate Files 2nd editon (2007)- I presented  two classification systems: Basic
Carbohydrate Classification System (BCCS) and the Advanced Carbohydrate Classification System (ACCS). A concise overview of the ACCS is presented here.
 
Carbohydrates are polyhydroxy aldehydes, ketones, alcohols, acids, their simple derivatives and
their polymers having linkages of the acetal (any organic compound formed by adding alcohol
molecules to aldehyde molecules) type. They may be classified according to their degree of
polymerization ((Polymerization is a chemical process that combines several monomers to form a polymer or polymeric compound.) and may be divided into three principal groups: sugars, oligosaccharides and polysaccharides. Sugars are the focus in the current article.

Monosaccharides, the basic units of a carbohydrate, are white crystalline solids that may be divided into aldoses, which have an aldehyde group on the first carbon atom, and ketoses, which typically have a ketone group on the second carbon atom. They may also be divided according to the number of carbon atoms that they contain. The suffix “-ose” is used to identify these molecules as sugars.

Triose: monosaccharide with three carbons
Tetrose: monosaccharide with four carbons
Pentose: monosaccharide with five carbons
Hexose: monosaccharide with six carbons
Heptose: monosaccharide with seven carbons

Sugars include monosaccharides, disaccharides, and polyols (sugar alcohols). The
monosaccharides are categorized as glucose, fructose, and galactose (formula C6H12O6). The disaccharides are composed of two monosaccharide units. The binding between the two
sugars results in the loss of a hydrogen atom (H) from one molecule and a hydroxyl group (OH)
from the other. The disaccharides are categorized as sucrose, lactose, trehalose, and maltose
(formula C12H22O11). Sugar alcohols, technically called polyols, are are found in numerous sports drinks, bars, and various other types of sweets. Polyols are generally known as sugar-free
sweeteners. They are carbohydrates, but they aren’t sugars. Chemically, polyols are considered polyhydric alcohols or sugar alcohols because part of their structure resembles sugar and part is similar to alcohols. However, these sugar-free sweeteners are neither sugars nor alcohols. The most widely used polyols are sorbitol, mannitol, and maltitol.

Proponents of the Sugar-Brain claim that consumption of sugar can activate the same brain reward mechanisms (dopamine pathway referred to as mesolimbic dopamine system) as the consumption of addictive drugs. This is true; so it is not a myth?  You have probably seen photos comparing the sugar-brain with the drug addiction-brain. Hold on before jumping to conclusions. The problem with the claim is the way it is presented (implications) and inferences made regarding the claim. The claim is a rhetorical device used to convince people that sugar consumption is bad, like drug consumption.
What is the mesolimbic dopamine system? Drug researchers have traditionally identified the mesolimbic dopamine system as the brain system mostly involved with drug addiction. This system may be extended to include cortical areas (in PFC)- mesolimbic cortical dopamine system (Kandel, 2012). Some sources refer to these systems as being the same; however, the mesolimbic dopamine system can be more accurately described as projecting from the ventral tegmental area to the nucleus accumbens (NA, often referred to as the major pleasure center in the brain), while the mesolimbic cortical dopamine system projects from the ventral tegmental area and extends to areas in the PFC. Their distinction is not important in the context of this article (either are involved with drug consumption and brain reward mechanisms- reward/pleasure circuitry). These circuits are rich in dopaminergic neurons. Dopamine cell bodies are located in the ventral tegmental area and substantia nigra. The focus here, is on the projection from the ventral tegmental area.

Objects, stimuli, activities or internal physical states can serve as rewards for humans and non-human animals. Rewards have positive value and facilitate feelings of pleasure and positive emotion; they act as positive reinforcers. Not only do rewards lead to the activation of dopaminergic activity, but so does expectations or anticipation of rewards. "[T]he flow of dopamine is set off by the simplest expectation of pleasure, even though the pleasure may not materialize" (Kandel, 2012, p.428).
The brain's reward mechanisms are activated when we enjoy art, experience beautiful scenery, are exposed to attractive faces, listen to pleasant music, are exposed to humor or novelty, drive a sports car and experience romantic love. The Sugar-Brain could easily be called the Love-Brain. 
 
Evaluating brain imaging is complicated, and there is often disagreement among those highly qualified regarding implications of imaging. If sugar consumption is not a rewarding act what can we expect regarding activity of brain reward circuitry? It is a drastic over-simplification to suggest that - because, consuming sugar may lead to activation (reiterating- the variability  in activation is large) of brain reward mechanisms- it should be held in the same regards as drug use. "Dopamineric neurons in the striatum respond to all kinds of pleasure." Eric Kandel- Nobel Laureate

Science sounding information is often inserted to enhance persuasive value.

References available upon request