When testing scientific hypotheses- predicted outcome of study involving potential relationships between at least two variables- scientists are not attempting to prove their hypotheses, but are attempting to falsify them. Offering proof for a hypothesis is logically impossible. There are too many alternative possibilities that could explain the outcome, and in order to prove something is true would mean saying it was true every time.
Scientists set up hypotheses that they attempt to falsify / disprove. Two mutually exclusive hypotheses are formed with the intent of falsifying one while gaining support for the other. The null hypothesis- no relationship, no difference- predicts when comparing different groups there will be no difference. The alternative hypothesis- there is a difference- predicts when groups are compared there will be a difference. An alternative hypothesis can be one tailed / directional- predicting the direction of the relationship or it can be two tailed- not predicting the direction of the relationship. With hypothesis testing- testing hypothesis in a research to study in order to determine whether we support or do not find support- we are attempting to falsify the null hypothesis. After forming the hypotheses and determining a significance level- criteria for rejecting null hypothesis- data is collected which supports or does not support the null hypothesis. The significance level or alpha level is usually set at .05. This means that we are highly confident that we are correct if we have determined that there is a less than 5% chance the null hypothesis is correct- we reject the null hypothesis. By default, when we reject the null hypothesis we infer that the alternative hypothesis is correct. The p-value can be defined as the probability that the null hypothesis is true, or the probability that the observed effects occurred due to chance. While the confidence level can be reflected as 1- p value. When the significance level is .05 we can say our confidence level is 95% that we have inferred the appropriate conclusion.
If the null hypothesis is rejected we can say there is evidence for a relationship. If we fail to reject the null hypothesis we can say there in no evidence of a relationship. It is important to be cognizant of the wording used when NHST. We use the words support and unsupport, rather than proof. Proof, as pointed out earlier is a logical impossibility. It is also important to point out if a hypothesis is unfalsifiable it is untestable and thus unscientific.
There is always a chance of our inferences being incorrect. When testing the null hypothesis there are four possible outcomes
Type 1 error- rejecting the null hypothesis when it is true
Correct- rejecting the null hypothesis when it is false
Type 2 error- failing to reject the null hypothesis when it is false
Correct- failing to reject the null hypothesis when it is true