Hypothesis Testing

Selecting the appropriate comparison test can be challenging especially in the learning stages. A Six Sigma project manager should understand the formulas and computations within the commonly applied tests.

In hypothesis testing, samples represents a small subset of the population which are used to infer conclusions about the population. There is always a chance or risk (known as alpha-risk and beta-risk) that the selected sample is not representative of the population and one could infer the incorrect conclusion. Assumptions are inferred that allows the estimation of the probability (known as p-value) of getting a wrong conclusion.

Statistical software has simplified the work to the point where comprehension of these tests is convenient to overlook.

CAUTION: A statistical difference doesn't always imply a practical difference, numbers don't always reflect reality.

Parametric Tests are used when:

Nonparametric tests are used when:

  • The above criteria are not met or if distribution is unknown:
  • These test are used when analyzing nominal or ordinal data.
  • Nonparametric test can also analyze interval or ratio data.

In general, the power of standard parametric tests are greater than the power of the alternative nonparametric test. As the sample size increases and becomes very large the power of the nonparametric test approaches its parametric alternative.

Nonparametric tests also assume that the underlying distributions are symmetric but not necessarily normal. When the choice exist on whether to use the parametric or nonparametric, if the distribution is fairly symmetric, the standard parametric tests are better choices than the nonparametric alternatives.

Comparison of Means using Parametric Tests

Comparison of Sample Means

Comparison of Variances

For 1 sample: Use Chi-square

For 2 samples: Use the F-Test or ANOVA for >2 variances. The F-test assumes the data is normal.

Levene's test is an option to compare variances of non-parametric data.

For >2 samples: Use Bartlett's Test for parametric data and Levene's Test for nonparametric data

Hypothesis Testing Steps

  1. Define the Problem
  2. State the Objectives
  3. Establish the Hypothesis (left-tailed, right-tailed, or two tailed test).
  4. State the Null Hypothesis (Ho)
  5. State the Alternative Hypothesis (Ha)
  6. Select the appropriate statistical test
  7. State the alpha-risk (α) level
  8. State the beta-risk (β) level
  9. Establish the Effect Size
  10. Create Sampling Plan, determine sample size
  11. Gather samples
  12. Collect and record data
  13. Calculate the test statistic
  14. Determine the p-value

If p-value < α, reject Ho and accept Ha

If p-value > α, fail to reject the Null, Ho

Try to re-run the test (if practical) to further confirm results. The next step is to take the statistical results and translate it to a practical solution.

It is also possible to determine the critical value of the test and use to calculated test statistic to determine the results. Either way, using the p-value approach or critical value should provide the same result.

Statistical Power 

The statistical power is 1 minus the beta-risk chosen. Usually the beta-risk is between 10-20% so Power typically range from 80-90%.

This is the likelihood of finding an effect when there is actually an effect. This is the chance of rejecting the null hypothesis when the null hypothesis is actually false. 

Statistical Power

Detectable Difference (δ)

A minimum detectable difference, δ, can also be specified. This detectable difference is used to examine a desired difference among:

  • Target (or given) value and a sample mean - using 1 sample t test
  • Two sample means - using Paired t or 2 sample t tests
  • > 2 sample means in ANOVA
  • Target (or given) value and a sample proportion
  • Two proportions


The minimum detectable difference desired relative to the standard deviation is the sensitivity of the test. It is the size of the difference expressed in standard deviations. 

Similar to the Coefficient of Variation in that the mean is expressed as relative magnitude in standard deviations. The numerator itself doesn't provide much information, it is when it (or the δ) are expressed in terms of standard deviations are you able to compare two or more values with more meaning. 


Create a Visual Aid of the Test

To simplify the process, break down the process into four small steps.

Create a table similar to the one below and begin by completing the top two quadrants. The bottom-left contains the results from the test and then converting those numbers into meaning is the practical result which belongs in the bottom-right quadrant.

Hypothesis Test Flow Chart

The null hypothesis is referred to as "Ho".

The alternate hypothesis is referred to as "HA".

This is the hypothesis being tested or the claim being tested. The null hypothesis is either "rejected" or "failed to reject". Rejecting the null hypothesis means accepting the alternative hypothesis.

The null hypothesis is valid until it is proven wrong. The burden of truth rest with the alternative hypothesis. This is done by collecting data and using statistics with a specified amount of certainty. The more samples of data usually equates as more evidence and reduces the risk of an improper decision.

The null hypothesis is never accepted, it can be "failed to reject" due to lack of evidence, just as a defendant is not proven guilty due to lack of evidence. The defendant is not necessarily innocent but is determined "not guilty".

There is simply not enough evidence and the decision is made that no change exists so the defendant started the trial as not guilty and leaves the trial not guilty.

Selecting the Hypothesis Test

If you have One X and One Y variable and......

Hypothesis Test Matrix 1 Y and 1 X

If you have >1 X and One Y variable and......

Hypothesis Testing Visual Aid

Hypothesis Test Module - Download

This module in PowerPoint provides lessons and more detail about commonly used hypothesis tests. This is often a new area of study for those learning about the Six Sigma methodology and represents a significant challenge on certification exams and in real-life application.

Click here to purchase the Hypothesis Test module
and view others that are available.

Hypothesis Testing Guide

Hypothesis Testing Guide

Hypothesis Testing on TI-83 or TI-84 Calculator

Hypothesis Testing using Excel

Return to the ANALYZE phase

Move on to the IMPROVE phase

Search active job openings related to Six Sigma

Return to Six-Sigma-Material Home Page

 Site Membership

Six Sigma Green Belt Certification
Black Belt Certification

Six Sigma

Templates & Calculators

Six Sigma Modules

The following presentations are available to download.

Click Here

Green Belt Program (1,000+ Slides)

Basic Statistics


Process Mapping

Capability Studies


Cause & Effect Matrix


Multivariate Analysis

Central Limit Theorem

Confidence Intervals

Hypothesis Testing

T Tests

1-Way Anova Test

Chi-Square Test

Correlation and Regression


Control Plan


Error Proofing

Statistics in Excel

Six Sigma & Lean Courses

Agile & Scrum Online Course