Design of Experiments (DOE)

Design of Experiments (DOE) is a study of the factors that the team has determined are the key process input variables (KPIV's) that are the source of the variation or have an influence on the mean of the output.

DOE are used by marketers, continuous improvement leaders, human resources, sales managers, engineers, and many others. When applied to a product or process the result can be increased quality performance, higher yields, lower variation of outputs, quicker development time, lower costs, and increased customer satisfaction.

It is a sequence of tests where the input factors are systematically changed according to a design matrix. The DOE study is first started by setting up an experiment with a specific number if runs with one of more factors (inputs) with each given two or more levels or settings.

This initial time and effort up front can be costly (this is up to the team to decide how many experiments to conduct) and time consuming but the end result will be the maximizing the outputs shown above in bold.

A DOE (or set of DOE's) will help develop a prediction equation for the process in terms of Y = f(X1,X2,X3,X4,....Xn).

GOAL:

1) Understand the influential variables

2) Quantify the effect of the variables on the outputs

For example, with two factors (inputs) each taking two levels, a factorial DOE will have four combinations. With two levels and two factors the DOE is termed a 2×2 factorial design. A memory tactic....Levels lie low and Factor Fly high

A DOE with 3 levels and 4 factors is a 3×4 factorial design with 81 treatment combinations. It may not be practical or feasible to run a full factorial (all 81 combinations) so a fractional factorial design is done, where usually half of the combinations are omitted.

Here are some characteristics of factorial experiments in general:

A Response is the output and is the dependent variable
Response = sum of process mean + variation about the mean
Factors are independent variables
Variation about the mean is sum of factors + interactions + unexplained residuals (or experimental error)

ANOVA is used to decompose the variation of the response to show the effect from each factor, interactions, and experimental error (or unexplained residual).

Statistical software will help manage the entire DOE.

Enter the factors
Set the levels (at least two for each factor)
Determine how many runs (full factorial, fractional factorial)
Run the experiment at each treatment level
Enter the response for each treatment level
Use statistical software to use ANOVA on the data
Continue to refine until prediction equation is obtained
IMPROVE the KPIV's
Last phase is CONTROL the KPIV's

Other methods of experimentation such as "trial and error" or "one factor at a time (OFAT)" are prone to waste, will provide less information and will not provide a prediction equation. These may seem easier to run and get results but the risk is a less robust solution and decisions made on a poor experiment.

These input factors behave to create an output, the team needs to make improvements in the IMPROVE phase that control the inputs. Controlling the input factors will provide the desired response.

The DOE will quantify the factor interactions and offer a prediction equation. This ANOVA will help indicate which factors and combinations are statistically significant and which are not thus giving direction to the priority of improvements.

DOE Assumptions since ANOVA is used to analyze the data:

The residuals are independent
The residuals have equal variance
The residuals are normally distributed
All inputs (factors) are independent of one another

Most prediction equations will be linear and reliable when using only two levels. This saves time, money, and other resources while obtaining a satisfactory prediction equation.

Prediction equations are useful to analyze what-if scenarios. Many times data can not be collected at all levels and factors so a prediction equation can be used to estimate the output.

The input factors are x's and the response is Y-hat.

Full Factorial

The following are characteristics of a Full Factorial DOE:

Usually large number of test needed. If the number of parameters is large then the number of test becomes significantly large (there are more and more interaction combinations and possibilities).
Testing every combination of factor levels.
Captures all interactions which of course is nice to have but this comes at a cost and time.

For instance, if there 9 factors and 3 levels for each factor that the team wants to test, then that is 3⁹ = 19,683 runs to determine all the interactions!

3 * 3 Full Factorial

Using the same vehicle throughout and maintaining all external variables as constant as possible a study is being created to find a prediction equation for the miles per gallon (MPG). There are 27 runs needed to bring out all the interactions (3³).

The team has determined that coefficient of friction of surface, ambient temperature, and tire pressure are three critical input factors (KPIV's) to study.

The goal isn't always to maximize MPG but to understand the impact on vehicle MPG based on these factors. The problem statement may be to improve the accuracy of MPG claims on this specific vehicle.

Fractional Factorial

A Fractional Factorial experiment uses subset of combinations from a Full Factorial experiment. They are applicable when there are too many inputs to screen practically or cost or time would be excessive.

This type of DOE involves less time than OFAT and a Full Fractional Factorial but this choice will result in less data and some interactions will be confounded (or aliased). This means that the effect of the factor cannot be mathematically distinguished from the effect of another factor.

Most processes are driven by main effects and lower order interactions so choose the higher order interactions for confounding. Lower confounding is found with higher resolution.

If a half fractional factorial experiment is determined to be most practical and economical where there are two levels and five factors then there will be a combination of 16 runs analyzed. Usually higher order interactions are omitted to focus on the main effects.

DOE jargon

Response (Y, KPOV): the process output linked to the customer CTQ. This is a dependent variable.

Factor (X, KPIV): uncontrolled or controlled variable whose influence is being studied. Also called independent variables.

Inference Space: operating range of factors under study

Factor Level: setting of a factor such as 1, -1, +, -, hi, low.

Treatment Combination (run): setting of all factors to obtain a response

Replicate: number of times a treatment combination is run (usually randomized)

ANOVA: Analysis of Variance

Blocking Variable: Variable that the experimenter chooses to control but is not the treatment variable of interest.

Interaction: occurrence when the effects of one treatment vary according to the levels of treatment of the other effect.

Confounding: variables that are not being controlled by the experimenter but can have an effect on the output of the treatment combination being studied. It describes the mixing of estimates of the effects from the factors and interactions.