Focus area: Transforming Processes
Format: Interactive Teaching + Live Practice
Duration: ~4 Hours
Audience: Engineers & Quality Professionals
Jump to Workshop Sections
1. Introduction: The Tool That Looks Harder Than It Is
Design of Experiments (DOE) occupies a peculiar position in the quality professional's toolkit: it is almost universally recognized as one of the most powerful methods for understanding and optimizing processes, and it is almost universally avoided by everyone who has not been formally trained in statistics. The name alone — 'Design of Experiments' — sounds like something that happens in a white coat behind locked laboratory doors, requiring advanced degrees and specialized software.
This session tears down that myth. The core concepts of DOE — understanding how multiple input variables simultaneously affect an output variable, and determining which variables matter most and by how much — are accessible to any engineer, quality professional, or manager who can follow a structured table and interpret a graph. The mathematics behind full factorial DOE with three factors and two levels per factor can be worked by hand, with nothing more than addition, subtraction, and division.
More importantly, DOE solves a problem that affects every organization: the inefficiency of changing one factor at a time (OFAT) when trying to optimize a process. OFAT experiments not only take longer to produce results — they systematically miss the interaction effects between variables that are often the most important information available about a process.
"DOE does not require a statistics PhD. It requires curiosity about your process, patience to set up a structured experiment, and the ability to read a table. Those three things you already have."
2. Why One-Factor-at-a-Time (OFAT) Fails
2.1 The Problem With 'Changing One Thing at a Time'
The intuitive approach to process optimization — change one variable, see what happens, then change another — has a fundamental flaw: it cannot detect interactions between variables. An interaction exists when the effect of one variable on the output depends on the level of another variable. Interactions are extremely common in real processes, and they are completely invisible to OFAT experimentation.
A Concrete Example: The Coffee Problem
Suppose you want to optimize a beverage process for taste (output). You have two variables to test: water temperature (Low = 180°F, High = 200°F) and brew time (Low = 3 minutes, High = 5 minutes).
If you run OFAT experiments — first test temperature while holding time constant, then test time while holding temperature constant — you might find that higher temperature improves taste and longer time improves taste, and conclude that the optimal setting is high temperature + long time.
But what if the interaction tells a different story? What if high temperature AND long time produces a bitter, over-extracted result (actually worse than baseline), while the optimal combination is high temperature + short time? OFAT would miss this completely — because you never tested those combinations simultaneously in a way that reveals the interaction.
DOE, by testing all combinations systematically, reveals the interaction — and points you to the actual optimum rather than the false optimum that OFAT would deliver.
2.2 The Efficiency Advantage
Beyond revealing interactions, DOE is more efficient than OFAT for understanding multiple variables. Consider three variables (A, B, C) each at two levels:
- OFAT approach: Test variable A (2 runs) + test variable B (2 runs) + test variable C (2 runs) = minimum 6 runs, no interaction information, conclusions about each variable depend on the fixed level of other variables.
- Full 2-level factorial DOE: 2³ = 8 runs, estimates all three main effects, all three two-way interactions, and the three-way interaction. Every variable is tested at each level while the other variables vary — producing estimates that average across all conditions.
- Net result: 8 runs vs. 6 runs — the DOE costs only 2 additional runs but delivers exponentially more information, including the interaction effects that OFAT cannot see.
3. Full Factorial 2-Level DOE: The Fundamentals
3.1 Key Terminology
| Term | Definition | Example |
|---|---|---|
| Factor | An input variable tested in the experiment. Chosen because it is suspected to affect the output. | Temperature, pressure, catalyst concentration, operator experience level, cycle time. |
| Level | The values at which each factor is set during the experiment. Full factorial uses two levels: Low (-1) and High (+1). | Temperature: Low = 200°F, High = 250°F. Pressure: Low = 50 PSI, High = 100 PSI. |
| Run | One experimental condition — a specific combination of factor levels. Each row in the DOE matrix is one run. | Run 5: Temperature = High, Pressure = Low, Catalyst = High. |
| Response (Output) | The measured result of each experimental run. The variable the experiment is trying to understand and optimize. | Yield percentage, tensile strength, defect rate, customer satisfaction score. |
| Main Effect | The average change in the response when a factor moves from Low to High, averaged across all levels of all other factors. | Moving temperature from Low to High increases yield by an average of 4.2%, averaged across all pressure and catalyst conditions tested. |
| Interaction Effect | The change in the effect of one factor depending on the level of another factor. | The effect of temperature on yield is +8% when pressure is High, but only +0.4% when pressure is Low. Temperature and pressure interact. |
3.2 The 2³ Full Factorial Design Matrix
For three factors (A, B, C) each at two levels, the full factorial design requires 2³ = 8 runs. The standard design matrix lists all combinations systematically, using -1 for Low and +1 for High:
| Run | A | B | C | AB Interaction | Response (Y) |
|---|---|---|---|---|---|
| 1 | -1 | -1 | -1 | +1 | Measure here |
| 2 | +1 | -1 | -1 | -1 | Measure here |
| 3 | -1 | +1 | -1 | -1 | Measure here |
| 4 | +1 | +1 | -1 | +1 | Measure here |
| 5 | -1 | -1 | +1 | +1 | Measure here |
| 6 | +1 | -1 | +1 | -1 | Measure here |
| 7 | -1 | +1 | +1 | -1 | Measure here |
| 8 | +1 | +1 | +1 | +1 | Measure here |
Note: The AB interaction column is computed by multiplying the A and B columns element-by-element: (-1)×(-1) = +1, (+1)×(-1) = -1, etc. Every interaction column is generated this way. This is the 'secret' of DOE: the structure of the design matrix makes all calculations possible with simple arithmetic.
3.3 Calculating Main Effects and Interactions by Hand
The main effect of any factor is calculated using the simplest possible arithmetic: average the response values where the factor is at High (+1), then subtract the average response where the factor is at Low (-1).
- Main Effect of A = Average(Y when A = +1) − Average(Y when A = −1)
- This averages across all combinations of the other factors — which is the mathematical property that makes factorial designs more efficient than OFAT.
- Interaction Effect of AB = Average(Y when AB column = +1) − Average(Y when AB column = −1)
This is the complete calculation. No regression software required. A calculator and the matrix above are all you need to compute every main effect and two-way interaction for a 2³ full factorial DOE. The 'complexity' of DOE is almost entirely in the setup and interpretation — not in the arithmetic.
4. Worked Example: Optimizing a Tablet Coating Process
4.1 Experiment Setup
A pharmaceutical manufacturer wants to optimize the appearance quality (measured 1–10, higher is better) of tablet coating. Three factors are suspected to influence appearance:
- Factor A — Coating Solution Viscosity: Low (-1) = 200 cP, High (+1) = 400 cP
- Factor B — Pan Speed: Low (-1) = 15 RPM, High (+1) = 25 RPM
- Factor C — Inlet Air Temperature: Low (-1) = 40°C, High (+1) = 60°C
4.2 Experimental Results
| Run | A | B | C | AB | AC | Appearance Score (Y) |
|---|---|---|---|---|---|---|
| 1 | -1 | -1 | -1 | +1 | +1 | 6.2 |
| 2 | +1 | -1 | -1 | -1 | -1 | 7.1 |
| 3 | -1 | +1 | -1 | -1 | +1 | 5.8 |
| 4 | +1 | +1 | -1 | +1 | -1 | 8.4 |
| 5 | -1 | -1 | +1 | +1 | -1 | 6.5 |
| 6 | +1 | -1 | +1 | -1 | +1 | 7.8 |
| 7 | -1 | +1 | +1 | -1 | -1 | 6.0 |
| 8 | +1 | +1 | +1 | +1 | +1 | 8.9 |
4.3 Calculating the Main Effect of Factor A (Viscosity)
Runs where A = +1 (High Viscosity): Runs 2, 4, 6, 8
Response values: 7.1, 8.4, 7.8, 8.9 → Average = (7.1 + 8.4 + 7.8 + 8.9) / 4 = 8.05
Runs where A = -1 (Low Viscosity): Runs 1, 3, 5, 7
Response values: 6.2, 5.8, 6.5, 6.0 → Average = (6.2 + 5.8 + 6.5 + 6.0) / 4 = 6.125
Main Effect of A = 8.05 − 6.125 = +1.925
Interpretation: Moving viscosity from Low to High improves appearance score by 1.925 points on average, across all combinations of pan speed and air temperature tested.
4.4 Summary Results
| Effect | Calculated Value | Practical Interpretation |
|---|---|---|
| Main Effect A (Viscosity) | +1.925 | Largest main effect. High viscosity consistently improves appearance. Prioritize for optimization. |
| Main Effect B (Pan Speed) | +0.875 | Moderate positive effect. Higher pan speed improves appearance, but less dramatically than viscosity. |
| Main Effect C (Temp.) | +0.475 | Small positive effect. Higher temperature slightly improves appearance. |
| AB Interaction (Visc×Speed) | +0.975 | Significant interaction. The benefit of high viscosity is amplified when pan speed is also high. Set both to High for maximum benefit. |
| AC Interaction (Visc×Temp.) | -0.225 | Small interaction. Negligible practical significance. |
Conclusion: The DOE reveals that the optimal process settings are High Viscosity + High Pan Speed (AB interaction is positive, meaning these two work better together than either does alone). This would not have been discovered by OFAT experimentation, because the interaction between A and B is only visible when both are varied simultaneously.
5. Interaction Graphs: Making Interactions Visible
5.1 Reading an Interaction Plot
The interaction plot is the most intuitive way to visualize and interpret interaction effects. It shows the response at each level of one factor, plotted separately for each level of a second factor. When the lines are parallel, there is no interaction. When the lines cross or diverge significantly, an interaction is present.
For the AB interaction in our example:
- When A is Low (Low Viscosity): Effect of B (Pan Speed) on appearance = from ~6.0 to ~5.9. Nearly flat — pan speed barely matters at low viscosity.
- When A is High (High Viscosity): Effect of B (Pan Speed) on appearance = from ~7.45 to ~8.65. Steep positive slope — pan speed matters a great deal at high viscosity.
The non-parallel lines confirm the interaction: Pan Speed matters when Viscosity is High, but not when Viscosity is Low. A practitioner who only tested pan speed at low viscosity would incorrectly conclude it has minimal impact on appearance.
5.2 When to Run a Full Factorial vs. Fractional Factorial
| Design Type | When to Use | Trade-Off |
|---|---|---|
| Full Factorial (2^k) | 3–5 factors, when interaction information is critical, when resources allow all runs. | All effects estimable. No assumptions required. Run count grows rapidly: 2³=8, 2⁴=16, 2⁵=32. |
| Fractional Factorial (2^(k-p)) | 5+ factors in an initial screening study. When some interactions can be assumed negligible. | Fewer runs (e.g., 2^(5-2) = 8 runs for 5 factors) but some effects are 'confounded' with others. |
| Plackett-Burman Design | Initial screening with many factors (7–23). Primary goal is identifying vital few significant factors. | Highly efficient. Designed for main effects only — significant interactions may be partially hidden. |
| Response Surface Methods (RSM) | After factorial screening identifies significant factors. Goal is finding the optimum operating point. | Uses star points beyond +1/-1 to fit curved (quadratic) response surfaces. Higher run count but maps the full response landscape. |
6. Workshop Flow for a 4-Hour Session
| Time Block | Duration | Content & Activities |
|---|---|---|
| 0:00 – 0:30 | 30 min | Opening: Why DOE and Why Now. Present the OFAT failure mode with the coffee interaction example. Poll: who has run a DOE? Who has wanted to but found it intimidating? Set the tone: we will work one through by hand today. |
| 0:30 – 1:15 | 45 min | DOE Fundamentals. Teach all key terminology with concrete examples. Walk through the 2³ design matrix, explaining why every combination must be run. Show how the AB column is generated. Groups practice generating AC and BC columns. |
| 1:15 – 2:00 | 45 min | Main Effect Calculations. Walk through the tablet coating example step by step. Groups calculate the main effect of Factor B (Pan Speed) and Factor C (Temperature) independently, then compare results. Discuss what each result means for process optimization. |
| 2:00 – 2:15 | 15 min | Break. Display the interaction plot concept. Groups sketch what a non-interacting (parallel lines) and interacting (crossing lines) plot looks like. |
| 2:15 – 3:00 | 45 min | Interaction Analysis. Calculate the AB interaction for the tablet coating example. Construct and interpret the interaction plot. What does it tell you that neither main effect alone could? Groups discuss: where in your own work would an interaction change your decisions? |
| 3:00 – 3:40 | 40 min | Live DOE Practice. Groups design a simple 2² or 2³ experiment for a real or realistic quality problem from their work. Define: factors, levels, response, run order, measurement plan. Present to the group and receive peer feedback. |
| 3:40 – 4:00 | 20 min | Design Selection and Q&A. Walk through the four design types and when to use each. Individual: identify one process in your work where DOE would be more informative than your current experimentation approach. Open Q&A. |
7. Discussion Questions for Q&A
Conceptual Understanding
- Think about the most significant process optimization effort you have been involved in. Was the experimentation approach OFAT or something structured? Were interaction effects considered? In retrospect, what might DOE have revealed that the actual approach missed?
- The AB interaction in the tablet coating example reveals that the effect of Pan Speed depends on Viscosity level. In the processes you work with, where do you most strongly suspect that important interactions exist between controllable factors?
- The full factorial design for 5 factors requires 2^5 = 32 runs. If resources allow only 16 runs, a half-fraction (2^(5-1)) is available. What is lost in the half-fraction, and how would you decide whether that trade-off is acceptable?
Application
- Design a 2³ factorial DOE for a real process improvement opportunity in your organization. Identify: the three factors you would test, their Low and High levels, the response variable you would measure, and any known constraints on running order or replications.
- After running a DOE, you find that Factor B has a large main effect but Factor A is involved in a significant AB interaction with B. What is the practical significance of the interaction for your operating decision, even if Factor A's main effect is small?
- When would you choose a Response Surface Method over a full factorial DOE? Describe a quality optimization scenario in your work where knowing the shape of the response surface (not just the direction of factor effects) would be critical.
8. Conclusion: The Experiment That Tells the Truth
OFAT experimentation tells you what happens when one thing changes. DOE tells you what happens when everything that matters changes together — which is a far more accurate model of how real processes work. Real processes are not controlled one factor at a time. They are influenced simultaneously by temperature and pressure and material lot and operator and ambient humidity. DOE is the experimental design that captures this reality.
The mathematics, as this session demonstrates, are accessible to anyone willing to follow a structured table and perform basic arithmetic. The challenge is not computational — it is conceptual: recognizing when a process question requires a designed experiment rather than an opportunistic observation, setting up that experiment correctly before running it, and interpreting the results with enough understanding to translate them into process decisions.
Quality professionals who develop this capability — who can design, run, and interpret a factorial experiment — will solve problems faster, find optima that OFAT would miss, and provide engineering insights that establish their value as analytical partners rather than compliance monitors. The tool is accessible. The results are powerful. The only remaining barrier is the decision to start.
If a 4th grader can do it... so can you. Design the experiment. Run the runs. Calculate the effects. Find the truth.
| KEY TAKEAWAYS 1. DOE is accessible without advanced statistics — the core 2-level factorial calculation requires only averaging and subtraction. 2. OFAT experimentation is systematically blind to interaction effects — the most important information for process optimization in many real systems. 3. The 2³ full factorial design tests all 8 combinations of 3 factors at 2 levels, estimating all main effects and all two-way interactions in just 8 runs. 4. Interaction plots (response vs. one factor, stratified by another) are the most intuitive tool for visualizing and communicating interaction effects. 5. Design selection (full factorial, fractional factorial, Plackett-Burman, RSM) depends on the number of factors, the importance of interactions, and the available run budget. |