Statistical process control
This section needs additional citations for verification. (March 2022)
Statistical process control (SPC) or statistical quality control (SQC) is the application of statistical methods to monitor and control the quality of a production process. This helps to ensure that the process operates efficiently, producing more specification-conforming products with less waste (rework or scrap). SPC can be applied to any process where the "conforming product" (product meeting specifications) output can be measured. Key tools used in SPC include run charts, control charts, a focus on continuous improvement, and the design of experiments. An example of a process where SPC is applied is manufacturing lines.
SPC must be practiced in two phases: The first phase is the initial establishment of the process, and the second phase is the regular production use of the process. In the second phase, a decision of the period to be examined must be made, depending upon the change in 5M&E conditions (Man, Machine, Material, Method, Movement, Environment) and wear rate of parts used in the manufacturing process (machine parts, jigs, and fixtures).
An advantage of SPC over other methods of quality control, such as "inspection," is that it emphasizes early detection and prevention of problems, rather than the correction of problems after they have occurred.
In addition to reducing waste, SPC can lead to a reduction in the time required to produce the product. SPC makes it less likely the finished product will need to be reworked or scrapped.
Statistical process control was pioneered by Walter A. Shewhart at Bell Laboratories in the early 1920s. Shewhart developed the control chart in 1924 and the concept of a state of statistical control. Statistical control is equivalent to the concept of exchangeability developed by logician William Ernest Johnson also in 1924 in his book Logic, Part III: The Logical Foundations of Science. Along with a team at AT&T that included Harold Dodge and Harry Romig he worked to put sampling inspection on a rational statistical basis as well. Shewhart consulted with Colonel Leslie E. Simon in the application of control charts to munitions manufacture at the Army's Picatinny Arsenal in 1934. That successful application helped convince Army Ordnance to engage AT&T's George Edwards to consult on the use of statistical quality control among its divisions and contractors at the outbreak of World War II.
W. Edwards Deming invited Shewhart to speak at the Graduate School of the U.S. Department of Agriculture and served as the editor of Shewhart's book Statistical Method from the Viewpoint of Quality Control (1939), which was the result of that lecture. Deming was an important architect of the quality control short courses that trained American industry in the new techniques during WWII. The graduates of these wartime courses formed a new professional society in 1945, the American Society for Quality Control, which elected Edwards as its first president. Deming travelled to Japan during the Allied Occupation and met with the Union of Japanese Scientists and Engineers (JUSE) in an effort to introduce SPC methods to Japanese industry .
'Common' and 'special' sources of variation
Shewhart read the new statistical theories coming out of Britain, especially the work of William Sealy Gosset, Karl Pearson, and Ronald Fisher. However, he understood that data from physical processes seldom produced a normal distribution curve (that is, a Gaussian distribution or 'bell curve'). He discovered that data from measurements of variation in manufacturing did not always behave the same way as data from measurements of natural phenomena (for example, Brownian motion of particles). Shewhart concluded that while every process displays variation, some processes display variation that is natural to the process ("common" sources of variation); these processes he described as being in (statistical) control. Other processes additionally display variation that is not present in the causal system of the process at all times ("special" sources of variation), which Shewhart described as not in control.
Application to non-manufacturing processes
Statistical process control is appropriate to support any repetitive process, and has been implemented in many settings where for example ISO 9000 quality management systems are used, including financial auditing and accounting, IT operations, health care processes, and clerical processes such as loan arrangement and administration, customer billing etc. Despite criticism of its use in design and development, it is well-placed to manage semi-automated data governance of high-volume data processing operations, for example in an enterprise data warehouse, or an enterprise data quality management system. 
In the 1988 Capability Maturity Model (CMM) the Software Engineering Institute suggested that SPC could be applied to software engineering processes. The Level 4 and Level 5 practices of the Capability Maturity Model Integration (CMMI) use this concept.
In No Silver Bullet, Fred Brooks points out that the complexity, conformance requirements, changeability, and invisibility of software results in inherent and essential variation that cannot be removed. This implies that SPC is less effective in the software development than in, e.g., manufacturing.
Variation in manufacturing
In manufacturing, quality is defined as conformance to specification. However, no two products or characteristics are ever exactly the same, because any process contains many sources of variability. In mass-manufacturing, traditionally, the quality of a finished article is ensured by post-manufacturing inspection of the product. Each article (or a sample of articles from a production lot) may be accepted or rejected according to how well it meets its design specifications, SPC uses statistical tools to observe the performance of the production process in order to detect significant variations before they result in the production of a sub-standard article. Any source of variation at any point of time in a process will fall into one of two classes.
- (1) Common causes
- 'Common' causes are sometimes referred to as 'non-assignable', or 'normal' sources of variation. It refers to any source of variation that consistently acts on process, of which there are typically many. This type of causes collectively produce a statistically stable and repeatable distribution over time.
- (2) Special causes
- 'Special' causes are sometimes referred to as 'assignable' sources of variation. The term refers to any factor causing variation that affects only some of the process output. They are often intermittent and unpredictable.
Most processes have many sources of variation; most of them are minor and may be ignored. If the dominant assignable sources of variation are detected, potentially they can be identified and removed. When they are removed, the process is said to be 'stable'. When a process is stable, its variation should remain within a known set of limits. That is, at least, until another assignable source of variation occurs.
For example, a breakfast cereal packaging line may be designed to fill each cereal box with 500 grams of cereal. Some boxes will have slightly more than 500 grams, and some will have slightly less. When the package weights are measured, the data will demonstrate a distribution of net weights.
If the production process, its inputs, or its environment (for example, the machine on the line) change, the distribution of the data will change. For example, as the cams and pulleys of the machinery wear, the cereal filling machine may put more than the specified amount of cereal into each box. Although this might benefit the customer, from the manufacturer's point of view it is wasteful, and increases the cost of production. If the manufacturer finds the change and its source in a timely manner, the change can be corrected (for example, the cams and pulleys replaced).
From an SPC perspective, if the weight of each cereal box varies randomly, some higher and some lower, always within an acceptable range, then the process is considered stable. If the cams and pulleys of the machinery start to wear out, the weights of the cereal box might not be random. The degraded functionality of the cams and pulleys may lead to a non-random linear pattern of increasing cereal box weights. We call this common cause variation. If, however, all the cereal boxes suddenly weighed much more than average because of an unexpected malfunction of the cams and pulleys, this would be considered a special cause variation.
The application of SPC involves three main phases of activity:
- Understanding the process and the specification limits.
- Eliminating assignable (special) sources of variation, so that the process is stable.
- Monitoring the ongoing production process, assisted by the use of control charts, to detect significant changes of mean or variation.
The data from measurements of variations at points on the process map is monitored using control charts. Control charts attempt to differentiate "assignable" ("special") sources of variation from "common" sources. "Common" sources, because they are an expected part of the process, are of much less concern to the manufacturer than "assignable" sources. Using control charts is a continuous activity, ongoing over time.
When the process does not trigger any of the control chart "detection rules" for the control chart, it is said to be "stable". A process capability analysis may be performed on a stable process to predict the ability of the process to produce "conforming product" in the future.
A stable process can be demonstrated by a process signature that is free of variances outside of the capability index. A process signature is the plotted points compared with the capability index.
When the process triggers any of the control chart "detection rules", (or alternatively, the process capability is low), other activities may be performed to identify the source of the excessive variation. The tools used in these extra activities include: Ishikawa diagram, designed experiments, and Pareto charts. Designed experiments are a means of objectively quantifying the relative importance (strength) of sources of variation. Once the sources of (special cause) variation are identified, they can be minimized or eliminated. Steps to eliminating a source of variation might include: development of standards, staff training, error-proofing, and changes to the process itself or its inputs.
Process stability metrics
When monitoring many processes with control charts, it is sometimes useful to calculate quantitative measures of the stability of the processes. These metrics can then be used to identify/prioritize the processes that are most in need of corrective actions. These metrics can also be viewed as supplementing the traditional process capability metrics. Several metrics have been proposed, as described in Ramirez and Runger. They are (1) a Stability Ratio which compares the long-term variability to the short-term variability, (2) an ANOVA Test which compares the within-subgroup variation to the between-subgroup variation, and (3) an Instability Ratio which compares the number of subgroups that have one or more violations of the Western Electric rules to the total number of subgroups.
Mathematics of control charts
Digital control charts use logic-based rules that determine "derived values" which signal the need for correction. For example,
- derived value = last value + average absolute difference between the last N numbers.
- ANOVA Gauge R&R
- Distribution-free control chart
- Electronic design automation
- Industrial engineering
- Process Window Index
- Process capability index
- Quality assurance
- Reliability engineering
- Six sigma
- Stochastic control
- Total quality management
- Barlow & Irony (1992)
- Bergman (2009)
- Zabell (1992)
- Deming, W. Edwards, Lectures on statistical control of quality., Nippon Kagaku Gijutsu Remmei, 1950
- Deming, W. Edwards and Dowd S. John (translator) Lecture to Japanese Management, Deming Electronic Network Web Site, 1950 (from a Japanese transcript of a lecture by Deming to "80% of Japanese top management" given at the Hotel de Yama at Mr. Hakone in August 1950)
- Why SPC?. SPC Press, Inc. British Deming Association. 1992.
- Larry English Improving Data Warehouse and Business Information Quality : Methods for Reducing Costs and Increasing Profits 1999
- Bob Raczynski and Bill Curtis (2008) Software Data Violate SPC's Underlying Assumptions, IEEE Software, May/June 2008, Vol. 25, No. 3, pp. 49-51
- Robert V. Binder (1997) Can a Manufacturing Quality Model Work for Software?, IEEE Software, September/October 1997, pp. 101-105
- Raczynski, Bob (February 20, 2009). "Is Statistical Process Control Applicable to Software Development Processes?". StickyMinds.
- Brooks, F. P., J. (1987). "No Silver Bullet—Essence and Accidents of Software Engineering" (PDF). Computer. 20 (4): 10–19. CiteSeerX 10.1.1.117.315. doi:10.1109/MC.1987.1663532.
- Fred P. Brooks (1986) No Silver Bullet — Essence and Accident in Software Engineering, Proceedings of the IFIP Tenth World Computing Conference 1986, pp. 1069–1076
- Ramirez, B.; Runger, G. (2006). "Quantitative Techniques to Evaluate Process Stability". Quality Engineering. 18 (1). pp. 53–68. doi:10.1080/08982110500403581.
- Barlow, R. E. & Irony, T. Z. (1992) "Foundations of statistical quality control" in Ghosh, M. & Pathak, P.K. (eds.) Current Issues in Statistical Inference: Essays in Honor of D. Basu, Hayward, CA: Institute of Mathematical Statistics, 99–112.
- Bergman, B. (2009) "Conceptualistic Pragmatism: A framework for Bayesian analysis?", IIE Transactions, 41, 86–93
- Deming, W E (1975) "On probability as a basis for action", The American Statistician, 29(4), 146–152
- — (1982) Out of the Crisis: Quality, Productivity and Competitive Position ISBN 0-521-30553-5
- Grant, E. L. (1946) Statistical quality control ISBN 0071004475
- Oakland, J (2002) Statistical Process Control ISBN 0-7506-5766-9
- Salacinski, T (2015) SPC - Statistical Process Control. The Warsaw University of Technology Publishing House. ISBN 978-83-7814-319-2
- Shewhart, W A (1931) Economic Control of Quality of Manufactured Product ISBN 0-87389-076-0
- — (1939) Statistical Method from the Viewpoint of Quality Control ISBN 0-486-65232-7
- Statistical Process Control (SPC) Reference Manual (2 ed.). Automotive Industry Action Group (AIAG). 2005.
- Wheeler, D J (2000) Normality and the Process-Behaviour Chart ISBN 0-945320-56-6
- Wheeler, D J & Chambers, D S (1992) Understanding Statistical Process Control ISBN 0-945320-13-2
- Wheeler, Donald J. (1999). Understanding Variation: The Key to Managing Chaos - 2nd Edition. SPC Press, Inc. ISBN 0-945320-53-1.
- Wise, Stephen A. & Fair, Douglas C (1998). Innovative Control Charting: Practical SPC Solutions for Today's Manufacturing Environment. ASQ Quality Press. ISBN 0-87389-385-9
- Zabell, S. L. (1992). "Predicting the unpredictable". Synthese. 90 (2): 205. doi:10.1007/bf00485351.