Probably one of the most significant concerns for anyone responsible for implementing, deploying and maintaining a quality management system is the prioritization of resources. Trend analysis is one technique that can help determine if something has changed with a process (quality, production, or service). Trend analysis can be used to monitor a process, especially non-manufacturing processes such as complaints, nonconformances and deviations to aid in the decision for escalation for corrective and preventive action (CAPA).
All quality practitioners realize there are unlimited needs and limited resources to support those needs. Trend analysis techniques can help an organization focus the company’s limited resources where they are needed the most. Trend analysis is simply using a statistically based control chart to monitor an activity or process. Just as traditional control charts used for monitoring a manufacturing or production process use statistical control limits based on the standard deviation (usually 2σ or 3σ), trending analysis charts are usually constructed using threshold limits, alert limits or action limits. When one of those limits is exceeded, an investigation is triggered to determine if further action is required. The initial investigation will help determine the need for a formal CAPA.
There are two types of variation present in every process: common causes, which are sources of variation that are inherent in every process, and special causes, which are sources of variation that arise because of special circumstances. Common cause variation affects every outcome of the process and everyone working in the process, and is referred to as the noise of the process. Common cause variation is generally considered to be the responsibility of management. Special cause variation is not an inherent part of a process. Special causes are also referred to as assignable causes, and they are generally considered to be the responsibility of the workers.
Trend analysis is also a good way to establish how much common cause variation (noise of the process) is present in the process and to determine whether special cause variation (assignable cause) is present. Control charts (trending charts) usually contain a centerline (the mathematical average of the sample or subgroup), upper and lower statistical control limits that define the constraints of common cause variation and performance data plotted over time.
Control charts generally fall into two classifications: variable charts and attributes charts. Variables charts are used for things that can be measured, such as weight, length, temperature, etc. Attributes charts are for things that are counted or rated as pass/fail. Table 1 provides a list of various types of variable and attribute control charts and their applications.
Table 1 Variable and attribute control charts types and applications. (Source: Practical Engineering, Process, and Reliability Statistics, 2014, Milwaukee, ASQ Quality Press).
When a process is in statistical control, meaning only common cause variation is present, 68 percent of the data will fall within ±1σ of the process average, 95 percent of the data will fall within ±2σ of the process average and 99.7 percent of the data will fall within ±3σ of the process average.
I generally advise companies to use ±2σ of the process average for alert limits and ±3σ of the process average for action limits. However, I have seen ±1.5σ and ±2.5σ used successfully.
Managing Outlier Data Points
Occasionally, a process will drift or even go out of control indicating the presence of special cause variation. When the process does go out of control, the data points may be considered outliers. Outliers should be investigated and removed from the data used to make the calculations for the process centerline and for the control and trend limits. Using the special cause variation data can/will affect the control limit and trend limit values or, as I like to call it, the elimination of bonus statistical tolerance.
We need to consider removing outlier data points:
There are several visual and analytical tools that can be used to determine if the suspect points are indeed outliers. Each outlier detection method has certain rules and unique applications. The analytical methods include:
Normal curves, control charts and box plots can also be used to detect outliers visually.
Figure 1 Normal curve with an outlier
The normal curve shown in Figure 1 has an outlier to the far right of the normal curve. Please note the point is above the axis to provide clarity.
Figure 2 Attribute control chart with an outlier
The attribute control chart shown in Figure 2 has an outlier at Point 11.
Figure 3 Box plot with an outlier
The box plot shown in Figure 3 has an outlier at value 76.5.
Another consideration would be how to “normalize” the data. For the example in Figure 4, the average number of complaints per month is approximately seven, with 1,000 units produced per month. If production was increased to 2,000 units per month, it would be reasonable to assume the number of complaints would double. That is why data normalization should be utilized. Some of the normalizing factors that I have seen include units sold, units shipped and number of patients treated.
There is one more consideration for the complaints per month threshold limit example in Figure 4. There may be a lag of several weeks, or even several months, from the time the unit was produced until the complaint was received. The lag should be considered when investigating the cause of the complaint. It should be apparent there is no one perfect way to analyze data trends; however, for a trending program to be successful, consistency is important. Pick a method and stick with it.
The most important reason to use trend analysis is to detect when something has changed beyond the normal noise of the process. In the complaints per month threshold limit example shown in Figure 4, the short green dashed line represents the normal noise of the process. For the month of August, 60 complaints were filed. Because the 60 complaints exceeded the normal noise of the process, a CAPA would be opened in addition to the normal individual complaint investigation to identify the cause of the spike in complaints.
Figure 4 provides an example of complaints per month with threshold limits. The long dashed red line represents the threshold limit, with the outlier data point of 60 complaints included. The short dashed green line represents the threshold limit, with the outlier data point of 60 complaints excluded. The gap between the short dashed green line and the long dashed red line is what I refer to as ‘statistical bonus tolerance.’ It should be evident that excluding the outlier data point provides a control limit which is more representative of the complaints per month, a more conservative approach.
Figure 4 Complaints per month threshold limit example
Although the example provided here was for complaints, the concept can also be applied to nonconformances, internal and external audit findings, service reports, maintenance reports, etc. The use of trend analysis is a great risk-based tool to trigger the CAPA process.
The discussion above focuses on data trending to aid the process of identifying trends. There are other tools and methods available to identify and analyze trends. I want to reinforce that alert limits and action limits used, should be based upon an organization’s risk acceptance determination threshold, industry practice, guidance documents and regulatory requirements.
I cannot emphasize enough the importance of documenting the tools and methods used. Best practice includes providing rationale for your organization’s use of risk tools and activities. The trending tools presented in this article can and should be utilized based upon industry practice, guidance documents and regulatory requirements.
A longer version of this blog post originally ran in Pharmaceutical Online and can be accessed at https://tinyurl.com/y925akva.
Mark Allen Durivage is the managing principal consultant at Quality Systems Compliance LLC and an author of several quality-related books. He earned a BAS in computer-aided machining from Siena Heights University and an master’s degree in quality management from Eastern Michigan University. Durivage is an American Society of Quality (ASQ) Fellow and holds several ASQ certifications, including certified quality manager (CQM/OE) and certified six sigma black belt (CSSBB). He also is a certified tissue bank specialist (CTBS) and holds a global regulatory affairs certification (RAC). Durivage resides in Lambertville, Michigan. Please feel free to email him at email@example.com with any questions or comments.