For Life Science Professionals

Emma Barsky
Len Grunbaum

Quantifying Quality for GxP Compliance
by Emma Barsky and Len Grunbaum, Regulatory Compliance and Operational Consultants



Oct 30, 2012 | Free Downloads | email | Print

Share This Article






Note: The views expressed in this article are those of the author and do not necessarily represent those of his/her employer, GxP Lifeline, its editor or MasterControl, Inc.
Reprinted with permission from www.gxpperspectives.com

GxP professionals understand the need for quality and quality systems and we discuss quality with one another on a daily basis. But how do we measure it? How do we quantify our results? Once again we turn to Len Grunbaum and Emma Barsky, regular contributors to GxP Perspectives, for their insights on how to quantify quality for the development, manufacture, and distribution of health products such as drugs, medical devices, and biologics.

Quantifying Quality

In its simplest form, the definition of "quality" is "how good something is." But what exactly does this mean for the life science industry, whose frame of reference is defined by regulations which are often vague and provide little or no guidance regarding how they should be implemented?

In light of this, we would like to offer some ideas regarding how to measure - quantify - how good your "quality" is in tangible and practical terms. We contend that such metrics are useful in order for company management to make sound decisions regarding whether and/or where the quality system (i.e., the operational infrastructure that promotes and facilitates "quality") requires improvement. The following key indicators are not all-inclusive (nor are the items mutually exclusive), but they provide meaningful ways to assess your "quality":

"In its simplest form, the definition of "quality" is "how good something is." But what exactly does this mean for the life science industry...."
  • Number of successful external and internal audits as a percentage of the total number of external and internal audits: the higher the percentage of successful external audits (e.g., by existing/potential clients, regulators), especially when you have a large number of them, the better your "quality." Passing one audit with flying colors is great but passing multiple audits with few minor or no observations is way better. It not only sets a trend regarding "legitimate" quality but it also validates the company's degree of quality from different perspectives. This scenario allows any company to claim that its quality system has withstood scrutiny from a variety of companies and/or regulatory agencies over a long period of time.

While "looking good" to the outsiders is great, "feeling good" about what is under the covers is even better. Therefore, if thorough internal audits do not find any issues that either directly (critical observation) or indirectly (major observation) impact subject/consumer safety and/or data/product integrity, then "quality" is inherent to the operations.

You may wonder how a subjective term like "success" is defined in this context. Fair question. A result of "no audit findings" (e.g., no FDA 483s, no audit observations) is the clearest measure of success. A relative handful of "minor/cosmetic" issues is not perfect but is certainly acceptable in this context. To the extent that the number of observations may be "critical" or "major," as defined above, the audit will certainly be viewed as less successful or even unsuccessful. One should also remember that it is a common thing in the industry to consider a large number of minor observations as a major issue because this scenario gives an impression of a negative trend, the latter of which is not conducive to having quality operations.

  • Number of "directed" (i.e., "for cause") audits as a percentage of total audits: because directed audits are performed to follow up on actual or perceived regulatory compliance problems, the higher the number of "directed" audits, the more questions will be raised about your "quality." "Directed" audits could be external (i.e., performed by existing clients or regulatory authorities) or internal (i.e., performed by internal quality staff). The higher the number of problems confirmed, the weaker the quality system. Even if these types of audits indicate in general that there are no actual problems, or a minimal number of problems, a large number of such audits should prompt questions regarding why the perception exists that the degree of "quality" is such that an investigation is required.

  • Number of investigations/CAPAs: an investigation is a formal and documented process performed to gather information (e.g., root cause, impact) regarding a specific problem encountered (e.g., a customer complaint, a missing controlled document) and which, depending on the outcome of the investigation, may lead to corrective and preventive actions. An "excessive" number (the definition of which is admittedly subjective in nature) of investigations, even if satisfactorily completed and closed, gives an impression that the underlying cause has never been properly identified and/or corrected.

  • Number of repeating issues as a percentage of the number of audits performed: repeating issues are symptomatic of a quality system that does not correct or otherwise effectively address problems. While isolated incidents are not necessarily a reflection on the company's overall quality, incidents that span multiple project teams and/or departments and/or are observed more than once may be indicative of quality-related problems. It is very difficult to convince anyone of the quality of operations when problems that are systemic in nature become evident.

    The higher the percentage of audits that contain repeating issues, the more likely that this may be viewed as 1) management indifference, 2) lack of management involvement, 3) inappropriateness of personnel qualifications and/or 4) inability/unwillingness to invest in "quality."

  • Number of business opportunities lost due to unsuccessful external audits as a percentage of the number of external audits: audits are sometimes performed as a basis for determining whether a business relationship should be consummated or continued (e.g., you will be chosen as a vendor/supplier, an existing relationship will be sustained) or expanded (e.g., a company will be awarded additional projects). Some life science companies (e.g. pharma, biotech) have to get clearance from the FDA prior to being able to market their product. Support companies (e.g., CROs, contract manufacturers) may have to undergo due diligence inspections to establish/maintain/enhance a business relationship. The higher the percentage of such opportunities lost (e.g., loss of a potential or existing client, project cancelled/not awarded, FDA did not grant an approval) because of poor audit results, as a percentage of external audits performed, the stronger the indication that your "quality" is dangerously weak. This, in turn, has a financial "bottom line" impact on the company: loss of business opportunities can also be translated into wasted R&D cost and/or lost anticipated revenue, both of which become a major risk to the company's financial health.

In addition to the items listed above, there is another important quantifiable component to "quality," which is too often being overlooked or not being considered at all. This component is what we define as "the monetary expenditure associated with 'quality.'" Namely, we are talking about an operationally quantifiable parameter - cost of establishing and maintaining "quality" operations. Most will argue that "quality" is very expensive no matter what. We firmly believe that it does not have to be that way if the underlying causes, which directly and unnecessarily contribute to the extra cost of doing business, are either eliminated or minimized. Here are a few examples to give you a flavor of what can contribute to increased costs when it comes to meeting the regulatory responsibility of instituting and sustaining "quality":

  • Regulatory compliance decisions that are not defined in writing and/or are not defensible.
  • Cumbersome and inflexible procedures that require more resources than necessary to execute them without "procedural deviations."
  • Inefficient procedures that require the same activity to be done more than once in order to be in compliance.
  • Ineffective procedures that do not reach the desired objective of being in compliance after the first execution.
  • Unclear procedures that result in too many on-going corrections in order to inject "quality" into operations.
  • Too many procedures that company staff must follow without any value added.
  • Contradictory procedures that lead to generating Notes-to-file, CAPAs, deviations, investigations, etc. because compliance to one procedure results in non-compliance with one or more other procedures.

The above-listed activities not only translate into the need to spend more time and money in an attempt to have operational quality, but a number of these items translate into further quality-related costs to the company. Examples of the latter include, but are not necessarily limited to, taking the time to respond to observations or even worse yet, an FDA-483 or a Warning Letter. We think the point we are trying to make is clear....

Our bottom line is that you can make both your QA and CFO happy by quantifying "quality" in terms that will be understood and appreciated by both. This means that sound decisions can be made regarding whether and/or where to apply precious company time and resources help ensure that your "quality" is as good as it can be without putting the business out of business.



Emma Barsky has over twenty-three (23) years of broad pharmaceutical experience in various areas of clinical and commercial QA/QC; outsourcing; project management and technical chemistry; and manufacturing and controls (CMC)-related operations. This forms the basis for her vast knowledge of the regulations and regulatory compliance for life science companies.

Since the launching of PSG, Ms. Barsky has leveraged her expertise to 1) conduct due diligence audits, 2) perform multiple operational, quality and computer system validation assessments, 3) develop and implement regulatory compliance and operational strategies, 4) conduct field and fraud investigations, 5) help companies to either build or enhance their quality infrastructure in order to successfully pass customer audits and regulatory authority inspections, 6) host customer and regulatory agency inspections and 7)manage multiple projects that required the assistance of subcontractors.

Len Grunbaum has more than forty (40) years of experience as a business systems analyst, computer systems analyst and programmer, EDP audit and quality assurance professional and consultant. He is a recognized expert in the pharmaceutical industry, particularly in the areas of computerized systems; IT infrastructure; electronic records; regulatory training and the operations that support them; computer system validation; quality assurance; and compliance with 21 CFR part 11. He has written extensively on these subjects, won awards for best article, and has delivered numerous tutorials and training sessions to customers and industry groups. He has also provided compliance enforcement training to the FDA; helped to build quality infrastructures for life science companies and organizations that support them; prepared companies for customer audits and regulatory authority inspections; hosted customer audits and regulatory authority inspections; and was asked to assist with field and fraud investigations.

Mr. Grunbaum was involved in organizational effectiveness; operational design; business process management; and computer systems design/development in the insurance and financial services industries. He designed a computer auditing model for the New Jersey practice of KPMG Peat Marwick. He also managed the computer auditing efforts for a major practice office of Coopers & Lybrand (now PriceWaterhouseCoopers), including all offshore activities in support of a major pharmaceutical customer.

Ms. Barsky (ebarsky@practicalsolutionsnj.com) and Mr. Grunbaum (lgrunbaum@practicalsolutionsnj.com) may be reached at (609)683-0758.


Share This Article





Watch Related Videos


Download Free Resources
White Paper: Contracting Organizations and the Need for Written Transfer Obligations and Quality Agreements