Thoughts from 2017 PDA Quality Metrics and Quality Culture Conference

Note: The original version of this article was published on my LinkedIn profile on February 23, 2017. Another version was published in the July/August 2017 issue of PDA Letter.

I had the pleasure spending two days networking and discussing quality metrics and quality culture with industry and FDA leadership at PDA's 2017 Quality Metrics and Quality Culture Conference last February. This year's conference included some of the most candid and honest dialog on these topics that I've seen between industry and FDA to data. I thought I would share some of the messages and my takeaways from the two-day event.

Day 1 - Focus on Quality Metrics

The first day focused on quality metrics and specifically the revised draft guidance issued by the FDA in November 2016. Some of the key messages coming from the FDA centered on the goals and benefits of the proposed program as well as their desire for continued feedback from industry. The FDA's metrics program is intended to improve the quality of drugs, accessibility of quality drugs (minimize shortages), and effectiveness of the FDA regulatory oversight. Additionally, the FDA desires to foster a joint culture of dialog with industry focused on what they learn from the data they collect as both sides share the responsibility for providing quality drugs. The Agency knows that dialog will not likely come easily at first but hope with time to see improved communication especially around areas that could impact the quality or availability of drugs. Many ""technical"" aspects of the program were discussed, including:

• An overview of key changes between the first and second drafts including recognition of the value of the feedback FDA received from industry on the first draft.
• Discussion on who will be doing the reporting and comparison of the benefits of site reporting versus product reporting.
• Voluntary Phase: Based on industry feedback, the Agency included a voluntary phase prior to implementation of the mandatory program for drugs and some biologics. Currently, there is no defined endpoint for the voluntary phase.
• Timing: The voluntary phase will be launch using notification through the Federal Register approximately 1-2 months in advance of the metrics portal opening and will include the details on what to submit, how to submit and how long the portal will remain opens. While not mentioned in the guidance, the FDA indicated at the conference that the portal would likely be open for approximately 1-3 months. The FDA anticipates launching the voluntary phase in early 2018.
• Metrics and data to be collected: The draft metrics were simplified from first version and focus on collection of data which will be used to determine Lot Acceptance Rate (LAR), Invalidated Out-of-Specification Rate (IOOSR), and Product Quality Complaint Rate (PQCR).
• How to recognize participation in the voluntary phase including discussion of the pros and cons of the proposed tier based approach.

Several interesting questions were discussed during the first panel session. One of these questions for the panel centered on why quality culture wasn't mentioned in new guidance (in the first draft, it was mentioned nine times). It was clear from listening to the panelists that just because ""quality culture"" is not written into the current draft of the guidance doesn't mean it's not important. The draft guidance document clarified and focused on a program that would be executable by both industry and the agency. They also feel that the proposed metrics can be very predictive of the future and are driven by an organization's quality culture. Industry representatives also presented their experience in analysis of quality metrics from their own internal programs. Some of the learnings included:

• Acknowledgment that the tools to automate data collection and analysis are out there...organizations do not have to invent something in order to perform robust analytics and reduce the burden on the organization.
• Metrics programs are inherently complex namely due to inconsistency in definitions and the variety of sources for the data...the complexity of these programs should not be underestimated.
• Metrics programs should aim to ""Find it once, fix it everywhere"" especially when there are signals of a larger issue or the potential for an issue to pop up elsewhere in the organization by working on impacting the ""drivers of the drivers"".
• Several speakers emphasized the need to not ""sweat the red"" when looking at a dashboard but also not to get too comfortable with the green (a great analogy was provided for this: think of a watermelon, the surface is all green and looks good but you never know what lies beneath...it may be good, it may be bad).
• Know that whatever you decide to measure changes behavior - sometimes for the better but also potentially for the worse.
• Don't forget to understand the context around the metric as this can help you determine if a blip is just a blip or indicative of a more serious issue.
• Don't get overly focused on the tools (e.g. dashboards) but instead focus on making information transparent and flow in an organization.
• Don't expect your metrics to stay the same year after year...after all, a robust quality metrics program requires continual improvement and will evolve.

Many of learnings I’ve personally experienced when implementing metrics programs. It was clear that both the FDA and industry understand that benefits of the program may not be realized immediately. It was repeatedly acknowledged that we are at a beginning of a journey. We have a lot to learn in order to achieve the ultimate goals of improving the quality of drugs, accessibility of quality drugs (minimize shortages), and effectiveness of the FDA regulatory oversight (specifically around inspections and post-approval changes).

Day 2 - Focus on Culture

On the second day, we shifted the focus of our discussions to quality culture starting with presentations from both the FDA and MHRA. From the FDA’s Jeffrey Baker, we heard about culture as both a noun and a verb which begs an interesting question: Are you looking at culture as a thing (something you possess) or a behavior (something you can grow and influence)? And if culture is more of a behavior, then we were reminded of the way adults learn behaviors via the ladder of inference, a theory which discusses how adults are continuously observing and in taking data, then translating that into action. The perspective shared by David Churchward, MHRA expert GMP inspector, indicated that quality culture requires knowledge, diligence, vigilance, management commitment, and transparency. Based on the MHRA’s 2015 inspection experience, many of the serious failures resulted from an absent or over-controlling senior management or a lack of vigilance. In fact, they still find that many of the conclusions from the Clothier Report (MH 149/1794) issued in 1972 still hold true and reminded us that people are still at the center of what we do.

When industry presented their perspectives on quality culture, I must admit I was quite pleasantly surprised with what I heard from industry's senior leaders. There was a significant portion of the time spent on the need to return to W. Edwards Deming's 14 Points for Management (first presented in his book Out of the Crisis and in particular, point number 8 - Drive out fear in order to build and sustain healthy and robust quality cultures.) I couldn't agree more. Fear is a significant motivator in human behavior and when you remove fear in an organization, the path to successfully changing behaviors becomes a bit easier. We also heard about the progress that the PDA team is making on their Quality Culture assessment tool, including perspectives share by one of the participants. The tool provides a structured framework aimed at assessing the quality culture of an organization. In the pilot of the tool, there are approximately 50 sites from 26 firms participating and 64 assessors have been trained. PDA is targeting finishing the pilot this year and rolling out the tool formally soon after that. Machelle Eppler, vice president and head of global quality compliance and regulatory at Patheon, shared her company’s experience in using the tool (five sites from several regions of the world participated). She re-emphasized that the tool provides a consistent language, framework and scoring method as well as a road-map for improvement. She also discussed the need to create the right environment (make people feel safe to be open and honest; there are no right and wrong answers) and to spend the necessary time planning (several weeks not days). They also found senior leadership engagement is the key. In fact, all communications around the process came from plant GMs and not quality. They also felt it important to communicate results and action plans - share successes and implement best practices. After all, she reminded us that improving quality culture is a journey.

The conference ended with one final panel discussion where we were reminded of the goals of the program we had discussed the previous day:

• Improving the quality of drugs
• Decreasing drug shortages by detecting signals earlier
• Decreasing inspection frequency and/or duration
• Increasing dialog between industry and FDA