How do you audit for compliance with ISO 14971:2012?

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and you identified a couple of gaps in your procedure. After you revised your Risk Management Procedure to be compliant with the revised Standard, then what are you supposed to do?

For the next few weeks I plan to torture all of you with holiday music. If you don’t like it, buy a satellite radio for Christmas sake.

Most QA Managers struggle over whether they should purchase ISO 14971:2012 or not. I wrote a couple of blog postings about this, but my point was not to debate this question. My point was that companies need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from the 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are 7 aspects of the ISO 14791 Standard that  do not meet the requirements of the MDD. Therefore, if your company has already verified that your Risk Management Process is compliant with the MDD–then you have nothing to change. However, if your Risk Management Process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these 7 aspects.

Once you have made your revisions, how do you audit for compliance with ISO 14971:2012?

Step 1: Planning the Audit

This will be an internal audit and since you (the QA Manager) are the process owner for the Risk Management process, you cannot also audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she is not trained on ISO 14971:2012. To address this gap, she must read the updated Standard to understand what’s new.

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December, because you want any CAPA’s resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important, because the Risk Management Procedure requires that top management assess the effectiveness of the Risk Management Process during Management Review meetings.

There are no previous audit findings to close from the last audit of the Risk Management Process, but the Director of Engineering has 7 specific items to emphasize from the 2012 revision of the Standard and a revised procedure for Risk Management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more generic, open-ended questions.

Specific Questions for 7 Items in ISO 14971:2012, Annex ZA:

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded.)

2. Please provide a few examples of how risks in the lowest category were reduced. (Sections 1 and 2 of the Annex I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling.)

3.  How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination.)

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk.)

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level.)

6. What were your team’s priorities for implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I.)

7. How was effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk.)

The above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still ok to include questions that use the element approach.

Generic Questions:

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the Risk Management Report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated Risk Management Procedure prior to conducting a risk analysis?

These generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by asking open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Conducting the Audit

The next step of the auditing process is to conduct the audit. During the audit, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the Risk Management Process and other processes such as: 1) document control, 2) creating technical documentation for regulatory submissions, and 3) the training process. The specific questions verify that each of the 7 elements identified in Annex ZA of ISO 14971:2012 are adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s) so that everyone is clear what the findings were, and if there were any nonconformities this is the time to clarify what needs to be done in order to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit, but it is critical to have the report completed soon enough so that CAPA’s can be initiated (not necessarily completed) prior to the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure the effectiveness. Then the effectiveness check is objective. For example, monitoring the frequently of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the Risk Management records referenced in the Essential Requirements Checklist indicates if the Risk Management process is  being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a train gap is just an example of a single lapse.

Advertisement

ISO 14971 – Buy the new 2012 version?…comment please

I’m sure that there are some that disagree with my determination that the latest revision of EN 14971, revision 2012, is unnecessary (the European Commission certainly does).

 You will have to go to my website to read my cheeky posting on this topic.

And here’s another cheeky attitude from the UK…(sorry, this is not a family channel).

Therefore, I would like to clarify why I feel this way by reviewing how risk is addressed in the MDD (93/42/EEC as modified by 2007/47/EC).

1. The term risk is mentioned only 4 times in the Articles in the MDD
2. The term risk is mentioned once in Annex II and III, twice in Annex VII, and three times in Annex VIII and X—for a total of 10 times.
3. The other 41 times risk is mentioned are in the Essential Requirements (i.e. – Annex I).

When companies submit a Design Dossier for review by a Notified Body, an Essential Requirements Checklist is included. This references, in table format, how all the requirements of Annex I are being met—including those related to risks. Throughout Annex I, a similar phrase is repeated many times. For example, in the first Essential Requirement (ER1) it states: “…any risks which may be associated with [a device’s] intended use [shall] constitute acceptable risks when weighed against the benefits to the patient and are compatible with a high level of protection of health and safety.” In ER2 it states: “the manufacturer must…eliminate or reduce risks as far as possible…”. There is no room in the MDD for consideration of cost or economic impact when the manufacturer is designing a device with regard to risks and benefits.

If a company’s Risk Management Procedure has been found to be acceptable by a Notified Body, and the company has addressed all the Essential Requirements (ERs) with regard to risk, then there should be no impact from these 7 deviations identified in EN 14971:2012. However, if your company has not addressed each of these ERs, then you might want to consider each of these areas:

1. Treatment of negligible risks
2. Discretionary power of the manufacturer as to the acceptability of risks
3. Risk reduction “as low as possible” (ALAP) verses “as low as reasonably possible” (ALARP)
4. Discretion as to whether as risk benefit analysis needs to take place
5. Discretion as to the risk control option/measures
6. Deviation as to the first risk control method
7. Information of the users influencing the residual risk

My final advice is to review Annex I and Annex X from the perspective of risk management. You may realize that you have some gaps that nobody noticed. After all, audits are just a sample.

PS – I think it’s ironic that the origins of the ALARP principle are UK case law (see link above).

How do you audit design controls using the process approach?

A new connection I made on LinkedIn joined the RA Review Group, and they suggested that anything related to the topic of Design and Development would be of interest for a blog topic. Therefore, I thought I would share a secret with everyone reading my blog…

This blog has been moved to the following location and the name has been changed: http://bit.ly/AuditDesign.

This blog website and the blogs within it are gradually being transferred over to my new website: http://www.MedicalDeviceAcademy.com. The titles may change, and there may be minor revisions to the content as the blogs are reviewed and edited. There will be a subscription list created for the new blog site. If you would like to be added to the list for the new blog site, please email me directly at: rob@13485cert.com.

I have left the links to the videos I love.

Entertainment for this week is Diana Krall‘s recording of Bésame Mucho. I have also included another recording by Andrea Bocelli with English and Spanish subtitles for anyone that wondered what the words meant. The song loses a little of it’s appeal in translation, but English is not one of the Romance Languages.

WANTED: Design Team Needs Über-Leader

“Mona Superwoman” by Teddy Royannez (France)

Last November Eucomed published a position paper titled, “A new EU regulatory framework for medical devices: Six steps guaranteeing rapid access to safe medical technology while safeguarding innovation.” While I have serious doubts that any government will ever be able to “guarantee” anything other than its own continued existence, I have an idea of how industry can help.

The position paper identified six steps. Each of these steps has a comparable action that could be taken in every medical device company. My list of six steps is:

Only the best leaders

1. Only one approach to design controls
2. Stronger internal procedures
3. Cross-pollination by independent reviewers
4. Clear communication of project status to management
5. Better project management skills

The most critical element to success is developing stronger design team leaders. Design teams are cross-functional teams that must comply with complex international regulations, while simultaneously the team must be creative and develop new products. This type of team is the most challenging type to manage. In order to be successful, design team leaders must be “Über-Leaders.”

The most critical skills are not technical skills, but team leadership skills. The role of a design team leader is to make sure that everyone is contributing without tromping on smaller personalities in the group. Unfortunately, there are more men in this role than women.

Why is this unfortunate? Because men suck at listening (takes one to know one).

We need a leader that will be strong but we also need someone that is in touch with the feelings of others and will use that skill to bring out the best of everyone on the team. This superwoman also needs to earn the respect of the male egos around the table. She needs to be an expert in ISO 14971, ISO 13485, Design Controls, Project Management, and managing meetings. Our beautiful heroine must also be a teacher, because some of our team members will not know everything—even if they pretend to.

The Über-Leader will always remind the team that Safety & Efficacy are paramount. As team leaders we must take the “high road” and do what’s right—even when it delays a project or fails to meet our boss’ unrealistic timetable. Superwoman must demand proof in the form of verification and validation data. It is never acceptable to go with an opinion.

She will remind us that compromise is the enemy, and we must be more creative to solve problems without taking shortcuts that jeopardize safety and efficacy. She will work harder on the project than anyone else on the team. She will keep us on schedule. She will whisper to get our attention, but she won’t be afraid to yell and kick our ass.

As Jim Croce says, “You don’t tug on Superman’s cape.” Superwoman is the only exception to this rule.

How do you control design changes?

Of JB’s recommended artists, the Josh Abbott Band was probably my favorite. I especially liked this one. I hope every man is lucky enough to know a girl like Texas. I’m lucky enough to have married a girl that grew up in Texas. They are something special.

We have been discussing the best ways to control design changes at work, and I thought it might present an opportunity to have more of an interactive discussion with my readers.

During my rounds as a 3^rd party auditor, I have seen quite a few design control procedures. The most complex consisted of 19 procedures (NOT recommended, but there were no nonconformities). The most simple consisted of one 4-page procedure, which I wrote, but I would never recommend being this brief. I have created a couple of polls in my LinkedIn profile for you to respond to if you would like to share your own company’s “design control stats”:

http://linkd.in/IJtoBL

The problem I see is that most projects are not new product designs. Sometimes the projects are not even major design changes. I think most changes involve supplier changes, component specification improvements, and design for manufacturability. These changes require review and approval of changes. These changes must also be recorded and retained as a Quality Record.

My own personal preference is to always open a design project—no matter how small the change is. In order to make the process flexible, I also prefer to define how many design reviews each project will have in the design plan rather than mandating that design reviews be held in a stage-gate fashion for 100% of projects.

Most companies will have a table of requirements with columns added to indicate if the requirements are mandatory for the project or optional. For example, “risk management plan needs to be updated? Yes/No.” I like this approach, because the table of requirements makes the decision making systematic.

Sometimes a change is only to a work instruction for a step in the manufacturing process. In these cases, some companies will use a document change order process to supplement the engineering change order process.

My feeling is that more complex products (i.e. – Class IIb & Class III in EU and Class III/PMA in US) will require more stringent design controls for the change. What does your company do to control design changes?

Best in Class Validation Program

This is one of the early music videos I remember from the 80’s.

The original question from a former client was: “What does a best in class CNC machining process validation program look like?” Although I intend to answer this question, I know a few other clients that have done a great job of this. Hopefully they will add their own opinions as a comment. Therefore, I am expanding the scope of this question to validation in general.

The problem with validation is that you can always do a more thorough validation. Only in the cases of processes such as sterilization, do we have ISO Standards that tell us what is required. Otherwise, we are normally the experts and we have to use our own judgment as to what is necessary. In general, the best approach is a risk-based approach.

For each design specification established for a component, we also need to identify what process risks are associated with failure to meet the specification. Most companies perform a process failure modes and effects analysis (pFMEA). This risk analysis has three quantitative components: 1) severity of the failure’s effect, 2) probability of occurrence, and 3) detectability.          The first factor, severity, is based upon the intended use of the device and how that component failure impacts that use. Usually it is important to have a medical professional involved in this portion of the estimation.

The second factor, probability, is typically quantified during the process validation activities. One company I audited developed a ranking scale for probability that was linked directly to CpK of the process. Higher CpK values received lower scores, because the process was less likely to result in an out-of-specification component. Another company I worked for used a six-point logarithmic scale (i.e. – 10e-6 = 1, 10e-5 = 2, 10e-4 = 3, 10e-3 = 4, 10e-2 = 5, and 10e-1 = 6). This logarithmic scale was based on sterilization validation where a sterility assurance level of 10e-6 is considered “validated”.

The third factor, detectability, is best estimated by using a quantitative scale that is based upon a gauge R&R study or some other method of inspection method validation.

Most companies struggle with determination of what is acceptable for design risk analysis. However, for process risk analysis it is usually much easier to quantify the acceptable risk level.

Once you have determined that a process is not acceptable at the current residual risk level, then you must take corrective actions to reduce the risk. The first step to achieve this should be to review the process flow. There are critical control points that can be identified in the process flow. One of those places is at the end of the process at the inspection step in the process.

The inspection step in the process flow affects detectability of defects. For many automated processes, such as CNC machining, it is not reasonable to perform 100% inspection. Therefore, these processes require validation. Most engineers make the mistake of trying to validate every dimension that is machined. However, only some of the dimensions result in device failures. These are the dimensions that are critical to validate. Best practice is to calculate the process capability for meeting each of these critical specifications (i.e. – CpK). A minimum threshold should be established for the CpK (refer back to the process risk analysis for ideas on linking CpK to risk acceptance). Any CpK values below the threshold require a more consistent process. These are the component specifications that should be the focus of process validation efforts.

During a process validation, it is often advisable to perform a design of experiment (DOE) in order to quantify the affects of each process variable. Typically a DOE will evaluate the impact on CpK for each variable at a high, low and middle value while other variables are maintained at nominal values. Any variables that appear to have a significant impact on the CpK are candidates for performing an operational qualification (OQ). For a machining process, this could include spindle speeds, feed rates, and material hardness. If variation of the variable has little or no impact upon the CpK, then there is probably little benefit to inclusion of this variable in an OQ.

The output of an OQ validation should be high and low limits for each process variable that will result in a “good” part. Performance Qualification (PQ) validation is the final step of the process validation. In the PQ, most companies will conduct three repeat lots at nominal values for the variables. If the OQ is designed well, there is often little added value in the PQ. Therefore, the sample size is typically three lots of 10 samples each. If the OQ validation does not clearly identify safe operating limits for the variables, or the process has marginal capability (i.e. – a low CpK), then the OQ should be repeated and an additional DOE may be needed.

Here are a few resources for those of you that are in “Deviceland” and may not be aware of guidance on validation in other related industries:

1. Guidelines for the Validation of Chemical Methods for the FDA Foods Program (3/22/2012) – http://www.fda.gov/downloads/ScienceResearch/FieldScience/UCM298730.pdf
2. 2.      Process Validation: General Principles and Practices (January 2011) –  http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM070336.pdf?utm_campaign=Google2&utm_source=fdaSearch&utm_medium=website&utm_term=Process%20Validation:%20General%20Principles%20and%20Practices&utm_content=1
3. Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods (9/8/2011) –  http://www.fda.gov/downloads/ScienceResearch/FieldScience/UCM273418.pdf
4. 4.      CPG Sec. 490.100 Process Validation Requirements for Drug Products and Active Pharmaceutical Ingredients Subject to Pre-Market Approval (3/12/2004) –  http://www.fda.gov/ICECI/ComplianceManuals/CompliancePolicyGuidanceManual/ucm074411.htm?utm_campaign=Google2&utm_source=fdaSearch&utm_medium=website&utm_term=validation&utm_content=3
5. Q 2 (R1) Validation of analytical procedures: text and methodology (June 1995) – http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000431.jsp&mid=WC0b01ac0580029593&jsenabled=true

Never Stop Learning

One of my family’s favorite songs is “Come on Get Higher” by Matt Nathanson. Two years ago I tried to purchase this for my wife as a Christmas present. Unfortunately, I couldn’t remember who sang the song. I tried searching the web for the lyrics and found out that Sugarland sings it. I remembered the logo on the album cover, went to the store and bought the album. After I got home I realized that the song wasn’t on the album. Back to the store I went and found another version of the album with some live versions of songs—including “Come on Get Higher.” Just to make sure I had the right song, I decided to open the package and play it. My music video selection for this blog is what I heard. I guess we never stop learning, but I did fall in love with Country music at the age of 38…

I am in Canada, it’s almost midnight, and this client has me thinking so hard that I can’t sleep. I am here to teach the company’s Canadian facility about ISO 14971:2007—the ISO Standard for Risk Management of medical devices.

                Most of the companies that request this training are doing so for one of two reasons: 1) several of their design engineers know almost nothing about risk management, or 2) they have several design engineers that are quite knowledgeable with regard to risk management but these engineers have not maintained their credentials and their last risk management training was to the 2000 version of the Standard. This company falls into the second category.

                I always tell students that I learn something by teaching each course. From this company, however, I have learned so much. This company has forced me to re-read the Standard a number of times and reflect on the nuances of almost every single phrase. I have learned more about this Standard in one month than I learned in the 3.5 years since I first took the course I am now teaching.

                I have developed a model for learning that explains this phenomenon. I call this model the “Learning Pyramid.” At the base of the pyramid there are “Newbies.”

               This is the first of four levels. At the base, students read policies and procedures with the hope of understanding.

                In the second level of the pyramid, the student is now asked to watch someone else demonstrate proper procedures. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” This is what our children call “sharing time,” but everyone over 40 remembers this as “show and tell.”

                In the third level of the pyramid, the student is now asked to perform the tasks they are learning. This is described as “doing,” but in my auditing courses I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next trainees will shadow the trainer during an audit as a demonstration of proper technique (level 2). During subsequent audits, the trainees will audit and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen and wait for what I call the “Teachable Moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a difficult subject.

                Finally, in the fourth level of the Learning Pyramid we now allow the trainee to become a trainer. This is where I am at—so I thought. I am an instructor, but I am still learning. I am learning what I don’t know.

                The next step in the learning process is to return to the first level. I am re-reading the Standard and procedures until I really understand the nuances that I was unaware of. Then I will search for examples in the real world that demonstrate these complex concepts I am learning. After searching for examples, I will test my knowledge by attempting to apply the newly acquired knowledge to a 510(k) or CE Marking project for a medical device client. Finally, I will be prepared to teach again.

                This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve, but never reach our goal of perfection…For further inspiration try reading “Toyota Under Fire.”

The Perfect Pecan Pie – Take a “SWAG”

Here’s a riddle…

                How do you know when you’re getting old?

                When country starts sounding really gooood.

Despite what you might think, my intent is not to make fun of country. Everyone’s tastes change over time. At a young age I fell in love with the sticky sweet sound of jazz singers, but in recent years I have started to warm up to the sound of modern country singers. Why?

                In order to properly appreciate a story about love lost and disappointment you need to have experienced life’s little pimples. I think life has finally beat me up enough times that I can personally relate to the deeper melancholy lyrics of country music. Another reason for my greater appreciation is that modern country is a cousin to jazz—kind of bluesy. I think you can see how I’ve grown in this week’s music video selection. My choice is the theme song from the movie Crazy Heart, “The Weary Kind”, written and performed by Ryan Bingham.

My series on the subject of Risk Management training continues (see my most recent blogs on the same topic). In my Risk Management Training, I use the example of making “The Perfect Pecan Pie” as a practical example of applying the principles of Risk Management.

                One of the most important steps of the Risk Management process is hazard identification. I described the process I use for hazard identification in my earlier blog on Risk Management planning. Section 4 of the 14971:2007 Standard defines the requirements for Risk Analysis. Hazard identification is only the first step in Risk Analysis. If done correctly, you should be able to identify hundreds of things that can go wrong with your pecan pie (i.e. – hazard identification). The next step in Risk Analysis is prioritizing these hazards. Prioritizing hazards should focus on the “severity of effect” first. I prefer to use a 5-point scale of even numbers (2, 4, 6, 8 & 10). The reason for this is that I like to create a risk matrix that is 5×5 and I want to emphasize severity over probability of occurrence—the two factors that make up risk. I learned this strategy from an auditing client (Thank you for sharingJ).

                Any potential pit fall that could prevent on-time delivery of the perfect pecan pie should be identified in your Risk Management File, but not every potential hazard requires risk controls. Once I have identified the potential hazards, I estimate the probability of occurrence next. Probability is estimated on a 5-point scale also (1, 2, 3, 4 & 5). The product of the two estimates is the estimated risk. I like to set a threshold for risk controls at 10. Therefore, any hazard that deserves a 10 for severity of effect will automatically require implementation of risk controls. For each product, companies should establish their own criteria for risk acceptability (i.e. – the Risk Management Policy). The potential benefit of the product should also impact this policy. High risk products should have great benefits too.

                Most people struggle with estimating these two numbers. Don’t worry! Take a “SWAG” (scientific wild-ass guess). What matters is that the risk analysis is reviewed and updated. Companies seldom get the risk analysis right the first time so it is critical to review post-production data and update the risk analysis based upon this data. If people tell you that your pie is too sweet, try to estimate what percentage feel that it is too sweet and what percentage feel it is just fine.

                For example, I used to think that a toasted flavor was ok. Most of my family likes this flavor, but the rest of the world seems to hate it. Once you figure this out, you need to change your risk controls to make sure the pie doesn’t burn—even a little. You might try decreasing the temperature or increasing the monitoring frequency. Either way you will decrease the potential frequency of burning a pie.

                You have now completed Section 4 of the 14971:2007 Standard. Please remember there are eight slices to every pie:)

The Perfect Pecan Pie – My Risk Management Plan

I hope everyone enjoys this selection for the music video. It’s one of my favorite songs by Sting—mostly because it seems to be more upbeat than many of his other tunes. I can almost hear him smiling as he wrote the lyrics.

                This blog posting is a continuation of my previous post on the subject of Risk Management Training, specifically the ISO 14791:2007 Risk Management Standard. In my Risk Management Training, I use the example of making “The Perfect Pecan Pie” as a practical example of applying the principles of Risk Management.

                The first step of the Risk Management Process, or any process, should be planning. My personal preference for planning Risk Management is to begin by brainstorming in order to create a list of potential Quality issues. During the brainstorming session, I will use a Cause & Effect Diagram (a.k.a – “Fishbone Diagram”) to ensure that I have covered as many of the important issues as possible. For those that are unfamiliar with this tool, there are six categories of causes for any problem. These are sometimes referred to as the six “M’s”, because each category begins with the letter “M”:

1)      Materials

2)      Method

3)      Machine

4)      Measurement

5)      Manpower

6)      Mother Nature

                Materials are the single most important component of any product. As the saying goes, “Garbage in equals garbage out.” The right, fresh ingredients are just as important to baking pies as biocompatible materials are to manufacturing implantable medical devices. For example, stale pecans are plain nasty; while granular sugar produces a sickeningly, sweet syrup. Pecan pies are derived from “chess” pies—pies that were cheese-like due to the custard consistency created by cooking butter, eggs, milk and sugar at a low temperature. Therefore, the filling of a pecan pie requires a six tablespoons of unsalted butter, three large cage-free eggs (size matters—don’t get extra large), one cup dark brown sugar (light brown is also too sweet), three-quarters cup light corn syrup, one tablespoon natural vanilla (artificial vanilla tastes totally different and overwhelms the praline flavor), and one cup of pecan halves (I’m told that Georgia pecans picked fresh from the tree are amazing, and roasting them enhances the flavor even more.).

                Manufacturing processes are always the second most important factor related to Quality. For the “Perfect Pecan Pie,” this is also true. Most people will try making their first pie by cooking the filling and the pie crust together. This can produce acceptable results if you are extremely lucky. For custards, however, it is much easier to get consistently beautiful pies by pre-cooking the pie-shell (and sealing it with egg yolk) and pre-cooking the filling separately in a double boiler (always use the right machine for the job). Once the filling gets to the desired temperature (~130F) then the filling should be poured into the pre-cooked crust for the final baking. The final baking should be at 275F for one hour (at sea level).

                If you choose to deviate from any of the above directions regarding the manufacturing process, good luck finding a material review board to approve the release of your pie. If you don’t seal the pie crust, it will leak and you will never get be able to serve an intact slice of pie. If you don’t use a double boiler, you get a mixture of caramel and burnt candy. If you overcook the filling, the consistency will be off. If you undercook the filling, the pie will be uncooked…another way to make it impossible to serve an intact slide of pie.

                When you are cooking a soup, stew or some other dish, measuring is a forgiving process. For baking, the ratio of ingredients, the degree of mixing, and the temperature for baking are critical. Any deviation usually leads to a disaster.

                The next category, manpower, addresses the issue of training. You would think that baking is all about skill. However, like all validated manufacturing processes, proper use of process controls can transform the most inept person into a brilliant baker. Most people struggle with the crust. Packard Consulting, however, has developed a fool-proof method for making a crust. The key is to cool the dough ball and press it into a glass pie dish. The reason for a glass dish is so that you can hold the uncooked shell up to the light to inspect it for “thin spots.” Then you cover the shell with foil, poke it several times with a fork to allow it to vent, bake it for 15 minutes at 400F, uncover it, brush it with egg yolk to seal the crust, and continue baking it for 10 more minutes—or until the crust is a golden brown on the edges.

                Finally, the oven temperature is most critical for the final baking—after pouring the pre-cooked filling into the pre-baked pie crust. In this case, we have an artificial environment (i.e. – Mother Nature). Unfortunately, very few ovens are calibrated accurately and the temperature is very inconsistent throughout the oven. Ovens are hottest on the top rack and the back of the oven is always hotter than the front. Therefore, you need to rotate the pie during the baking process or it gets cooked unevenly. Another critical step is to “map” the oven temperature. You must determine where in the oven (i.e. – which rack position) to place the pie when the oven is set at 275F. In some ovens, the temperature is so far off that it is necessary to raise or lower the setting by 15 degrees.

                Now that I have given you the recipe for the “Perfect Pecan Pie,” you might be tempted to make one. Before you do, I recommend getting a piece of paper and documenting every step you take—including any visual observations, the taste of the dough, and the taste of the filling. This information will become your risk management file. As you perfect your technique, learn the idiosyncrasies of your kitchen appliances, and you find sources for each ingredient…you will need to prevent these secrets from becoming lost. Your collection of notes is a Risk Management File.

                You have now completed Section 3 of the 14971:2007 Standard. Keep drooling and I promise to serve up another slice:)

 PS – Here’s a cool drumming lesson that gave me a much better appreciation for the layers of rhythm within the song I chose for this blog’s background entertainment.

The Perfect Pecan Pie – Recipe # 14,971

Hats off to Woodson and the rest of the Packers! My team was the Patriots, but I’m happy to see that the Packers showed some real heart and overcame their injuries to win the trophy. As for the half time show…I thought it was more lights than music. I included a link to the half-time show at the end of this blog, but I thought the Black Eyed Peas video called “The Time (Dirty Bit)” is much more entertaining.

                For those of you familiar with the ISO 14791:2007 Risk Management Standard, you may have already figured out the topic of this blog. For the rest of you…did you really think I tried perfecting my recipe for pecan pie nearly 15,000 times?

                A couple of years ago, a client of mine asked me to give them a training course on Risk Management—specifically an overview of ISO 14971:2007. In my struggle to find a fresh way of engaging the interest of my client’s employees, I developed the concept of using the principles of Risk Management in a more tasty application. Back in 2006 I developed a five-minute presentation on how to make “The Perfect Pecan Pie.” For this new three-hour presentation, I tortured my students with a homemade pecan pie that I placed in the middle of the conference room table.

                This presentation included several tools to help my students remember the principles of Risk Management. First, the alliteration of the letter “P” throughout the presentation beat my message steadily into their subconscious. My second weapon was the smell of a warm, fresh, pecan pie. Third, I used analogies to the pecan pie making process for each aspect of the Risk Management Process. Fourth, I used vivid descriptions throughout my presentation to help everyone visualize the sweet, praline confection at each step of the baking process. And finally, I burned the experience into their brains forever with the taste of the Perfect Pecan Pie.

                Not everyone loves Pecan Pie as much as I do, and not everyone has tried making the Perfect Pecan Pie as many times as I have. I have made gooey pies, sickeningly sweet pies, pies that crack and crumble, pies that were barely cooked, pies without a crust, and pies without a filling. If you plan to coordinate your entire presentation around the concept of a homemade pecan pie you made in your kitchen last night, you had better have a proven Risk Management process to reduce the risk of embarrassing yourself.

                Please read my next several blogs as I unveil the secret to making the Perfect Pecan Pie—one slice at a time:)

                Here’s some of the half time show…(assuming the link is not taken down).