A new connection I made on LinkedIn joined the RA Review Group, and they suggested that anything related to the topic of Design and Development would be of interest for a blog topic. Therefore, I thought I would share a secret with everyone reading my blog…
This blog has been moved to the following location and the name has been changed: http://bit.ly/AuditDesign.
This blog website and the blogs within it are gradually being transferred over to my new website: http://www.MedicalDeviceAcademy.com. The titles may change, and there may be minor revisions to the content as the blogs are reviewed and edited. There will be a subscription list created for the new blog site. If you would like to be added to the list for the new blog site, please email me directly at: rob@13485cert.com.
I have left the links to the videos I love.
Entertainment for this week is Diana Krall‘s recording of Bésame Mucho. I have also included another recording by Andrea Bocelli with English and Spanish subtitles for anyone that wondered what the words meant. The song loses a little of it’s appeal in translation, but English is not one of the Romance Languages.
I have been reviewing the trends for how people find my website, and a large number of you appear to be very interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.
First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:
There are two reasons why the process approach is recommended. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments the process approach will catch it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked. Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e. – the element approach).
My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit. For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment.
The tool that BSI uses to teach the process approach is the “Turtle Diagram”. The following picture illustrates where the name came from.
Process Auditing – “Turtle Diagram”
The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.
After getting a general overview of the process, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.
The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gages used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list the auditor should select people to interview and follow-up with a request for training records.
The sixth step of the process is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.
The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company Quality Objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.
In my brief description of the process approach I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company and it is easy to understand. More importantly, however, the incoming inspection process does a great job of covering more clauses of the Standard than most audits. Therefore, new auditors get a great appreciation for how almost all the clauses can be addressed in one process audit.
If you have questions, or you would like a copy of the turtle diagram I use for documentation of audits, please submit a request on my website contact us page.
I thought I would expand my usual musical range to include some of the music I grew up with. I hope you enjoy this fantastic performance.
I hear this question, or a question with similar wording, quite frequently when I am auditing. Typically the question is in response to a better way to do something that seems simple and efficient. Most people seem to approach regulatory requirements with the approach of…let’s bury the regulator. While it’s true that we expect a certain amount of paperwork with each regulatory requirement, we frequently are accepting of a much broader range in stack heights. For example, a design controls procedure could be a one page flow-chart that references forms and work instructions. A design controls procedure could also be twelve separate documents with a minimum length of ten pages and a maximum of forty pages per document. As long as the procedure has sufficient detail for the people performing these tasks and all the required elements are included, ISO clauses 7.3.1-7.3.7, then we have no choice but to identify the procedure as conforming.
The above example is the perspective of an auditor looking for CONFORMITY!
However, some people are inspectors that are looking for NONCONFORMITY!
In the case of inspectors, it is critical to present your information in such a way that it is easy for the inspector to see how you meet the requirements of the regulations. One of the best ways to do that is to reference the requirements directly in your procedures.
For those that prefer finesse…try to organize information in accordance with the regulations. For example, if I am writing a procedure for an ISO registration audit, I write the procedure to specifically address the ISO sub-clauses. I might even use a document control number like: SOP-73 for my “Design and Development” procedure. Alternatively, if I’m writing a procedure for a JPAL audit, I might change my document control number to SOP-3036 for my “Design and Development” procedure. This matches up with JPAL Ministerial Ordinance #169, Articles 30 through 36. In this case, the document control number suggests compliance with the Japanese regulations. A little subconscious suggestion couldn’t hurt.
In my previous blog posting, I suggested a slight change to the scheduling of internal audits. In order to make sure this meets FDA requirements, the key is to READ THE REGULATIONS AGAIN. With regard to internal auditing, the applicable FDA regulation is: 21 CFR 820.22:
“Each manufacturer shall establish procedures for quality audits and conduct such audits to assure that the quality system is in compliance with the established quality system requirements and to determine the effectiveness of the quality system. Quality audits shall be conducted by individuals who do not have direct responsibility for the matters being audited. Corrective action (s), including a reaudit of deficient matters, shall be taken when necessary. A report of the results of each quality audit, and reaudit(s) where taken, shall be made and such reports shall be reviewed by management having responsibility for the matters audited. The dates and results of quality audits and reaudits shall be documented.”
The above requirement is quite vague with regard to how many auditors and how many days must be spent auditing. These are the variables I suggested changing in my previous posting. The FDA regulations are specific, however, with regard to documenting the “reaudit” of any deficiencies found during an audit. This prescriptive requirement can be met by reviewing previous audit findings of all audits with the audit program manager during the audit preparation process. The audit program manager can facilitate the assignment of which auditor will reaudit each finding. This may require a few more minutes of audit preparation, but this should not measurably impact the overall time allocated to an audit.
Somehow the above prescriptive requirement slipped my mind. I do this out of habit when I am performing internal audits on behalf of clients, but if I am auditing the internal audit process of a client—now I’ll remember to point out this additional requirement that is specific to the FDA and not included in the ISO Standard. This is why we should always READ THE REGULATIONS AGAIN.
For this week’s video I picked a song played by Stevie Ray Vaughan and Jeff Healey–Look at Little Sister. They are two of the worlds greatest guitar players ever and both died way too soon.
Just in case you can’t connect…here’s another great Jeff Healy performance.
Each week I audit a different company or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an audit schedule (see my example above).
On the surface, this example seems like a good schedule. There are 12 auditors performing two audits each per year. If each auditor spends a day auditing and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing and each person spends less than 2% of their work year auditing.
Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more”. My example also has another fundamental weakness. The audit schedule does not take full advantage of the process approach. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration is relevant, these clauses should be investigated as part of auditing that area. For example, when the incoming inspection process is audited it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.
If the concept of process auditing is fully implemented, the following clauses can easily be audited in the regular course of reviewing other processes: 4.2.1) Quality System Documentation, 4.2.3) Document Control, 4.2.4) Record Control, 5.3) Quality Policy, 5.4.1) Quality Objectives, 6.2.2) Training, 6.3) Maintenance, 6.4) Work Environment, 7.1) Planning of Product Realization & Risk Management, 7.6) Calibration, 8.2.3) Monitoring & Measurement of Processes, 8.5.2) Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.
Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to better understand how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream, because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering and quickly identify retraining opportunities.
One final aspect of the example audit schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports instead of one. This doubles the amount of time auditors spend in preparation and follow-up activities associated with an audit. Second, doubling the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit, because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.
It is not possible for me to provide a generic audit schedule that will work for every company or even to show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit, and therefore the combined resources equal 24 days (~$10,000) allocated to auditing and each person spends 2.5 % of their work year auditing. I have provided a copy of this improved plan below. My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.
About 10 years ago my CD collection was stolen and I haven’t heard this tune since. Sass Jordan might be a little raw for your average professional but everyone needs to loosen up sometimes. Just-in-case you were wondering, I think this CD (Rat) was next to the Greatest Hits of Ella Fitzgerald—which they left behind. I love the singing by both women but for very different reasons.
Recently a client asked me to create a training course on how to train operators. I could have taught the operators myself, but there were so many people that needed training that we felt it would be more cost effective to train the trainers.
Usually I have multiple presentations archived that I can draw upon, but this time I had nothing. I had never trained engineers on how to be trainers before—at least not formally. I thought about what kinds of problems other Quality Managers have had in training internal auditors and how I have helped the auditors improve. The one theme I recognized was that most auditors needed feedback.
I finally decided to use the Deming Cycle (Plan-Do-Check-Act, or PDCA) as my framework for the training. Most QA Managers are very experienced and have little trouble planning an audit schedule. The next step is to do the auditing. The problem is that there is very little objective oversight of the auditing process. The Standard requires that “Auditors shall not audit their own work.” Therefore, most companies will opt for one of two solutions for auditing the internal audit process: 1) hire a consultant, or 2) ask the Director of Regulatory Affairs to audit the internal auditing process.
Both of the above strategies meet the requirements of the Standard, but neither strategy helps to make internal auditors better. I have interviewed a lot of audit program managers, maybe 50+, and the most common feedback for auditors is “change the wording of this finding” or “you forgot to close this previous finding.” This type of feedback is related to the report writing phase of the audit process. I rarely hear program managers explain how they help auditors improve at the other parts of the process.
When auditors are first being trained we typically will provide examples of best practices for audit preparation, checklists, interviewing techniques AND reports. After the auditors have been “shadowed” by the program manager for an arbitrary three times the auditors are now miraculously “trained.” Let’s see if I can draw an analogy that will make my point…
That kind of sounds like watching your 16 year-old drive the family car three times and then giving them a license. I guess that’s why my new Ford Festiva was severely dented on all four sides within 6 months. You might think my father was a Saint, but I think he might have totaled his tenth car by age 18. At least I contained the damage to one vehicle.
Anyway the key to training auditors to audit, or anyone on anything, is consistent follow-up over a long period of time.
The question is…was my training successful?
Well, how much follow-up training of the trainers did the client ask for?
I just wanted to thank everyone for reading my blog. It's hard to make the dry and boring worth reading about. I have also created a formal Thank You page on my personal website: http://13485cert.com/qc-is-dead/.
Blog Stats
113,240 Views All Time!
Countdown to 2-Day Course!
Best Practices in Audit Program ManagementMarch 9, 2013
13485cert
QC is Dead 