How to finish your audit schedule by December 31st

There are 34 days until the end of 2012. You have four supplier audits and three internal audits to complete. Of course all but two of these audits are overdue. What should you do?

Well if I were Minerva McGonagall, I would let you borrow my time turner and you could just turn back time and perform all seven audits at the same time.

If your calendar is looks more like this one, then you only have 24 days before “the end” and completing those last seven audits might not be so important.

Other options that might be readily available to you include:

1. get some help

2. perform remote audits

3. reschedule some of the audits for next year

There are some great cartoons and jokes about doing more with less, but if you intend to complete seven audits before the end of the year you might need some help. There really isn’t any time left to train someone so that they are capable of conducting an effective audit by themselves. In fact, I expect training a new auditor to take at least 6 months before I think they are ready to work solo. Even if you are less demanding than I am, you still would need time for classroom training and shadowing a couple of audits. Therefore, the best I think you could hope for is one or two solo audits of the seven you need to complete.

Realistically, your only source of help would be auditors that are already trained and consultants. The last month of the year is historically very busy for everyone–especially Quality Assurance Auditors. Therefore, consultants will not be cheap and you should commit to any qualified consultants that are available without too much delay (then again, maybe they are available because they are not very good). If you have any in-house auditors that are already trained, do everything you can to get some of their time in the next few weeks.

Option 2 is to perform remote audits. This is a viable option for you to justify for a supplier with a great quality track record or suppliers in other countries. However, a remote audit is not the same as asking a supplier to complete a survey. ISO 19011:2011 provides some guidance specific to remote auditing in table B.1 of Annex B.

For a remote audit, you should still sample just as many records—if not more. You should conduct interviews by phone, Skype or some similar technology. You should analyze any available data to help identify which processes appear to be effective and which processes need to improve. For anyone that is doing this for the first time, I recommend using an audit checklist that is design for an on-site audit and attempting to complete all the questions remotely. Just remember, “Always request data.”

Option 3 is to reschedule some audits for January 2013. I have suggested this so many times to clients, but very few follow this advice. If your company is late in conducting some audits, the important thing to do is to document this, reschedule the audits, and take corrective action(s) to prevent it from recurrence. If you wait until January, you will have additional time to train an auditor as well. Finally, consultants historically have more time available in January than December.

In parallel with your efforts to catch-up on your schedule, I also recommend the following:

1. Create a quality objective that measures “on-time delivery” of audits and audit reports. This is an effective metric for managing an audit program.
2. Investigate the reasons for audits being overdue. If the occurrence was preventable, then I recommend initiating a CAPA. This will have two affects. First, your 3^rd party auditors will see that you have identified the problem yourself and taken appropriate corrective action(s). If you also discuss this during a Management Review, this information can be used effectively to change the grading of an audit finding to a “minor” or to potentially eliminate the finding altogether. Second, it will ensure that this doesn’t happen again.

If you would like to learn more about scheduling of audits and managing audit programs, please subscribe to this blog and connect with me on LinkedIn. There will be several more blogs on these topics in preparation for a FREE webinar on February 14^th. You can also connect with Brigid Glass on LinkedIn. She will be my co-instructor for three courses in April 2013.

Advertisement

Attention Auditors! – Have you read ISO 19011?

If you have ever taken a lead auditor course, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Management Systems”. In November of last year, this standard was updated and the changes were not superficial.

The background entertainment for this week is one of my favorite modern rock songs, but it never seemed to get much air time. I hope you enjoyed the 90’s.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting both internal and external audits, and how to determine auditor competency. Improvements to the New 2011 Version of the Standard include:

1. Broadening the scope to all management systems
2. Clarifying the relationship between ISO 17021 and ISO 19011
3. Introduction of the remote audit methods
4. Introduction of risk as an auditing concept
5. Confidentiality is a “new” principle
6. Clause 5, Managing an audit program, was reorganized
7. Clause 6, Performing an audit, was reorganized
8. Clause 7, Competence and evaluation of auditors, was reorganized & strengthened
9. Annex B is new and the contents of the help boxes was moved to this Annex
10. Annex A now includes examples of discipline-specific knowledge and skills

One of the most common points of confusion in the lead auditor course is the difference between 1^st, 2^nd and 3^rd party audits. In the previous revision of this Standard, this was just a note at the bottom of page 1 and the top of page two. The note was not very clear either. The new version of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear:

The above table is just an example of the improvements made to ISO 19011, and of course there is little value-add to clarifying a definition. Figure 1 from the new version, “Process flow for the management of an audit program, is a better example of a “value-add”. This vertical flow chart is reminiscent of Figure 1 from ISO 14971:2007. It categorizes the various stages of audit program management into the Plan-Do-Check-Act (PDCA) cycle. I highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard. Unfortunately Figure 2, “Typical audit activities”, does not categorize the stages of audit activities (Clauses 6.2 – 6.7 of the revised Standard) into the PDCA cycle. I guess they needed to leave some improvement for the next revision.

The new version retained the opening meeting checklist that was in the previous revision (Clause 6.4.2), and Clause 6.4.9 has a brief closing meeting checklist. Figure 3, “Overview of the process of collecting and verifying information”, is a poor example of a flow chart. Should I make a better one? (Send me an email if you think I should.)

The most valuable changes in this revision are Clause 5.3.2, “Competence of the person managing the audit program”, and all of Clause 7. Most of the audit procedures I read neglect to define the qualifications and method for determining competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures I read include qualifications for a “Lead Auditor”, but I seldom see anything regarding competency. Unfortunately, this Standard only specifically addresses “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When I teach people how to be a lead auditor, I spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how 3^rd party auditors become qualified as a Lead Auditor. 3^rd party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meeting, conducting the audit, closing meeting, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e. – Stage 2 certification or recertification), and another qualified lead auditor must evaluate you and provide feedback.

The last big additions to this Standard were the Appendices. Annex A provides examples of discipline-specific knowledge and skills of auditors. This section is a little on the boring side. I prefer to tell a story about the internal auditor that was auditing incoming inspection—but they had no idea how to check for calibration or how to measure components. Appendix B, the finale, has a table (Table B.1) that provides some guidance on how to conduct remote audits (i.e. – desktop audits). I was pleased to see that conducting interviews is a major part of remote auditing in this table. Section B.7 provides some suggestions with regard to conducting interviews, but if you exhibit all 13 of the professional behavior traits found in Clause 7.2.2 then you really don’t need any advice on how to speak with people. For the rest of us mortals, we could use a five day course on interviewing alone.

Additional guidelines are available on the ISO website.

How to Train an Auditor on the Process Approach

Country music fans are loyal blog readers too.

            I have been reviewing the trends for how people find my website, and a large number of you appear to be very interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:

1. Design & Development
2. Purchasing
3. Incoming inspection
4. Assembly
5. Final Inspection
6. Packaging
7. Sterilization
8. Customer Service
9. Shipping
10. Management Review
11. CAPA
12. Internal Auditing

There are two reasons why the process approach is recommended. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments the process approach will catch it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked. Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e. – the element approach).

My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit. For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment.

The tool that BSI uses to teach the process approach is the “Turtle Diagram”. The following picture illustrates where the name came from.

Process Auditing – “Turtle Diagram”

The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview of the process, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gages used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list the auditor should select people to interview and follow-up with a request for training records.

The sixth step of the process is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company Quality Objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.

In my brief description of the process approach I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company and it is easy to understand. More importantly, however, the incoming inspection process does a great job of covering more clauses of the Standard than most audits. Therefore, new auditors get a great appreciation for how almost all the clauses can be addressed in one process audit.

If you have questions, or you would like a copy of the turtle diagram I use for documentation of audits, please submit a request on my website contact us page.

The Supplier Audit Agenda

This blog has been moved to the following location and the title has been changed: http://bit.ly/SupplierAuditAgenda.

This blog website and the blogs within it are gradually being transferred over to my new website: http://www.MedicalDeviceAcademy.com. The titles may change, and there may be minor revisions to the content as the blogs are reviewed and edited. There will be a subscription list created for the new blog site. If you would like to be added to the list for the new blog site, please email me directly at: rob@13485cert.com.

Which Suppliers Should You Audit?

This blog has been moved to the following location, and the title has been changed: http://bit.ly/WhichSuppliers.

This blog website and the blogs within it are gradually being transferred over to my new website: http://www.MedicalDeviceAcademy.com. The titles may change, and there may be minor revisions to the content as the blogs are reviewed and edited. There will be a subscription list created for the new blog site. If you would like to be added to the list for the new blog site, please email me directly at: rob@13485cert.com.

How many hours does your company spend auditing?

For this week’s video I picked a song played by Stevie Ray Vaughan and Jeff Healey–Look at Little Sister. They are two of the worlds greatest guitar players ever and both died way too soon.

Just in case you can’t connect…here’s another great Jeff Healy performance.

Each week I audit a different company or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an audit schedule (see my example above).

On the surface, this example seems like a good schedule. There are 12 auditors performing two audits each per year. If each auditor spends a day auditing and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing and each person spends less than 2% of their work year auditing.

Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more”. My example also has another fundamental weakness. The audit schedule does not take full advantage of the process approach. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration is relevant, these clauses should be investigated as part of auditing that area. For example, when the incoming inspection process is audited it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.

If the concept of process auditing is fully implemented, the following clauses can easily be audited in the regular course of reviewing other processes: 4.2.1) Quality System Documentation, 4.2.3) Document Control, 4.2.4) Record Control, 5.3) Quality Policy, 5.4.1) Quality Objectives, 6.2.2) Training, 6.3) Maintenance, 6.4) Work Environment, 7.1) Planning of Product Realization & Risk Management, 7.6) Calibration, 8.2.3) Monitoring & Measurement of Processes, 8.5.2) Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.

Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to better understand how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream, because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering and quickly identify retraining opportunities.

One final aspect of the example audit schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports instead of one. This doubles the amount of time auditors spend in preparation and follow-up activities associated with an audit. Second, doubling the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit, because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.

It is not possible for me to provide a generic audit schedule that will work for every company or even to show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit, and therefore the combined resources equal 24 days (~$10,000) allocated to auditing and each person spends 2.5 % of their work year auditing. I have provided a copy of this improved plan below. My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.