13485cert

Posts Tagged ‘Internal audit’

Learn the Process Approach & How to Use Turtle Diagrams

In Internal Auditing, Supplier Audit, Supplier Audits on January 21, 2013 at 11:13 am

Example of Turtle Diagram provided by Jan Roovers

Example of Turtle Diagram provided by Jan Roovers

Checklists are great for making sure that all aspects of the regulations are covered, but is there a way to get more out of your audits? Imagine how nice it would feel to eliminate that “Control of Records” audit, and several other audits from your schedule that consist primarily of reviewing mountains of paperwork. Register for the audio seminar hosted by FX Conferences on February 19th and you will understand why separate audits of support processes are largely unnecessary. You will be able to complete turtle diagrams for any process in minutes, and you will learn how to strategically select auditors according to the process flow.

Audio seminar with FX Conferences on February 19th:

Click Here“Adding Value to Audits Using the Process Approach”

This seminar will demonstrate how to use turtle diagrams and the process approach to perform audits. The presentation materials include a blank turtle diagram template and a sample, completed turtle diagram for an incoming inspection process. You will also learn how to assign auditors differently in order get more value from your audits.

This audio conference covers:

  • What is the process approach to auditing?
  • Why is the process approach more efficient than audit checklists?
  • What is a turtle diagram?
  • In what order should you ask audit questions?
  • Who should you assign to each process and why?
  • How you can add more value during audits?

How to recruit, hire and train an auditor

In Internal Auditing, ISO 19011, Supplier Audit, Supplier Audits, Supplier Qualification, Training on December 24, 2012 at 11:44 pm

Part 3: Training

Passing a webinar on auditing does not make you competent.

Does your company ask incoming inspectors to update CAD drawings when there is a design change? Of course not. Your company has engineers that are trained to use SolidWorks, and it takes a new engineer a while to become proficient with the software. Auditing is a skill that you learn—just like SolidWorks.

My favorite holiday movie…I’ll be watching this later tonight!

I’ve never met a manager that wondered where the value was in having an engineer update a drawing, but many managers view internal and supplier audits as a necessary evil. Instead of asking the expert how few audit days you can get away with, ask the expert: “What is the purpose of auditing?”

The purpose of internal auditing is to confirm that the management system is effective and to identify opportunities for improvement. The purpose of supplier auditing is to confirm that a supplier is capable of meeting your needs and to identify opportunities for improvement. Therefore, if an auditor has no nonconformities and no opportunities for improvement were identified—what a waste of time!

To receive value from auditing, you need auditors that are competent. In clause 6.2.1 of the ISO 13485 Standard it says, “Personnel performing work affecting product quality shall be competent on the basis of appropriate education, training, skills and experience.” As the audit program manager, make sure you recruit people that demonstrate auditing competency.

Education

First, educational background is important for auditors. You cannot expect someone who has never taken a microbiology course in their life to be an effective auditor of sterilization validation. Likewise, someone that has never taken a course in electricity and magnetism will not be effective as an auditor for active implantable devices. Therefore, determine what types of processes the auditor will be auditing. Then make sure that the person you hire to be an auditor has the necessary education to understand the processes they will be auditing.

Training

Second, an auditor needs to be trained before they can audit. The auditor needs training in three different aspects: 1) the process they will be auditing, 2) the standard that is the basis for assessing conformity, and 3) auditing techniques. If you are going to be auditing printed circuit board (PCB) manufacturers with surface-mount technology (SMT), then you need to learn about the types of components used to make PCBs and how these components are soldered to a raw board. I know first-hand that anyone can learn how SMT works, but it took me a few months of studying.

If your company is only selling medical devices in the USA, then you will need to learn 21 CFR 820 (i.e. – the QSR). However, if your company also sells devices in Europe in Canada you will need to learn ISO 13485, the MDD (93/42/EEC as modified by 2007/47/EC), and the Canadian Medical Device Regulations (CMDR). I learned about ISO 13485 in a four-and-half day lead auditor course in Florida, I learned about the MDD in a three-day CE Marking Course in Virginia, and I learned about the CMDR in a two-day course taught by Health Canada in Ontario. A 50-minute webinar on each regulation is not sufficient for auditing.

Finally, you need training on the techniques of auditing. A two day course is typically needed. I took a 50-minute webinar and passed a quiz before conducting my first internal audit, but I was not competent.

Skills

Third, an auditor needs specific skills to be effective as an auditor. The most critical skills are: 1) communications skills, 2) organizational skills, and 3) analytic skills. Communications skills must include the ability to read and write exceptionally well and the auditor needs to be able to verbally communicate with auditees during meetings and interviews. The most difficult challenge for auditors is covering all the items in their agenda in the time available. The auditor rarely has more time than the need to audit any topic, and audit team leaders must be able to manage their own time as well as simultaneously managing the time of several other auditors. 

Experience

Last, but certainly not the least important aspect of auditor competency is experience. This is why 3rd party auditors are required to act as team members under the guidance of a more experienced auditor before they are allowed to perform audits on their own. This is required regardless of how many internal or supplier audits the person may have conducted in the past. More experienced auditors are also required to observe new auditors and recommend modifications in their technique. Once a new auditor has completed a sufficient number of audits as a team member, the auditor is then allowed to practice leading audits while being observed. After six to nine months, a new auditor is finally ready to be a lead auditor on their own. An internal auditor does not need the same degree of experience as a 3rd party auditor, but being shadowed 2-3 times is not sufficient experience for an auditor (1st or 2nd party). For more information about this topic, please read my blog posting on auditor shadowing.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

If you are an audit program manager, and you would like to improve your own competency, please contact me to learn about a new advanced course specifically for audit program managers. I am teaching a course with Brigid Glass. The course is designed specifically for audit program managers—not for inexperienced auditors. It will be a two-day course, and we are offering the course in three different cities: San Diego, CA (April 11/12), Orlando, FL (April 15/16) and Las Vegas, NV (April 17/18). Please Contact Me if you would like to learn more about the course.

Click Here

I am also teaching a one-hour, audio seminar with FX Conferences on January 9th:

“Are Your Suppliers Qualified? Prove It.”

This seminar will cover the areas of supplier qualification, supplier evaluation and supplier auditing. We already have a large number of companies signed-up for the seminar, and I am looking forward to having you join us.

This blog started as a single posting, but I realized that the blog was much too long. Therefore, I split the blog into three separate postings. This is the final “Part 3 of 3”. I hope you have enjoyed it. If you have suggestions for my next posting, please let me know.

How to recruit, hire and train an auditor

In Internal Auditing, ISO 19011, Supplier Audit, Supplier Audits, Supplier Qualification on December 24, 2012 at 4:39 am

Part 2: Hiring

Welcome Aboard

If you are an audit program manager that is training a new auditor from another department, treat them like a new hire!

Once you have identified someone that you want to “hire” as an internal auditor, your next step should be to develop an “Onboarding” plan for them with their boss. If you are hiring someone that will be a dedicated auditor, please ignore my quotation marks above. In most companies, however, the internal auditors are volunteers that report to another hiring manager. Therefore, as the audit program manager you need to get a firm commitment from the auditor’s boss with regard to the time required to train the new auditor and to actually perform audits on an on-going basis.

The Trans Siberian Orchestra is a must see–especially if you can take you family to see the performance live.

Winning Over the Boss

In my previous posting I said that, “The biggest reason why you want to be an auditor is that it will make you more valuable to the company.” The auditor’s boss may or may not agree with this statement, but the boss knows that the salary is coming out of their budget either way.

Therefore, talk with the auditor’s boss and find out what the auditor’s strengths and weaknesses are. Find out which skills the boss would like to see the auditor develop. By doing this, the two of you can develop a plan for making the auditor more valuable to their boss AND the company.

Making Re-Introductions

Ideally, auditors are extraverted and they have been with the company long enough to know the processes and process owners that they will be assigned to audit—especially if they will be auditing upstream and downstream from their own process area. In the past the auditor was a customer or a supplier, but now the relationship with a process owner will change. Auditors are required to interview process owners and this involves asking tough questions that might not be appropriate in the auditor’s normal job duties. Therefore, as the audit program manager, you should re-introduce the auditor to the process owner in their new capacity as auditor.

During this re-introduction, it is important to make three points:

  1. the auditor is going to be trained first,
  2. you will be shadowing the auditor during the audit, and
  3. the auditor’s job is to help the process owner identify opportunities for improvement.

By making the first point, you are reminding the process owner of the scheduled audit—well in advance. You are also informing the process owner that this auditor will have new skills, and the process owner should have some tolerance for mistakes that new employees make. You might also mention that you would like to get the process owner’s feedback after the audit so the auditor knows what areas they need to improve to become better auditors.

The second point should put the process owner at ease—assuming the process owner has a good relationship with you as the audit program manager. It is important to be descriptive when “shadowing” is mentioned. Both the process owner and the auditor may not understand the process or the purpose of shadowing. The following blog posting might help with this: “How do you shadow an auditor? Did you learn anything?”

The third point is the most critical step in onboarding a new auditor. For an auditor to be successful, they must ADD VALUE!

As an auditor, you cannot pretend to add value.

The process owner should know their process and they probably know which areas are weakest. The audit program manager should encourage the process owner to list some specific areas in which they are having problems. Ideally, the process owner would be informed of this need prior to the re-introduction. Then the process owner can be better prepared for the meeting, and hopefully they will have a few target areas already identified. Targets with associated metrics are the best choice for a new auditor, because these targets reinforce the process approach to auditing.

Next Steps

Once your new auditor has been re-introduced to the process owners they will be auditing, you need to begin the training process. As with any new employee, it is important to document the training requirements and to assess the auditor’s qualifications against the requirements of an auditor. Every new auditor will need some training, but the training should be tailored specifically to the needs of the auditor.

The training plan for a new auditor should include the following:

  1. a reading list of company procedures specific to auditing and external standards that are relevant;
  2. scheduled dates for the auditor to shadow another experienced auditor;
  3. scheduled dates for an experienced auditor to shadow the auditor during the first two process audits (upstream and downstream);
  4. goals and objectives for the internal audit program; and
  5. any training goals that the auditor’s boss has identified for the auditor.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

If you are an audit program manager, and you would like to improve your own competency, please contact me to learn about a new advanced course specifically for audit program managers. I am teaching a course with Brigid Glass. The course is designed specifically for audit program managers—not for inexperienced auditors. It will be a two-day course, and we are offering the course in three different cities: San Diego, CA (April 11/12), Orlando, FL (April 15/16) and Las Vegas, NV (April 17/18). Please Contact Me if you would like to learn more about the course.

Click Here

I am also teaching a one-hour, audio seminar with FX Conferences on January 9th:

“Are Your Suppliers Qualified? Prove It.”

This seminar will cover the areas of supplier qualification, supplier evaluation and supplier auditing. We already have a large number of companies signed-up for the seminar, and I am looking forward to having you join us.

This blog started as a single posting, but I realized that the blog was much too long. Therefore, I split the blog into three separate postings. This post is “Part 2 of 3”. The final part in the series will be posted tomorrow–December 24, 2012.

How to recruit, hire and train an auditor

In Internal Auditing, ISO 19011, Supplier Audit, Supplier Audits, Supplier Qualification on December 22, 2012 at 7:57 pm

Part 1: Recruiting

Stop begging people to help you audit. Learn how to recruit auditors more effectively.

Stop begging people to help you audit. Learn how to recruit auditors more effectively.

Nearly 100% of the people I train as auditors were not hired specifically to be auditors. Instead, auditing is something extra that they were asked to do in addition to their regular job. This situation creates three problems for the audit program manager:

  1. you have difficulty getting enough people to perform the audits;
  2. most auditors will come from your department, so who is going to audit you; and
  3. the auditors have little or no motivation to get better at auditing.

Stop begging for “volunteers” from other departments and start recruiting.

My favorite holiday song of all time! I sing this to myself in car rides during July.

When I am recruiting someone to audit, I always get asked two questions:

  1. Who/What will I be auditing?
  2. What will I have to do?

You need to motivate people to become auditors, because it requires extra work. The answer to #2 should be specific. I recommend creating a “sell sheet” that explains the process of performing an audit. I also like to create sell sheets that are educational. Therefore, I recommend adapting the flow chart in ISO 19011:2011 (Figure 2 on page 15). I would add time estimates for each step of the process (6.2 – 6.7). This will serve as a training tool for future auditors, and it will eliminate the fear of unknown time commitment for your potential recruit.

In order to answer #1, I recommend you assign the recruit processes that are upstream and downstream. I have recommended this concept in previous postings, but essentially you are assigning the person to audits of internal suppliers and internal customers. By doing this, utilizing the process approach will be more natural to the auditor and they will have a vested interest in doing a thorough audit. This also creates a situation where the auditor is typically assigned to at least two process audits per year.

The next question is one that your potential recruit will never ask, but they are always thinking it…

Why should I become an auditor?

The biggest reason why you want to be an auditor is that it will make you more valuable to the company.

Auditors are required to interview department managers and ask tough questions. This gives the auditor a better understanding of the organization as a whole, and it gives them insight into how other managers work. This insight is pure gold.

If you want to be effective and get promoted, you need to demonstrate value to your boss and top management. If you don’t understand what other departments need, how can you help them? No manager will promote a selfish, power-hungry hog. They promote team players that make others better. Auditing gives you the insight necessary to understand how you can do that.

Auditing other departments will also give you insider information as to where new job openings will be. Sometimes you can’t wait for your boss to get promoted. In that case, you might want to know more about other departments in your company.

Each corporate culture is different, but the audit program manager needs to “sell” the recruit on volunteering to be an auditor.

Where to find recruits

Due to the cross-functional nature of auditing, I have found that my own personal experience working in multiple departments was invaluable. I have a better understanding of how a department functions than other auditors, because I have worked in that department at another company. Operations, engineering and research experience are extremely valuable for auditing, but I think the experience that transfers the best to auditing is service.

If your company is large enough to hire full-time auditors, I recommend searching for potential auditors at your suppliers and their competitors. These people will bring unique knowledge that is critical to a successful supplier selection process, and these individuals will increase the diversity in your company—instead of duplicating knowledge and expertise.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This blog started as a single posting intended to help a Compliance Manager in the Twin Cities. Unfortunately, I ran out of time to finish the blog and it has been a couple of weeks since my last post. When I restarted the blog this weekend, I realized that the blog was much too long. Therefore, this is part 1 of 3. Part 2 will be about hiring auditors, and part 3 will be about training auditors.

For those of you that want to learn more, I am teaching a course with Brigid Glass in April. The course is designed specifically for audit program managers—not for inexperienced auditors. It will be a two-day course, and we are offering the course in three different cities: San Diego, CA (April 11/12), Orlando, FL (April 15/16) and Las Vegas, NV (April 17/18). Please Contact Me if you would like to learn more about the course.

Click Here

I am also teaching a one-hour, audio seminar with FX Conferences on January 9th:

“Are Your Suppliers Qualified? Prove It.”

This seminar will cover the areas of supplier qualification, supplier evaluation and supplier auditing. We already have a large number of companies signed-up for the seminar, and I am looking forward to having you join us.

How do you audit for compliance with ISO 14971:2012?

In Internal Auditing, ISO 14971, Risk Analysis, Risk Management on December 2, 2012 at 1:41 pm

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and you identified a couple of gaps in your procedure. After you revised your Risk Management Procedure to be compliant with the revised Standard, then what are you supposed to do?

For the next few weeks I plan to torture all of you with holiday music. If you don’t like it, buy a satellite radio for Christmas sake.

Most QA Managers struggle over whether they should purchase ISO 14971:2012 or not. I wrote a couple of blog postings about this, but my point was not to debate this question. My point was that companies need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from the 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are 7 aspects of the ISO 14791 Standard that  do not meet the requirements of the MDD. Therefore, if your company has already verified that your Risk Management Process is compliant with the MDD–then you have nothing to change. However, if your Risk Management Process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these 7 aspects.

Once you have made your revisions, how do you audit for compliance with ISO 14971:2012?

Step 1: Planning the Audit

This will be an internal audit and since you (the QA Manager) are the process owner for the Risk Management process, you cannot also audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she is not trained on ISO 14971:2012. To address this gap, she must read the updated Standard to understand what’s new.

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December, because you want any CAPA’s resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important, because the Risk Management Procedure requires that top management assess the effectiveness of the Risk Management Process during Management Review meetings.

There are no previous audit findings to close from the last audit of the Risk Management Process, but the Director of Engineering has 7 specific items to emphasize from the 2012 revision of the Standard and a revised procedure for Risk Management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more generic, open-ended questions.

Specific Questions for 7 Items in ISO 14971:2012, Annex ZA:

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded.)

2. Please provide a few examples of how risks in the lowest category were reduced. (Sections 1 and 2 of the Annex I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling.)

3.  How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination.)

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk.)

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level.)

6. What were your team’s priorities for implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I.)

7. How was effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk.)

Auditor TipThe above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still ok to include questions that use the element approach.

Generic Questions:

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the Risk Management Report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated Risk Management Procedure prior to conducting a risk analysis?

Auditor TipThese generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by asking open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Conducting the Audit

The next step of the auditing process is to conduct the audit. During the audit, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the Risk Management Process and other processes such as: 1) document control, 2) creating technical documentation for regulatory submissions, and 3) the training process. The specific questions verify that each of the 7 elements identified in Annex ZA of ISO 14971:2012 are adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s) so that everyone is clear what the findings were, and if there were any nonconformities this is the time to clarify what needs to be done in order to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit, but it is critical to have the report completed soon enough so that CAPA’s can be initiated (not necessarily completed) prior to the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure the effectiveness. Then the effectiveness check is objective. For example, monitoring the frequently of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the Risk Management records referenced in the Essential Requirements Checklist indicates if the Risk Management process is  being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a train gap is just an example of a single lapse.

How do you shadow an auditor? Did you learn anything?

In Elsmar Cove, FDA Inspections, Internal Auditing, ISO 13485, ISO 19011, Supplier Audit, Supplier Audits, Supplier Quality on November 25, 2012 at 5:43 am

If you are shadowing, you are taking notes so you can discuss your observations with the person you are shadowing later.

Somewhere in your procedure for “Quality Audits”, I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for internal auditor or a lead auditor course. If the course had an exam, then you might even have evidence for training effectiveness. Demonstrating competency is much harder. One way is to review internal audit reports, but writing reports is just part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require that the auditor “shadow” a more experienced auditor several times, and then the trainee will be “shadowed” by the trainer.

I can’t remember posting any music from John Mayer and the song title fits our subject for this blog.

Shadowing 1st Party Audits:

ISO 19011:2011 defines 1st party audits as internal audits. When 1st party auditors are being shadowed by trainer, or vice versa, there are many opportunities for training. The key successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer is asking the trainee what they should do BEFORE they do it. If the trainee is not sure, the trainer should explain what, why and how at that moment with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

Here are five (5) mistakes that I have observed trainers make when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit than taking every advantage of training opportunities. The trainee may be capable of auditing on their own, but this is no excuse for tag teaming the auditee. This is unfair to the trainee AND the auditee. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover some time in their schedule. Time management is after all one of the hardest skills for auditors to master.

2. Staying in the conference room, instead of going to where the work is done, is a common criticism of auditors. If the information you need to audit can be found in a conference room, then you could have completed the audit remotely. This type of audit teaches new auditors very little other than how to take notes. These are basic skills that auditors should master in a classroom prior to shadowing.

3. Choosing an administrative process is a mistake, because administrative processes limit the number of aspects of the process approach that can be practiced by an auditor-in-training. Administrative processes rarely have equipment that requires validation or calibration, and both the process inputs and outputs consist only of paperwork, forms or computer records. With raw materials and finished goods to process, the job of the auditor is more challenging because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick skinned or they don’t belong in a role where they are going to criticize others. Before you begin telling other people how to improve, you first need to self-reflect and identify your own strengths and weaknesses. Understanding your own perspective, strengths, weaknesses, and prejudices is critical to being an effective assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this I mean shadowing another auditor without asking questions. If you are a trainee you should be mentally pretending you are doing the audit. Whenever the trainer does something different from the way you would do things, you should make a note so you can ask, “Why did you do that?” If you are trainer you should also be mentally pretending you are doing the audit. It is not enough to be present. You job is to identify opportunities for the trainee to improve. The better the trainee, the tougher your job becomes. This is why I training other auditors has helped me improve my own auditing skills.

Shadowing 2nd Party Audits:

If you are developing a new supplier quality engineer that is responsible for performing supplier audits, it is recommended to observe the auditor during some actual supplier audits. Supplier audits are defined as 2nd party audits in the ISO 19011:2011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls in place to consistently manufacture conforming product for your company. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a 2nd party audit.

The two most valuable process for a 2nd party auditor to sample are: 1) incoming inspection, and 2) production controls. Using the process approach to auditing, the 2nd party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these process. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment in use by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

My recommendation is to have the trainee shadow the trainer during the process audit of the incoming inspection process and for the trainer to shadow the trainee during the process audit of production processes. In between the two process audits, the trainee should be asking questions to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observe processes that may involve trade secrets or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather information that will be needed for following audit trails to calibration records, document control or for comparing with the validated process parameters. The “teachable moment” is immediately after the trainee missed an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Shadowing 3rd Party Audits:

Use your FDA inspections and ISO certification audits as an opportunity to shadow experienced auditors and to learn what they are looking for.

If you are going to shadow a 3rd party auditor, I recommend two specific people to “shadow” the auditor. First, the process owner should be the guide for whichever process is being audited. This is the person that will be responsible for addressing any nonconformities found in the area, and they should be present during interviews–although they should be coached on when to comment and when to remain quiet and simply observe.  Second, the person that performed an internal audit of the process being audited should be present if at all possible. This person will benefit from seeing how a professional 3rd party auditor performs a process audit, because they will know which things to look for in the future so that auditees in that area are prepared for the next external audit.

If you are an audit program manager, and you would like to learn “What Makes World Class Audit Programs Different?”, please contact me. I am co-teaching an advanced course for audit program managers in April 2013.

For other sources of information related to auditor shadowing, please check out the following links:

1. Internal Auditor Training – Shadowing external auditor? – from Elsmar Cove

2. Developing Supplier Quality Auditor Training Programs – by Seth Mailhot at NixonPeabody

Attention Auditors! – Have you read ISO 19011?

In Audit Schedule, Internal Auditing, International Standard, ISO, ISO 19011, PDCA, Procedures, Quality Management Systems on July 20, 2012 at 2:58 pm

If you have ever taken a lead auditor course, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Management Systems”. In November of last year, this standard was updated and the changes were not superficial.

The background entertainment for this week is one of my favorite modern rock songs, but it never seemed to get much air time. I hope you enjoyed the 90’s.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting both internal and external audits, and how to determine auditor competency. Improvements to the New 2011 Version of the Standard include:

  1. Broadening the scope to all management systems
  2. Clarifying the relationship between ISO 17021 and ISO 19011
  3. Introduction of the remote audit methods
  4. Introduction of risk as an auditing concept
  5. Confidentiality is a “new” principle
  6. Clause 5, Managing an audit program, was reorganized
  7. Clause 6, Performing an audit, was reorganized
  8. Clause 7, Competence and evaluation of auditors, was reorganized & strengthened
  9. Annex B is new and the contents of the help boxes was moved to this Annex
  10. Annex A now includes examples of discipline-specific knowledge and skills

One of the most common points of confusion in the lead auditor course is the difference between 1st, 2nd and 3rd party audits. In the previous revision of this Standard, this was just a note at the bottom of page 1 and the top of page two. The note was not very clear either. The new version of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear:

The above table is just an example of the improvements made to ISO 19011, and of course there is little value-add to clarifying a definition. Figure 1 from the new version, “Process flow for the management of an audit program, is a better example of a “value-add”. This vertical flow chart is reminiscent of Figure 1 from ISO 14971:2007. It categorizes the various stages of audit program management into the Plan-Do-Check-Act (PDCA) cycle. I highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard. Unfortunately Figure 2, “Typical audit activities”, does not categorize the stages of audit activities (Clauses 6.2 – 6.7 of the revised Standard) into the PDCA cycle. I guess they needed to leave some improvement for the next revision.

The new version retained the opening meeting checklist that was in the previous revision (Clause 6.4.2), and Clause 6.4.9 has a brief closing meeting checklist. Figure 3, “Overview of the process of collecting and verifying information”, is a poor example of a flow chart. Should I make a better one? (Send me an email if you think I should.)

The most valuable changes in this revision are Clause 5.3.2, “Competence of the person managing the audit program”, and all of Clause 7. Most of the audit procedures I read neglect to define the qualifications and method for determining competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures I read include qualifications for a “Lead Auditor”, but I seldom see anything regarding competency. Unfortunately, this Standard only specifically addresses “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When I teach people how to be a lead auditor, I spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how 3rd party auditors become qualified as a Lead Auditor. 3rd party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meeting, conducting the audit, closing meeting, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e. – Stage 2 certification or recertification), and another qualified lead auditor must evaluate you and provide feedback.

The last big additions to this Standard were the Appendices. Annex A provides examples of discipline-specific knowledge and skills of auditors. This section is a little on the boring side. I prefer to tell a story about the internal auditor that was auditing incoming inspection—but they had no idea how to check for calibration or how to measure components. Appendix B, the finale, has a table (Table B.1) that provides some guidance on how to conduct remote audits (i.e. – desktop audits). I was pleased to see that conducting interviews is a major part of remote auditing in this table. Section B.7 provides some suggestions with regard to conducting interviews, but if you exhibit all 13 of the professional behavior traits found in Clause 7.2.2 then you really don’t need any advice on how to speak with people. For the rest of us mortals, we could use a five day course on interviewing alone.

Additional guidelines are available on the ISO website.

How to Train an Auditor on the Process Approach

In Audit Schedule, Internal Auditing on June 8, 2012 at 5:53 am

Country music fans are loyal blog readers too.

            I have been reviewing the trends for how people find my website, and a large number of you appear to be very interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:

  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

There are two reasons why the process approach is recommended. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments the process approach will catch it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked. Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e. – the element approach).

My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit. For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment.

The tool that BSI uses to teach the process approach is the “Turtle Diagram”. The following picture illustrates where the name came from.

Illustration of the Process Approach

Process Auditing – “Turtle Diagram”


The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview of the process, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gages used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list the auditor should select people to interview and follow-up with a request for training records.

The sixth step of the process is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company Quality Objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.

In my brief description of the process approach I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company and it is easy to understand. More importantly, however, the incoming inspection process does a great job of covering more clauses of the Standard than most audits. Therefore, new auditors get a great appreciation for how almost all the clauses can be addressed in one process audit.

If you have questions, or you would like a copy of the turtle diagram I use for documentation of audits, please submit a request on my website contact us page.

%d bloggers like this: