Common mistakes auditors make

When organisations are seeking compliance, like my current company applies ISO 27001, they rely on auditors to give them good advice through external and internal audit. But as with any profession, auditors need some special skills to be better than others. The auditors will give advice based on standard and their experience. However, sometimes you doubt the results of the auditor's assessment. And yes, not all the auditor's advice or reviews are reliable and trusted.

Keep an eye out for these following mistakes that an auditor can make:

1. They impose their opinions without facts Any decision should be supported by an instruction in the Standard/Policy/rule.

Auditors can have preconceived ideas of best strategies and will recommend certain practices regardless of your organisation’s situation. You should only ever follow advice if the auditor can explain how it helps meet a specific compliance requirement.

Eg: When an auditor performed an audit, he found that the project did not do test reports, including test progress report and test summary report. He comments that test reports are required. But he forgot the context of that project. That is an Agile project, and please remember that in Agile development, test progress reporting may be incorporated into task boards, defect summaries, and burndown charts, which may be discussed during a daily stand-up meeting

2. They report findings but don’t provide evidence Auditors must always provide proof when highlighting areas of non-compliance.

The point is the auditor needs something concrete that they can point to, rather than citing a vague violation or general ‘feeling’ of non-compliance.

This helps the organisation understand exactly what the failure is and what it needs to do to fix the issue.

Eg: During an audit, an auditor found that many projects do not have an organized document management. But he didn't save the evidence. So the audit reports and suggestions will be difficult to convince the leadership

3. They tick off checklists without considering the bigger picture Checklists are a great way of quickly assessing whether a list of requirements are met, but what they offer in convenience they lack in in-depth analysis.

Organisations are liable to see that a requirement has been ticked off and assume that it’s ‘mission accomplished’. However, there may still be room to improve your practices, and it might even be the case that your activities aren’t necessary.

A good auditor will use the checklist as a summary at the beginning or end of their audit, with a more detailed assessment in their report, or they’ll use a non-binary system that doesn’t restrict them to stating that a requirement either has or hasn’t been met.

Eg: If the checklist has a question related to Project progress:

• Is the implementation progress of the project on time? (based on the plan) -> If not, where is it late? -> What are the next actions? -----> Check if the actual time or resource exceeds 120% vs plan? --------> If yes, the project plan needs to be updated

4. They believe the paperwork and ignore the facts Any organisation can create policies that demonstrate their commitment to meeting ISO 27001’s requirements or other standards/policy/rule, but it doesn’t mean employees actually follow those instructions.

A bad auditor might be satisfied by documentation and a cursory look at whether it’s been implemented. They must be more rigorous than that.

Auditors shouldn’t be satisfied with just what the organisation wants them to see; they should be digging deeper to check whether the rules are being followed consistently.

Eg: about the KPT implementation, you can see the project's KPT file but doesn't mean that the project executes KPT at the end of each phase / sprint

5. They feel obliged to find errors Auditors sometimes try to stamp their authority by pointing out areas of non-compliance as soon as possible. This isn’t necessarily a bad thing, but it is if they’re exaggerating the scale of a shortcoming to prove a point.

It shouldn’t take long for a good auditor to find genuine faults, as even the best-prepared organisation will have room for improvements.

Auditors should keep this in mind at the start of their assessment, otherwise they’ll end up with an unfairly long list of faults or an inconsistent interpretation of the requirements.

Eg: In an ISO audit by external body, one NC about keeping incident evidence is raised -> It should be comment for improvement, not actual NC.

6. They allow cost-cutting to starve the audit This mistake occurs more often in internal audits, with organisations acknowledging the need to assess their practices but unable or unwilling to provide the necessary resources.

An underfunded audit will lead to rushed and incomplete results that have little value, and a good auditor will be able to tell if the scale of the project is too big for what’s been budgeted.

Eg: PQA team wants to implement some audit. Maybe managers agree to that, but leaders/team members have not fully grasped the meaning of the audit. As a result, they do not provide enough resources to participate in the assessment.