Can you automate controls testing and still provide independent assurance?

This article is a part of Risk Update 4. 

---

Developments in artificial intelligence, machine learning and robotics have demonstrated that many activities thought to be too complex to be automated, can now be outsourced to a ‘digital’ workforce.

What is Controls Testing?

Controls are activities and processes that organisations put in place to help manage and reduce the inherent risks. Controls testing involves assessing the adequacy and existence of controls that is relied upon by organisations to mitigate risks.

Controls testing can be an arduous task for auditors and depending on the complexity of the controls environment, often involves a considerable investment of time and resources. Therefore, a key question that is often asked by management is how can we reduce the cost and time spent on compliance activities?

Manual vs Automated Testing

There are a number of benefits when it comes to using automation, with one of the primary reasons being to reduce the cost and time spent on compliance activities. As most of these activities are heavily reliant on cognitive abilities, the predominant approach has been to rely on people to perform these tasks.

Before we explore the role of automation, we should first understand what the typical activities are within a controls test. In any audit review, the following activities will need to be performed by the Internal Audit function:

Five considerations to make before adopting automation

1. Can the control testing be automated?

Automation of controls testing is dependent on the complexity of the control and the level of judgement required by the Internal Auditor when evaluating if a control is meeting its control objectives. For example, controls such as terminations testing can be automated as these have defined variables that can be configured in a supporting tool. This could include performing an assessment to determine whether the employee’s termination date in the organization’s network (e.g. Active Directory) is greater than the end date in the central HR system. This could be an indicator that removal of terminated user roles are not being performed in a timely manner.

On the other hand, controls that require the Internal Audit judgement may not be suitable for automation of controls testing. An example of this includes determining that changes are sufficiently tested and approved prior to migration to production. Testing over these type of attributes is reliant on an auditor’s judgement, as automation would be insufficient to determine the nature and complexity of the change.

Key Considerations:

- Consider the complexity of the control and level of judgement required.
- Control being automated should have defined variables, which can be configured in a supporting tool (e.g. – terminations testing).
- The automated control should require minimal Internal Auditor judgement (e.g. – testing for changes).

2. Do you have the right tools and expertise?

Automation of control testing is dependent on the available tools and expertise of the personnel to configure them. Examples of tools that can be used to assist with automation include BluePrism, Alteryx and ServiceNow. These tools allow  for automation through the process of collation, processing, documenting and output of testing results.

It is important to note that to properly utilize such tools, it requires appropriate personnel who are knowledgeable and experienced in using the tool. Hence, an organisation must consider whether it has the skills and capabilities necessary to create and manage bespoke automated workflows.

The tools should also be able to select samples (where necessary) in line with the organisation’s requirements. For example, the tool should be able to randomly select samples from a population, where an organization’s policy mandates random sampling.

Key Considerations:

- Determine whether the expertise of key personnel to automate controls testing is available in-house or needs to be sourced
- Determine the supporting tools that can used for automation. A cost benefit analysis should also be performed.

3. Is the source data complete and accurate?

As part of the automation of controls testing, it is important to ensure that the population used for the testing is complete and accurate. This is a key requirement in controls testing to ensure that no data has been accidentally or intentionally excluded.

Within the automation of controls testing, it is important to build in verification procedures to ensure that the data source is validated for ensure that the appropriate parameters are applied to generate it.

In addition, it is also good practice to perform a manual reperformance of the completeness and accuracy of the source data. This is to ensure that the right level of assurance is provided to key stakeholders by ensuring no emission of the population.

Key Considerations:

- Ensure verification procedures for completeness are built in supporting tool.
- Consider implementing manual reperformance of population to validate completeness.

4. Will the level of assurance be sufficient?

The Internal Audit function provides assurance over whether the controls meet their control objectives. This is shared with the Audit and Risk Committee (ARC). In addition, there may be arrangements with the External Auditor of the organisation for a ‘Reliance on Internal Audit Approach’.

Hence, it is important to determine if the level of assurance through controls automation is sufficient to meet the requirements of ARC and the external auditors (where applicable). Automation should be thoroughly tested at implementation through manual re-performance to validate the actual outcomes are in line with the expected outcomes. The procedures for how automation of controls testing is done for each control objective should be sufficiently documented.

Key Considerations:

- Determine the level of assurance based on stakeholders (e.g. – reliance by external audit).
- Determine the level of manual re-performance required to validate control conclusions.

5. Is the automation designed to be future proof?

Whilst automation is a great way to streamline the controls testing process, it is important to consider the compatibility with future upgrades. The automation should be designed in a way so that even if there are changes to the underlying data, the program, workflow or script enabling the automation can also be easily adapted.

Where feasible, hard coding should be minimised. This is to ensure that the automation is compatible with future upgrades without requiring considerable re-scripting.

Key Considerations:

- Consider minimising use of hard coding and using dynamic variables to ensure compatibility with future upgrades. 

Conclusion

While true end-to-end automation of controls testing may still be out of reach, a semi-automated control testing approach could help bring efficiency to the tests and allow internal auditors to focus their time on performing more value adding activities. Controls testing can be automated, however it’s dependent on the complexity of your organisation, the availability of your tools and skillset of the workforce. As with all new technologies, careful consideration over organisational fit and readiness needs to be made before leaping to a wide-scaled adoption.