Governing and Auditing AI Automated Processes

Dive In

While the benefits of using AI for business process automation are not hidden, it is equally important to keep a check on the machine itself. Governing and auditing AI automated processes is important because these systems are adaptive and are not just rule-based. In contrast to traditional automation, AI makes decisions that are often opaque. Such decision-making behind the curtains can infuse bias, unfairness, lack of transparency and accountability, privacy issues, and other unwanted consequences into business processes.

This blog highlights six core principles involved in the governance and audit of AI automated processes.

Core Principles for AI Process Governance & AI Process Audit

Fairness

AI models learn from the data they are ingested for training. It can inadvertently absorb and augment any kind of personal, historical, political, or societal bias present in that data. That is not much different from why we do not allow children to watch all kinds of content on the internet.

Suppose that a company incorporates an AI-powered promotion decision model. If the company had a history of promoting employees with a master’s degree, then based on the historical data on promotion decisions of that company, the AI model may weigh ‘having a master’s degree’ as a strong factor and possibly, using it as a protected attribute. The company need not necessarily hold such a philosophy when considering candidates for promotions. But, in this case, the AI model can inadvertently provide biased results.

Such fairness issues can also emerge in process AI models. For example, in the selection of suppliers, AI may place heightened emphasis on factors like the tenure of association with the company – which need not necessarily be a ground for the selection of a supplier.

In other words, this principle of ‘fairness’ could be seen as efforts to ensure that an AI model does not make ‘unfair’ decisions. In the context of process governance and audit, the key takeaway here is that process stakeholders should always question and examine the factors behind AI’s decisions.

Robustness

Robustness in an AI system or model refers to its ability to uphold its intended performance, stability, and integrity when it encounters unexpected conditions – good or bad. Performing under lab conditions is one thing, but performing under real-world situations is quite another. Two important requirements for ensuring robustness in an AI system are highlighted ahead.

A robust AI should have resilience to data perturbations. It is common for AI systems to encounter variations in input data. In such situations, an AI model should not go haywire or fail to deliver the intended outcomes. For example, an AI-powered, self-driving car should be able to correctly identify traffic signs even if the visibility is low due to poor lighting or poor weather conditions.

A strong AI system must have a strong resistance mechanism to shield itself against adversarial attacks. Adversarial attacks are aimed at disrupting the performance of an AI model. In an adversarial attack, data inputs to an AI model are intentionally manipulated so that the AI makes mistakes. These data manipulations are planned in such a way that they cannot be easily detected by humans.

Explainability

Explainability in AI means that the stakeholders should be able to comprehend the reasoning behind AI’s decisions. For example, if AI rejects a loan application, it should also disclose the factors behind that decision. Another important factor is interpretability. Interpretability is the degree to which a user can understand the AI reasoning behind the decisions. This makes it necessary to tailor these explanations, keeping in mind the user or user group in question. For example, someone from HR need not necessarily understand jargon from the field of AI or data science. Both these concepts – explainability and interpretability play an important role in drawing the best results from AI-driven process optimisation.

Transparency

AI’s decision-making is complex and remains behind the curtain. It is important for the stakeholders involved to be able to look into and understand how the AI model arrives at any decision.

For transparency, stakeholders involved in developing, deploying, or using a process AI system must be aware of the model’s design, training data, accessibility rights, capabilities, and limitations. It is also important to clearly establish the roles of the people or teams involved in the process of model development, evaluation, deployment, and upkeep.

Privacy & Compliance

AI models deal with large volumes of data that often include personal information. In general, the concern is securing the privacy of individuals. In the case of business processes, the issue revolves around data access rights and data visibility. For instance, HR does not need to have access to any sensitive information of Finance or Operations. The same is true the other way round. It also applies to third parties like suppliers and customers. Even within a team or a department, the same principle must be maintained. Organisations must also adhere to applicable regulatory norms governing privacy rights.

Three effective strategies for securing privacy in AI systems are:

• Model Anonymisation: (E.g. changing names, not affecting the model’s outcomes)
• Differential Privacy: (E.g. adding irrelevant information, not affecting the model’s outcomes)
• Data Minimisation: (collecting only the required data)

Human Role and Accountability

Human role and accountability are necessary to ensure that a human or an organisational position, team, or department is ultimately accountable for the outcomes and impact of a process AI system.

In process AI solutions, the role and accountability of individuals and teams must be clearly defined – from the CEO down to the AI engineers, data scientists, business intelligence, privacy experts, users/process owners, and anyone who is involved in the development, implementation, and use of an AI model (process AI).

In delivering process AI consulting services, BPX maintains that there should always be a provision for human intervention or human decision-making whenever AI makes any erroneous judgment, or if it is known in advance that human involvement is necessary at any stage in a business process.

Wrapping Up

AI’s opaque decision-making can infuse bias, unfairness, lack of transparency and accountability, privacy issues, and other unwanted consequences into business processes. This necessitates strong governance and audit mechanisms for business process AI solutions. This blog highlighted six core principles towards addressing the issue.

Fairness – AI models can inadvertently absorb and augment any kind of personal, historical, political, or societal bias present in the input data. For improved process governance and audit, process stakeholders should always question and examine the factors behind AI’s decisions.

Robustness – Robustness in an AI system or model refers to its ability to uphold its intended performance, stability, and integrity when it encounters unexpected conditions – good or bad. A robust AI for workflow automation solution should have resilience to data perturbations. A strong AI system must also have a strong resistance mechanism to shield itself against adversarial attacks.

Explainability – Explainability in AI means that the stakeholders should be able to comprehend the reasoning behind AI’s decisions. Interpretability makes it necessary to tailor these AI explanations, keeping in mind the user or user group in question.

Privacy & Compliance – In the context of intelligent process automation (IPA) for business operations, the issue of privacy revolves around data access rights and data visibility. Organisations must also adhere to applicable regulatory norms governing privacy rights. Three effective strategies for securing privacy in AI systems are Model Anonymisation, Differential Privacy, and Data Minimisation.

Transparency – One of the fundamental requirements for securing transparency when using process automation using AI is that the stakeholders involved in developing, deploying, or using a process AI system are aware of the model’s design, training data, accessibility rights, capabilities, and limitations.

Human Role and Accountability – Absolute reliance on AI is not a prudent choice. There should always be a provision for human intervention or human decision-making whenever AI makes any erroneous judgment, or if it is known in advance that human involvement is necessary at any stage in a business process.