KEYS TO THE SUCCESSFUL AUDITING OF AI
The following are some helpful points for making AI audits successful:
Learn about AI design and architecture to determine the proper scope. AI involves a variety of technologies, people, and processes and therefore requires a high level of attention to controls, policies, and governance. AI architecture can include programming, data warehousing, stream processing platforms, machine learning toolkits, algorithms, cloud computing, cloud storage, computing clusters, computational cores, application software testing and debugging, data processing and modeling, and commercial off-the-shelf (COTS) software. From a skills perspective, AI projects require data scientists, data engineers, data architects, ai ethicists, and programmers with skills in Python, R, SQL, and matrix lab (MATLAB).
Involve all stakeholders. AI integrates various enterprise technologies and involves multiple internal teams and external third parties. Internal stakeholders include engineering and security teams on the technical side and business leaders looking at AI strategy. The daily use of cloud computing in AI implies that third parties control some infrastructures. When using cloud computing, auditors need to address risks (e.g., vendor lock-in and partitioned knowledge) differently than on-premise applications.
Educate and proactively communicate about AI with stakeholders. Because AI is immature and limited, the organization’s stakeholders may need to be aware of its use and strategy. AI auditors need to be proactive in addressing AI concerns and break down and simplify complex designs and issues in a way that stakeholders can understand. Auditors must be aware of the different contexts for AI discussions and be able to adjust the level of conversation accordingly.
Adopt and adapt existing frameworks and regulations. The absence of new AI-specific frameworks should not be a barrier to a successful audit. COBIT 2019 and other existing frameworks can be adopted to address most AI use cases that will arise in practice. From a regulatory perspective, existing charters such as the United States Health Insurance Portability and Accountability Act (HIPAA), the Fair Lending Act, and the European Union’s General Data Protection Regulation (GDPR) can also be adapted to provide legal guidance. Until more specific AI standards are introduced, existing frameworks and regulations (adapted by an accountant familiar with the AI landscape) and legal advice will suffice.
Focus on transparency through an iterative process. Transparency is a crucial objective for the AI auditor due to the complexity of the AI environment. For example, algorithms need to be optimized in multiple rounds by data scientists and data engineers. Some enterprise-based commercial off-the-shelf solutions may already include machine learning components. Similarly, the review process must ensure oversight of current and new AI developments and encourage continuous improvement and detailed documentation throughout the AI lifecycle. Indeed, AI itself can become a tool for the AI auditor.
(According to the recommendations of the EU/ISACA)
Murat
(Author of the book “MINDFUL AI — Reflection Artificial Intelligence”)
NEW RELEASE — Available on Amazon: MINDFUL AI
