Advanced in AI audit (AAIA) (AAIA) - Isaca

Detecting data drift is critical in maintaining the reliability and accuracy of AI models, especially in dynamic environments like healthcare where patient populations and data characteristics can change over time. According to the ISACA Advanced in AI Audit™ (AAIA™) Study Guide, data drift refers to changes in the input data's statistical properties compared to the data on which the model was originally trained. If not detected, data drift can degrade model performance and lead to erroneous predictions. The most effective approach to detect data drift is to continuously compare the statistical distributions of incoming (production) data with those of the training data set. This allows organizations to identify deviations in data patterns, which can be early indicators that the AI model’s predictions may no longer be valid or optimal.<br />
<br />
As stated in the AAIA™ Study Guide under "AI Model Monitoring and Maintenance": “Monitoring input data for distributional changes compared to the model’s training data is an essential step in identifying data drift. Statistical tests and visualizations can help auditors and AI operators detect when the underlying data characteristics have shifted, prompting further investigation or retraining needs.”<br />
While options such as retraining the model (option C) or adversarial testing (option D) are valuable<br />
for ongoing performance and robustness, they do not inherently detect data drift—they respond to<br />
or stress-test existing issues. Applying overrides (option B) is a human-in-the-loop safeguard, not a method for drift detection.<br />
<br />
Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: "AI Model Monitoring and Maintenance," Subsection: "Detection and Management of Data Drift"

Transparency in AI systems is a key requirement to ensure trust, accountability, and ethical compliance. According to the ISACA AAIA™ Study Guide under the "AI Governance and Risk Management" section, understanding the decision-making process of an AI system falls under the principle of explainability. Explainability refers to the degree to which an observer can understand the internal mechanics of an AI system and the rationale behind its outputs. “Reviewing the explainability of AI outputs allows auditors and stakeholders to determine whether model decisions are interpretable and justifiable. High transparency means stakeholders can trace<br />
<br />
how and why a decision was made.”<br />
While algorithm complexity and computational cost are technical considerations, they do not directly<br />
facilitate the audit of decision-making transparency. Similarly, training data diversity is essential for bias reduction but does not explain how decisions are derived. Therefore, option D is the most aligned with auditing transparency.<br />
<br />
Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: “AI Governance and Risk Management,” Subsection: “Transparency and Explainability”

The AAIA™ Study Guide emphasizes that the foundation of any high-performing and ethical AI system lies in the quality and integrity of its data. For high-risk AI systems, such as those used in healthcare, finance, or criminal justice, it is essential to base models on trustworthy data. This ensures reliable predictions, reduces bias, and mitigates risk. “Trustworthy datasets are characterized by accuracy, completeness, consistency, and ethical sourcing. In high-risk AI applications, ensuring data quality at every stage is crucial to system reliability and compliance.”<br />
While backups, user training, and MFA are important for security and operational resilience, they do<br />
not address the core challenge of ensuring model accuracy and fairness at the development and design phase. Therefore, option C is the most effective practice.<br />
<br />
Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: “AI Fundamentals and Technologies,” Subsection: “Data Governance and Management”

False positives in fraud detection AI systems often stem from poorly optimized hyperparameters. Hyperparameters control aspects of the model’s learning process such as the learning rate, decision thresholds, and complexity penalties. When these parameters are not tuned correctly, the model can become overly sensitive and flag normal behavior as suspicious, leading to customer complaints. “Hyperparameter tuning is essential to balance sensitivity and specificity in AI models. Improper tuning can result in a high rate of false positives or negatives, particularly in systems like fraud detection that require nuanced pattern recognition.” Options A and B relate to data governance but do not directly cause false positives in predictions.<br />
<br />
Option C (compute scale training) may affect model efficiency, not accuracy. Thus, D is the most appropriate answer.<br />
<br />
Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: “AI Operations and Performance,” Subsection: “Model Tuning and Optimization”

AI-related incidents often differ significantly from traditional IT incidents due to their dependence on data, model behavior, and algorithm performance. According to the AAIA™ Study Guide, incident management programs must include capabilities specifically tailored to AI, such as detecting and mitigating model drift and safeguarding against data poisoning or integrity attacks. “AI incident response frameworks must account for issues unique to machine learning, including model drift, adversarial inputs, and data integrity breaches. An effective program incorporates detection, response, and recovery mechanisms for these AI-specific threats.”<br />
While options A and B contribute to improving incident response over time, and option D suggests<br />
best-practice alignment, only option C directly addresses active response capabilities for high-risk, real-time AI vulnerabilities.<br />
<br />
Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: “AI Governance and Risk Management,” Subsection: “Incident and Risk Management in AI Contexts”