
What you will read
These are the questions about AI governance that come up most on Google and in AI chats. Each answer starts short, shows what the research says, and ends with what I think.
- What is AI governance?
- What is responsible AI?
- Can a company blame the AI for a mistake?
- What is explainable AI?
- What is the AI black box?
- How do you evaluate an AI system before using it?
- What is the NIST AI Risk Management Framework?
- What is ISO/IEC 42001?
- Who should oversee AI in a company?
- How do you keep records of AI decisions?
01
What is AI governance?
AI governance is the set of rules, responsibilities and practices for handling the risks of artificial intelligence. It includes deciding who is accountable for decisions, documenting what is done and testing systems in conditions similar to real use.
Organizations that provide or use AI already have management standards and international principles to guide this work. These address not just how the technology works, but also accountability, transparency, privacy and respect for human rights.
- NIST’s voluntary guide defines governance as the policies, roles, documentation and culture for handling AI risks, assigns responsibility to executive leadership and calls for testing by experts who did not build the systems.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 · National Institute of Standards and Technology (NIST), NIST, 2023
- ISO/IEC 42001 sets requirements for establishing, maintaining and continually improving AI management in organizations of any size that provide or use the technology.ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system · ISO/IEC JTC 1/SC 42, International Organization for Standardization (catalogue entry at NSAI, Ireland's national standards body), 2023
- The OECD AI Principles, updated in May 2024, call for respect for human rights, transparency, safety and accountability.OECD AI Principles · OECD, OECD.AI, 2024
What I think: I would start each new use of AI by defining who is accountable for decisions and testing the system in conditions similar to real use before promising results.
02
What is responsible AI?
Responsible AI means developing and using artificial intelligence with testing, protection for people's rights, and clear responsibility for decisions and risks. It includes protecting privacy, controlling harmful bias, explaining how systems work, and keeping human oversight.
The OECD AI Principles, updated in May 2024 and endorsed by Brazil, call for respect for human rights, transparency, and accountability. NIST's guide turns these concerns into tasks such as assigning responsibility, documenting decisions, and testing systems in conditions close to real use.
- NIST calls for executive leadership to take responsibility for AI risks and for experts who did not build the system to participate in testing under conditions similar to real use.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 · National Institute of Standards and Technology (NIST), NIST, 2023
- The OECD AI Principles call for respect for human rights, fairness, privacy, transparency, safety, and accountability.OECD AI Principles · OECD, OECD.AI, 2024
- UNESCO's recommendation places human dignity and human oversight among the foundations of AI and calls for systems that can be audited and traced.Recommendation on the Ethics of Artificial Intelligence · UNESCO, UNESCO, 2021
What I think: I would expand AI use only after testing its performance and risks in conditions close to real use, assigning responsibility, and establishing protections for people's dignity.
03
Can a company blame the AI for a mistake?
Blaming AI does not guarantee escaping responsibility: a Canadian tribunal held Air Canada responsible for incorrect information from the chatbot on its website.

The issue also extends to software manufacturers. According to the European Commission, the new product liability directive includes AI systems and applies to products placed on the market after 8 December 2026.
- In Moffatt v. Air Canada, the tribunal rejected the idea that the chatbot was responsible for its own actions and ruled that the company is responsible for information on its website.Moffatt v. Air Canada, 2024 BCCRT 149 · Civil Resolution Tribunal of British Columbia (tribunal member Christopher C. Rivers), Civil Resolution Tribunal, British Columbia, Canada, 2024
- CBC News reported that Air Canada was ordered to compensate a passenger who bought full-fare tickets after the chatbot gave incorrect advice about a bereavement discount.Air Canada found liable for chatbot's bad advice on plane tickets · Jason Proctor, CBC News, 2024
- The European Commission says the new directive allows manufacturers to be held liable for software defects, including those that arise after release through updates or machine learning.Liability for defective products (Directive (EU) 2024/2853) · European Commission, European Commission, Single Market and Economy, 2024
What I think: I recommend preparing AI customer service for these demands: test before making promises and ensure the system admits when it does not know and calls a person.
04
What is explainable AI?
Explainable AI is a set of methods that lets people understand and oversee how an AI reaches its decisions. An explanation should be understandable and reflect how the system actually produced its result, not merely sound convincing.
The challenge also applies to language models: Anthropic researchers report that even their developers do not understand how they do most of what they do. Their early analysis tools sometimes caught the model giving a plausible-sounding argument to agree with the user instead of following logical steps.
- Wikipedia defines explainable AI as a research field aimed at letting people understand and oversee the reasoning behind an AI's decisions.Explainable artificial intelligence · Wikipedia contributors, Wikipedia, 2026
- A NIST report proposes that AI give reasons for its outputs, provide understandable explanations that reflect its actual workings, and operate only within its intended conditions and confidence limits.Four Principles of Explainable Artificial Intelligence (NISTIR 8312) · P. Jonathon Phillips et al., NIST, 2021
- Computer scientist Cynthia Rudin argues that high-stakes decisions, such as those in healthcare and criminal justice, should use models that are understandable by design rather than trying to explain a black box afterward.Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead · Cynthia Rudin, Nature Machine Intelligence 1, 206-215, 2019
- Anthropic researchers report that their tools, still able to capture only part of the model's computation, identified cases where it produced plausible arguments to agree with the user.Tracing the thoughts of a large language model · Anthropic Interpretability team, Anthropic, 2025
What I think: In my view, as tools to examine these models emerge, the way forward is to test whether their explanations match their actual workings and require clear limits and human oversight, without confusing a convincing justification with understanding.
05
What is the AI black box?
The AI “black box” is the difficulty of understanding why a system reached a decision, even for the people who built it. A convincing explanation from the AI itself may not show how it actually reached its result.
This matters when AI influences decisions about jobs, public benefits, healthcare or criminal justice. According to the European Commission, being unable to discover the reason for a decision can make it harder to identify whether someone was unfairly disadvantaged.
- The encyclopedia defines explainable AI as methods that help people understand and oversee AI decisions, in contrast to systems whose decisions even their designers cannot explain.Explainable artificial intelligence · Wikipedia contributors, Wikipedia, 2026
- Computer scientist Cynthia Rudin argues that, for high-stakes decisions, black-box models should be replaced with models designed to be understandable from the start.Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead · Cynthia Rudin, Nature Machine Intelligence 1, 206-215, 2019
- Anthropic researchers report that their still-limited tools caught the model offering a plausible argument to agree with the user instead of following logical steps.Tracing the thoughts of a large language model · Anthropic Interpretability team, Anthropic, 2025
- The European Commission warns that an unexplained AI decision can make it harder to identify unfair treatment in hiring or applications for public benefits.AI Act · European Commission, Shaping Europe's digital future, European Commission, 2026
What I think: I recommend measuring errors before expanding AI use and requiring sources, clear limits on what it knows, and human review.
06
How do you evaluate an AI system before using it?
Evaluating an AI system requires testing it under conditions similar to real use. Tests should measure accuracy, risks and whether it admits not knowing.
Evaluation is already required for some public-sector uses: an April 2025 White House memo requires tests and a documented impact assessment before US federal agencies deploy high-impact AI.
- NIST recommends testing under conditions similar to real use, involving experts who did not build the system and documenting responsibilities.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 · National Institute of Standards and Technology (NIST), NIST, 2023
- A Stanford study compared 30 models in the same scenarios, measuring not just accuracy but also qualities such as fairness, bias and efficiency.Holistic Evaluation of Language Models · Percy Liang, Rishi Bommasani, Tony Lee et al., Transactions on Machine Learning Research (TMLR), 2023
- A study published in Nature argues that evaluations can reward made-up answers when guessing earns more points than admitting uncertainty.Evaluating large language models for accuracy incentivizes hallucinations · Adam Tauman Kalai et al., Nature 653, 1047–1051, 2026
What I think: I would expand an AI system's use only after measuring its errors and checking that it admits not knowing, rather than rewarding answers at any cost.
07
What is the NIST AI Risk Management Framework?
The NIST AI Risk Management Framework is a voluntary guide from the National Institute of Standards and Technology for organizing how organizations handle artificial intelligence risks. It divides that work into four functions: govern, map, measure and manage.
According to NIST, version 1.0 is being revised, and the guide gained a supplement specifically for generative AI in July 2024.
- The guide calls for documented responsibilities, executive accountability for risk decisions and testing in conditions similar to real use, with experts who did not build the system.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 · National Institute of Standards and Technology (NIST), NIST, 2023
- NIST's page reports that the AI RMF gained a generative AI profile and that version 1.0 is being revised.AI Risk Management Framework · National Institute of Standards and Technology (NIST), NIST, Information Technology Laboratory, 2026
What I think: I would follow NIST's updates and measure AI results and risks in conditions close to real use before promising results.
08
What is ISO/IEC 42001?
ISO/IEC 42001 is a standard that sets requirements for establishing, maintaining and continually improving how an organization manages artificial intelligence. It applies to organizations of any size or type that provide or use AI, to support responsible development and use.
For organizations seeking to manage AI, the standard sits alongside other related guidance: NIST’s AI resource center lists a mapping by Microsoft between its AI risk management framework and ISO/IEC 42001.
- The standard’s catalogue entry says ISO/IEC 42001 sets requirements for establishing, implementing, maintaining and continually improving an AI management system.ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system · ISO/IEC JTC 1/SC 42, International Organization for Standardization (catalogue entry at NSAI, Ireland's national standards body), 2023
- NIST’s AI resource center lists a mapping by Microsoft between the AI RMF and ISO/IEC 42001.AI RMF Crosswalks · NIST Trustworthy and Responsible AI Resource Center, NIST AI Resource Center, 2025
What I think: I would use ISO/IEC 42001 to organize AI management and measure results with each improvement before promising benefits.
09
Who should oversee AI in a company?
Executive leadership should take responsibility for AI risk decisions, according to NIST. For high-risk systems, the EU AI Act requires oversight to be assigned to people with the necessary competence, training and authority.
Having a person watch over AI does not guarantee effective oversight. A study of government algorithm oversight policies found a false sense of security when people could not perform the review expected of them.
- NIST recommends documented responsibilities, executive accountability for risk decisions, and testing under conditions similar to real use with experts who did not build the system.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 · National Institute of Standards and Technology (NIST), NIST, 2023
- Article 26 of the EU AI Act requires organizations using high-risk AI to appoint supervisors with competence, training and authority, and to monitor the system.Regulation (EU) 2024/1689 (AI Act), Article 26: Obligations of deployers of high-risk AI systems · European Union, AI Act Service Desk, European Commission (consolidated text as at 27 July 2026), 2024
- In a hiring study, researchers observed that participants followed AI recommendations even when they rated the tool poorly.No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy · Kyra Wilson et al., AIES 2025, 2025
What I think: I would expand AI use only after assigning responsibility for its risks and testing the system with experts who did not build it.
10
How do you keep records of AI decisions?
To keep records of AI decisions, use automatic logs of system events and maintain documentation of intended uses and evaluated performance. Pair those records with oversight by trained people who have the authority to act.
The European Commission notes that it is often impossible to find out why an AI made a decision, making unfair treatment in hiring or public benefit applications harder to identify. The EU AI Act provides for activity logging and human oversight for high-risk systems.
- Article 12 of the EU AI Act requires high-risk systems to log events automatically throughout their lifetime so that risks can be identified and their operation monitored.Regulation (EU) 2024/1689 (AI Act), Article 12: Record-keeping · European Union, AI Act Service Desk, European Commission (consolidated text as at 27 July 2026), 2024
- Article 26 requires organizations using high-risk AI to keep its automatic logs for at least six months and assign oversight to people with the necessary competence, training and authority.Regulation (EU) 2024/1689 (AI Act), Article 26: Obligations of deployers of high-risk AI systems · European Union, AI Act Service Desk, European Commission (consolidated text as at 27 July 2026), 2024
- Researchers proposed “model cards,” short documents describing a model’s intended uses and evaluated performance across conditions and groups, rather than recording individual decisions.Model Cards for Model Reporting · Margaret Mitchell et al., FAT* 2019, 2019
What I think: I would only expand AI use with records, performance evaluation and human oversight with the authority to act.
Who writes this
I’m Vitor Araujo, an engineer. I build AI products for companies, and I speak and write about AI ethics and the philosophy of technology. More about me.
Other guides









