GOVERNANCE STRUCUTRES

– MODULE 4
CHAPTER 18 - INTRODUCTION
 INTRODUCTION
Bridging the gap from ethics to practice in Human-Centered AI (HCAI):
High Hopes and High Risks
• Expected benefits of HCAI include better healthcare diagnoses, cybercrime prevention, and
environmental protection.
• Potential risks include misuse of AI (e.g., rogue robots), unfair bias, privacy violations, and threats to
human rights.
• Key idea: HCAI design and evaluation methods are essential to maximize benefits and minimize
risks.
Shift from Traditional AI to HCAI
• Traditional AI focused on mimicking human behavior (e.g., playing games, facial recognition).
• Modern AI focuses more on practical applications in real-world systems.
• HCAI goes further by using AI to support and enhance human performance rather than just replicate it.
 INTRODUCTION
• 3. Applications and Design of HCAI
• HCAI tech includes: user interfaces, apps, web design, communication tools, and social media.
• These tools:
• Boost user confidence (self-efficacy)
• Support creativity
• Clarify human responsibility
• Encourage social participation

• User-centered design and stakeholder engagement are vital.

• User experience testing is just as important as measuring how well the algorithm works.
 INTRODUCTION
• Defining HCAI
• Stanford University defines HCAI as technology that understands and responds to human feelings,
language, and behavior to serve human needs.
• But modern AI (especially deep learning) can make it hard to predict failure points.
• Explainable interfaces and clear control panels help improve understanding and trust.

HCAI Represents a New Approach (Synthesis)

• Old AI focused on machine autonomy.
• HCAI puts human autonomy at the center by:
• Prioritizing user experience
• Focusing on human satisfaction and control

• Businesses are starting to adopt this model, but widespread adoption will take time.
 INTRODUCTION
• Challenges in Moving from Ethics to Practice
• Two main complexities make ethical HCAI hard to implement:
• Complex systems (like self-driving cars or healthcare apps) are hard to fully test and
evaluate.
• HCAI systems are built from many rapidly changing parts (e.g., code libraries, training
data, software tools), making consistent oversight difficult.
• These challenges require:
• Improved engineering methods
• Independent reviews
• Pilot projects and ongoing research
 INTRODUCTION
The ethical guidelines from two major sources: the Berkman Klein Center and the
IEEE’s “Ethically Aligned Design”, both of which provide foundational principles for
responsible and Human-Centered AI (HCAI).

• Ethical Guidelines Are Widespread

• More than 500 reports from companies, professional bodies, governments, NGOs,
and advocacy groups have outlined ethical principles for AI.
• These guidelines show a growing global consensus that AI should serve human
interests, rights, and values—not just technical goals or business profits.
 INTRODUCTION
• Berkman Klein Center Report (2020) – 8 Key Themes
• This report is a comprehensive summary of 36 major ethical frameworks. It identifies eight
recurring themes that many organizations agree on:

Theme What It Means

AI must protect individuals' personal data and prevent misuse or unauthorized
Privacy
access.
Accountability Developers and users of AI systems must take responsibility for outcomes.
Safety and Security AI systems must be safe to use and secure from hacking or unintended behavior.
Transparency and Explainability AI decisions should be understandable by users; systems should not be "black boxes".
Fairness and Non-discrimination AI must treat all users fairly, avoiding bias against any group.
Human Control Humans should retain control over AI decisions, especially in high-risk contexts.
Professional Responsibility Developers must act ethically, respecting the broader social impact of their work.
Promotion of Human Values AI should support dignity, autonomy, equity, and democratic principles.
 INTRODUCTION
• IEEE’s “Ethically Aligned Design” – Expanded Ethical Focus
• The IEEE report—the result of a 3-year project involving 200+ contributors—proposes 8
complementary principles. These overlap with the Berkman Klein themes but go further in some areas

Principle Meaning
Human Rights AI must respect and uphold internationally recognized human rights.
Well-being AI should enhance individual and societal health, happiness, and safety.
Data Agency Individuals should have control over their own data (e.g., consent, usage).
Effectiveness AI systems must actually fulfill their intended purpose, reliably.
Transparency Systems should be open about how decisions are made.
Accountability Like Berkman, IEEE calls for clear responsibility in case of harm.
Awareness of Misuse Designers must consider how their systems could be misused and prevent it.
Competence Only qualified professionals should develop or deploy critical AI systems.
 INTRODUCTION
• Common Ground Between Both Reports
• Both frameworks highlight core principles like:
• Transparency
• Accountability
• Fairness
• Human-centered control

• Their overlap indicates strong global agreement on what ethical AI should look like.
• Why These Principles Matter
• They guide AI developers, policymakers, and organizations to:
• Avoid harm
• Build trust
• Promote equity and justice

• But: Having principles is not enough. The challenge lies in turning these values into real-world policies,
practices, and enforcement mechanisms—which is why governance structures are needed.
 INTRODUCTION
1. The Ethics-Practice Gap
• Despite strong ethical principles in HCAI, applying them in practice remains a challenge.
• Scholars like Alan Winfield and Marina Jirotka emphasize that this gap is a major issue.

2. Four-Layer Governance Structure for HCAI

• This model provides practical guidance to turn ethical intentions into real-world HCAI
systems. It consists of four nested levels:
 INTRODUCTION
• Team Level – Reliable Systems
• This level focuses on the software engineering teams who are directly
responsible for designing, building, and testing AI systems. The goal is to create
technically sound, trustworthy, and safe systems that align with human values.
These practices are the first line of defense against failures, ethical breaches, and
harmful outcomes.
 INTRODUCTION
Below are the key technical practices used by teams to ensure reliability:
1. Audit Trails
•What it is: A digital record that logs every significant action taken by the system or its users.
•Why it matters: Helps trace back what happened and why in case of failure or misuse.
•Example: In a healthcare AI system, audit trails can record who accessed patient data and
what decisions the AI made based on it—essential for accountability and compliance.
2. Workflow Design
•What it is: Designing clear, step-by-step processes for how data flows through the system
and how decisions are made.
•Why it matters: Ensures consistency, avoids hidden logic or shortcuts, and makes the
system more understandable for humans.
•Example: In an AI loan approval system, workflow design maps out how the application is
reviewed, what data is used, and when human review is required.
 INTRODUCTION
3. Bias Testing
• What it is: Techniques used to detect and reduce unfair or discriminatory behavior in AI models.
• Why it matters: Prevents AI from unintentionally harming certain groups, such as minorities or women.
• Example: Before deploying a hiring AI, engineers might test it to ensure it doesn’t favor male resumes over female
ones for the same qualifications.
4. Verification and Validation (V&V)
• What it is:
• Verification ensures the system is built correctly (i.e., “Are we building the system right?”).
• Validation ensures the system does what it’s supposed to do (i.e., “Are we building the right system?”).
• Why it matters: Reduces bugs, improves reliability, and ensures the AI meets user needs and legal
requirements.
• Example: For a self-driving car, verification checks sensor integration, while validation ensures the car can safely
navigate real roads.
 INTRODUCTION
• 5. Explainable User Interfaces (UIs)
• What it is: Interfaces that help users understand how and why the AI made a decision.
• Why it matters: Builds trust, supports better decision-making, and allows users to challenge or
override AI decisions when needed.
• Example: A medical AI tool might show a visual explanation of why it recommended a diagnosis (e.g.,
highlighting parts of an X-ray image).
• By applying these five practices, software engineering teams ensure that HCAI systems are:
• Transparent
• Fair
• Accountable
• Safe for real-world use
• These are the building blocks of reliable and ethical AI at the ground level.
 INTRODUCTION

• Organization Level – Safety Culture

At this level, the focus shifts from individual software engineering teams to the larger
organization (e.g., a tech company or research institution). The goal is to build a
safety-first mindset into the management practices, leadership behaviors, and
workplace culture of the entire organization. This ensures that ethical and safe AI isn’t
just about the code—it’s part of how the organization operates at every level
 INTRODUCTION
• Here are the key components:
1. Leadership Commitment
• What it is: Leaders actively supporting and investing in safety, ethics, and responsible AI development.
• Why it matters: Sets the tone from the top. If leaders prioritize short-term profit over safety, that
attitude spreads.
• Example: A CEO publicly commits to ethical AI practices, funds internal AI ethics teams, and includes
safety goals in company strategy.
 INTRODUCTION
2. Employee Training
• What it is: Ongoing education for employees on topics like:
• AI ethics
• Bias detection
• Regulatory compliance
• Safe software development practices
• Why it matters: Developers, designers, and managers need to understand the risks of HCAI
systems and how to prevent them.
• Example: Regular workshops or mandatory courses on responsible AI and ethical design for
all technical staff.
 INTRODUCTION
3. Reporting of Failures and Near Misses
• What it is: A system for reporting when something goes wrong—or almost goes
wrong—without fear of punishment.
• Why it matters: Encourages a culture of learning, not blame. Helps catch problems
early and avoid disasters.
• Example: An employee notices a pattern of biased outputs in an AI tool. The issue is
reported and addressed before the system goes live.
 INTRODUCTION
4. Internal Reviews
• What it is: Formal reviews within the organization to assess:
• System safety
• Ethical compliance
• Risk management
• Why it matters: Adds a layer of internal oversight and ensures that systems are examined
from multiple perspectives before release.
• Example: An internal review board checks if an AI recommendation system aligns with
company values and legal standards.
 INTRODUCTION
5. Industry Standards
• What it is: Adopting widely accepted practices and benchmarks from professional
organizations or regulatory bodies.
• Why it matters: Ensures the organization isn't just meeting its own standards, but
also aligning with best practices in the field.
• Example: Using ISO standards for AI risk management or following IEEE’s "Ethically
Aligned Design" guidelines.
A strong safety culture ensures that ethical, reliable AI is not just the responsibility of
engineers—it’s embedded across the organization. With the right leadership,
training, and review processes, companies can prevent harm, build trust, and stay
ahead of regulations.
 INTRODUCTION

Industry Level – Trustworthy Certification

• This level emphasizes external accountability through independent, third-party
oversight. While teams and organizations manage internal processes, industry-level
certification adds a layer of public trust by ensuring systems meet broader ethical
and safety standards.
• These certifications and reviews come from independent bodies that are not part of
the organization developing the AI system. Their role is to evaluate, verify, and
certify that AI systems operate safely, fairly, and transparently.
 INTRODUCTION
1. Auditing Firms
• What they do: Conduct formal audits of AI systems, processes, and data handling to check for
compliance with ethical, legal, and technical standards.
• Why it matters: Adds financial and operational accountability; reveals bias, privacy issues, or
performance failures that internal teams might miss.
• Example: An auditing firm evaluates a predictive policing AI tool to ensure it doesn't
disproportionately target certain communities.
2. Insurance Companies
• What they do: Evaluate risk levels and offer insurance coverage for AI system failures, similar to
cyber security insurance.
• Why it matters: Forces companies to identify and minimize risk to qualify for coverage. Insurers
may set safety standards or require third-party audits.
• Example: A company deploying an autonomous vehicle must meet specific safety benchmarks to get
liability insurance from a provider.
 INTRODUCTION
3. NGOs (Non-Governmental Organizations)
• What they do: Advocate for public interests, monitor AI system impacts, and help develop ethical
guidelines and best practices.
• Why it matters: NGOs bring civil society perspectives to AI governance—especially representing
vulnerable or underrepresented groups.
• Example: An NGO examines facial recognition systems for racial bias and provides policy
recommendations to industry stakeholders.
4. Professional Societies
• What they do: Offer ethical codes, industry standards, certifications, and training for AI
practitioners.
• Why it matters: Promotes a shared culture of responsibility among engineers, researchers, and
designers.
• Example: IEEE or ACM providing certification programs and ethical guidelines that AI developers must
follow.
 INTRODUCTION
• Why Trustworthy Certification Is Important
• Ensures AI systems go beyond company claims to meet independent ethical benchmarks.
• Builds public trust by demonstrating external accountability.
• Encourages industry-wide learning, as external reviewers can identify patterns and share best
practices

Industry-level certification brings external credibility and oversight to HCAI. Through auditing,
insurance, NGOs, and professional societies, this layer ensures that AI systems are not just effective—but
also ethical, safe, and accountable to the public.
 INTRODUCTION
• Government Level – Regulation
• This is the outermost layer in the governance framework and represents the broadest form of
oversight. At this level, government agencies and legislation step in to ensure that HCAI systems
align with public values and do not cause harm. These regulations are legally binding—unlike
company policies or industry standards, which are often voluntary.

• Purpose of Government Regulation

• To protect public interest by making sure HCAI systems are:
• Safe (do not cause physical, financial, or social harm)
• Transparent (people understand how decisions are made)
• Reliable (function as intended under various conditions)
• Fair and non-discriminatory
 INTRODUCTION

• Key Functions of Government Regulation

• Set Legal Standards: Define what AI systems must comply with (e.g., privacy laws, data protection
rules, bias testing).
• Enforce Compliance: Through penalties, audits, bans, or other legal action.
• Encourage Innovation: By setting clear rules, companies are motivated to build systems that meet
societal expectations.
 INTRODUCTION
• Example: GDPR in Europe
• GDPR (General Data Protection Regulation) is a landmark privacy law that governs how companies
collect, store, and use personal data.
• Under GDPR:
• Users must be informed about how their data is used.
• Individuals have a right to an explanation when decisions are made by automated systems (this led to growth in
explainable AI).

• Impact: GDPR has pushed companies and researchers to develop more transparent and
user-understandable AI systems, especially in high-risk areas like health and finance.
• Other Examples of Government Regulation
• Automotive safety laws in the U.S. led to better design in self-driving technologies.
• AI Act (EU): A proposed law to regulate AI systems based on their risk level, from minimal to high-risk
applications.
• Federal Trade Commission (U.S.): Has investigated companies for deceptive or discriminatory use of
algorithms.
 INTRODUCTION
• Why Regulation Is Critical
• Voluntary ethics codes aren't enough—some companies may still prioritize profit over
responsibility.
• Regulation levels the playing field, forcing all actors to meet the same safety and fairness
standards.
• It gives the public a voice, especially when market forces or company interests clash with social
values.
Government regulation is essential to ensure AI works for everyone, not just corporations. It
enforces legal accountability and drives innovation in areas like transparency and fairness—helping
society transition to safe and trustworthy HCAI systems.
GOVERNANCE STRUCUTRES
– MODULE 4
CHAPTER 19- RELIABLE SYSTEMS BASED ON SOUND
SOFTWARE ENGINEERING PRACTICES
 WHAT ARE RELIABLE SYSTEMS IN HCAI?

• Reliable HCAI systems are designed to work not just technically well, but also ethically
and socially responsibly. These systems are built to support human needs, protect user
trust, and function safely under real-world conditions.
• Reliable HCAI systems are those that:
• Function safely and as intended
• Respect ethical principles
• Clarify human responsibility
• Are resilient against errors and misuse
• To build such systems, software engineering teams must follow structured technical
practices that ensure accountability, traceability, and continuous improvement.
 WHAT ARE RELIABLE SYSTEMS IN HCAI?
• 1. Function Safely and as Intended
• The system must consistently perform its tasks without causing harm, whether physical (like in
healthcare) or social (like in loan approvals).
• Reliability means predictable and stable behavior under normal and stress conditions.
• Example: A robot assisting in a hospital must follow safety protocols, even if a sensor briefly fails or
data is noisy.
• 2. Respect Ethical Principles
• The system should align with core values like fairness, privacy, transparency, and human dignity.
• It must not discriminate, invade privacy, or take actions that go against users' rights or public interest.
• Example: A hiring algorithm must treat all applicants equally and avoid bias based on race, gender, or
age.
 WHAT ARE RELIABLE SYSTEMS IN HCAI?
3. Clarify Human Responsibility
•The system must record who did what and when, making it easy to trace decisions back to the
responsible team or individual.
•This helps when analyzing failures, preventing blame-shifting, and building trust.
•Example: If a healthcare AI makes a wrong diagnosis, audit logs can show if it was due to a bug, misuse,
or flawed training data.
4. Resilient Against Errors and Misuse
•Systems should be designed to handle unexpected situations, such as hardware malfunctions, incorrect
data, or deliberate attacks.
•They must also have built-in safeguards to detect and recover from problems quickly.
•Example: A self-driving car should recognize when a sensor fails and safely pull over instead of
continuing blindly.
 How Do We Build These Systems?
• To create reliable HCAI systems, software engineers use structured technical practices,
such as:
• Audit trails
• Bias and safety testing
• Explainable user interfaces
• Version control of algorithms and training data
• Continuous monitoring and updates
• These practices ensure accountability (who is responsible), traceability (what
happened and why), and improvement over time.
 Audit Trials and Analysis Tools
• Inspiration from Flight Data Recorders (FDRs)
• Example: FDRs and cockpit voice recorders in aviation help investigate crashes and
improve aircraft safety.
• Lesson for HCAI: We can use "smart black boxes" in AI systems (e.g., robots, autonomous
vehicles) to:
• Understand failures
• Schedule preventive maintenance
• Enhance training and design
• Use of Audit Trails in Different Sectors
• Autonomous Cars: Track driving behavior, rule compliance, and accident causes.
• Healthcare: Log usage of medical devices to detect errors or misuse.
• Stock Market: Analyze algorithmic trading errors or fraud.
• Cybersecurity & Environment: Monitor system behaviors and detect threats or anomalies.
 Audit Trials and Analysis Tools
• 1. Autonomous Cars
• Purpose: Track how the AI system operates in real-world driving.
• Data Logged:
• Steering, braking, and acceleration behavior
• Obstacle detection and decision-making
• GPS location and speed
• Sensor inputs (e.g., LIDAR, cameras)
• Use Cases:
• Accident analysis: Helps determine whether the AI or human was at fault.
• Legal liability: Provides evidence in court or insurance claims.
• Driver training: Data used to improve driving habits and vehicle performance.
• System improvements: Engineers review logs to correct flaws in navigation, object
recognition, or emergency behavior
 Audit Trials and Analysis Tools
• 2. Healthcare
• Purpose: Monitor the use of medical devices and AI diagnostic tools to ensure safe, correct
usage.
• Data Logged:
• Who accessed a system or device
• What diagnosis or dosage was recommended
• Timing of inputs, alerts, or interventions

• Use Cases:
• Error detection: Find out if a device malfunctioned or was misused.
• Accountability: Clear up misunderstandings (e.g., exonerate staff wrongly blamed for a
device error).
• Regulatory compliance: Prove adherence to safety protocols.
• Incident investigation: Understand causes of medical errors or near misses.
 Audit Trials and Analysis Tools
• 3. Stock Market / Financial Trading
• Purpose: Record every step in high-frequency or algorithmic trading to ensure fair and lawful
transactions.
• Data Logged:
• Trade timestamps
• Buy/sell decisions
• Algorithm parameters and version
• Market data at time of trade

• Use Cases:
• Fraud detection: Identify manipulation, insider trading, or system tampering.
• Crash investigation: Understand “flash crashes” caused by faulty or overly aggressive algorithms.
• Client transparency: Allow customers and regulators to see what happened and when.
• Algorithm optimization: Use historical logs to refine trading strategies.
 Audit Trials and Analysis Tools
• 4. Cybersecurity & Environmental Monitoring
• Purpose: Detect anomalies, breaches, or system failures in sensitive systems.
• Data Logged:
• Login attempts and user activity
• System updates and configurations
• Environmental data changes (e.g., air quality, temperature, radiation)

Use Cases in Cybersecurity:

• Threat detection: Spot intrusion attempts or unusual behavior.
• Incident response: Trace the origin and pathway of a cyberattack.
• Compliance audits: Verify that proper protocols and controls were followed.
Use Cases in Environment:
• Pollution monitoring: Log sensor readings to track contamination sources.
• System calibration: Validate that monitoring equipment is functioning properly.
• Policy enforcement: Supply evidence for regulatory action against violators.
 Audit Trials and Analysis Tools
• Challenges in Implementing Audit Trails
• What data to collect?: Forensic analysis needs accurate, relevant records.
• Data volume: Storing video, audio, LIDAR, and logs is expensive and complex.
• Privacy & security: Sensitive data must be encrypted and protected.
• Data ownership: Ongoing debates on who owns audit data (e.g., operators, insurers,
regulators).
• Legal access: Questions about who can use the data in court, research, or the media.
• How to Use Audit Trails Effectively
• Include ML algorithm versions, training data, and code versions at the time of failure.
• Use visualizations and analysis tools to interpret logs efficiently.
• Combine with incident databases to spot patterns and improve safety industry-wide.
 Software Engineering Workflows

• Software engineering workflows are structured steps and methods used to design,
develop, test, deploy, and maintain HCAI systems. In traditional software, the focus is on
code. In HCAI, there's an added emphasis on data, user interaction, trust, and ethical
accountability.
• These workflows must adapt to the complex demands of machine learning, diverse
application areas (like healthcare and cybersecurity), and the need for human-centered
design.
 Key Features of HCAI Software Engineering
Workflows
• New Challenges in AI-Integrated Systems
• Bias Testing: Ensures fairness in algorithms and data.
• Explainable Interfaces: Help users understand and trust system decisions.
• Benchmark Testing: Used to validate AI performance across real-world scenarios.
• Domain-Specific Tuning: Workflows must adapt to sector needs—like privacy in
healthcare or precision in defense.
 Key Features of HCAI Software Engineering
Workflows
• 2. Five Machine Learning Problem Types (by Zhang et al.)
• Workflows must be designed to handle these core ML tasks:

Type Purpose Example

Classification Assign data to categories Diagnosing a disease from symptoms
Regression Predict continuous values Estimating house prices
Clustering Group similar data without labels Segmenting customers by behavior
Dimension Reduction Simplify data for easier processing Visualizing complex biological data
Control Make decisions that maximize rewards Game-playing AI, robotics
 Key Features of HCAI Software Engineering
Workflows
• 3. Typical HCAI Development Process (Multistep Workflow)
The steps in a typical workflow include:
• User requirement gathering
• Data collection, cleaning, and labeling
• Feature engineering – creating useful inputs for models
• Model training and evaluation
• Deployment
• Continuous monitoring – adapts systems to real-world changes
• User experience design and testing – critical for transparency and trust
 Key Features of HCAI Software Engineering
Workflows
• 4. Agile vs. Waterfall Models
• Waterfall Model:
• Linear and phase-based (requirements → design → development → testing →
deployment).
• Easy to manage but rigid and prone to failure if user needs change.
• Agile Model:
• Iterative, flexible, and user-centered.
• Involves frequent testing, user feedback, and updates in short cycles ("sprints").
• Better suited for fast-changing needs and AI-driven systems.
• IBM and Microsoft both recommend agile practices for HCAI, with continuous refinement
and early data exploration.
 Key Features of HCAI Software Engineering
Workflows
• 5. Importance of User Experience (UX) in HCAI Workflows
• HCAI systems should not only work technically, but be understandable and usable by
humans.
• UX testing must ensure that users can:
• Interpret system decisions
• Trust or challenge outcomes (especially in critical decisions like parole or medical treatment)

• This is more than usability—it's about transparency and accountability.

Key Features of HCAI Software Engineering Workflows

• 6. Agile Development Principles for HCAI (Rephrased from Agile Manifesto)

Some key principles that align with HCAI values:
• Deliver software frequently and involve customers continuously.
• Welcome changing requirements—even late in the process.
• Prioritize working software and motivated, trusted teams.
• Build systems with simplicity and continuous improvement.
• Include usability testing and user feedback to ensure human-centered design.
 Key Features of HCAI Software Engineering
Workflows
• 7. Microsoft’s Nine-Stage Workflow for ML Projects
Stages:
• Model Requirements
• Data Collection
• Data Cleaning
• Data Labeling
• Feature Engineering
• Model Training
• Model Evaluation
• Model Deployment
• Model Monitoring
• While it appears sequential (like Waterfall), Microsoft applies agile principles such as frequent user
contact and iterative improvements.
 Verification and Validation Testing
• What Is Verification and Validation Testing in HCAI?
• In HCAI systems, verification and validation (V&V) ensure that the AI works correctly,
fairly, and safely, and that users can trust and understand it. This process is essential to:
• Confirm the system does what it's supposed to do (correct outputs).
• Avoid harmful, biased, or unintended results.
• Make AI systems usable and explainable for end users.
• Strengthen accountability and public trust.
• HCAI systems are more complex than traditional software because they involve machine
learning (ML), human interaction, and real-life consequences, making V&V more
challenging and essential.
 Verification and Validation Testing
• Different Types of AI Require Different Testing
• 1. Supervised Learning
• Learns from labeled training data (e.g., spam email detection).
• Requires V&V to confirm models generalize accurately and fairly across diverse user
groups.
• 2. Unsupervised Learning
• Learns from unlabeled data by finding patterns (e.g., customer segmentation).
• Needs testing for data quality, cluster relevance, and unexpected groupings.
• 3. Reinforcement Learning
• Learns from trial and error in a sequence of actions (e.g., game-playing AI, robotics).
• Testing ensures the system doesn't learn harmful or unintended strategies.
 Verification and Validation Testing
• Key Testing Techniques in HCAI
• 1. Traditional Case-Based Testing
• Uses specific inputs and known expected outputs.
• Helps developers check if the system behaves as expected in normal and edge cases.
• Example: Feed a mortgage application to an AI and confirm it makes the correct decision.
• 2. Differential Testing
• Compares the outputs of two versions of a system (before and after an update).
• Ensures updates haven’t broken anything.
• Limitation: Flawed past results may carry over, and new features might not have a match
for comparison.
 Verification and Validation Testing
• 3. Metamorphic Testing
• Uses logical relationships in outputs to check consistency.
• Example: If a person qualifies for a $100,000 mortgage, they should also qualify for $90,000.
• Great for systems where creating exact expected outputs is hard.
• 4. User Experience (UX) Testing
• Involves real users performing tasks while observed by testers.
• Focuses on:
• Usability
• Clarity of system feedback
• Whether users understand the system enough to trust or challenge decisions

• Example: Testing whether users can understand and respond to AI decisions in parole or hiring
scenarios.
 Verification and Validation Testing
• 5. Red Team Testing
• Simulated attacks by external experts to find weaknesses.
• Inspired by military and cybersecurity methods.
• Helps uncover vulnerabilities like:
• Data poisoning
• Adversarial examples (e.g., tricking a self-driving car with stickers on road signs)
• Tools like MITRE’s ATT&CK matrix provide structured ways to simulate these threats.
 Verification and Validation Testing
• Why Diverse Data and Documentation Matter
• AI performance depends heavily on training data, which must reflect the diversity of
real-world users.
• For example, a cancer detection AI must be trained on data from the specific population it
will serve (age, race, conditions).
• X-ray detection systems showed performance differences across hospitals due to varying
equipment and patient demographics.
 Verification and Validation Testing
• Tools for Documentation
• Datasheets for Datasets (Microsoft): Track who collected and cleaned the data, how it was
used, and its limitations.
• Model Cards (Google): Summarize model performance, fairness, intended use, and ethical
considerations.
Special Testing for Physical and Life-Critical Systems
• Mobile robots, weapons, medical devices need testing for:
• Safe operation
• Task completion rate
• Time, quality, and quantity of tasks

• Mature industries (aviation, automotive, medical) offer strong benchmarks and

certification models.
 Bias Testing to Enhance Fairness
• Why Bias Testing Matters in HCAI
• Bias in AI systems can lead to unfair, discriminatory, and harmful decisions, especially
in high-impact areas like:
• Parole decisions
• Mortgage approvals
• Job interviews
• Healthcare
• Facial recognition
• Bias testing ensures that HCAI systems treat people fairly, uphold ethical standards, and
build public trust.
 Bias Testing to Enhance Fairness
• Why Bias Testing Matters in HCAI
• When AI is used in areas that directly affect people’s lives, the consequences of bias can be severe:
• Bias in AI systems can lead to unfair, discriminatory, and harmful decisions, especially in high-impact
areas like:
• Parole decisions : Biased algorithms may predict a higher risk of reoffending for certain racial groups,
leading to unjust imprisonment or denial of parole.
• Mortgage approvals: If training data reflect past discrimination (e.g., redlining), the AI may unfairly deny
loans to people from marginalized communities.
• Job interviews: AI systems may favor applicants based on traits like name, gender, or voice, excluding equally
qualified candidates due to hidden biases.
• Healthcare: AI may offer lower-quality care recommendations for underrepresented populations if the
training data lacks diversity (e.g., underdiagnosing diseases in Black patients).
• Facial recognition: Systems often have lower accuracy for women and people with darker skin, leading
to false arrests or misidentification.
• Bias testing ensures that HCAI systems treat people fairly, uphold ethical standards, and build public
trust.
 Bias Testing to Enhance Fairness
• How Bias Becomes Dangerous (Cathy O’Neil’s “Weapons of Math Destruction”)
• Cathy O'Neil identified three properties that make biased algorithms especially harmful:
• Opacity – Users can't see or understand how the algorithm makes decisions.
• These algorithms are black boxes: the public, and even decision-makers, can’t see how they work.
• People affected by decisions (e.g., denied a loan or job) can’t question or appeal them because they
don’t understand the logic behind the output.
• Example: A parolee is denied early release based on a risk score, but neither the parole board nor
the individual knows how that score was calculated.
• Scale – These systems affect millions through institutions and corporations.
• These systems are used by governments, schools, banks, police, and employers, often reaching
millions of people.
• One biased algorithm can therefore systematically harm entire communities
• Example: A biased hiring algorithm filters out thousands of job candidates based on gendered
resume patterns.
 Bias Testing to Enhance Fairness
• How Bias Becomes Dangerous (Cathy O’Neil’s “Weapons of Math Destruction”)
• Harm – The outputs can seriously impact people’s lives (e.g., jail time, job denial).
• Biased outputs can lead to serious damage, especially when tied to life-changing decisions like:
• Incarceration or parole
• Employment opportunities
• Access to credit or housing
• Quality of medical care
Example: A healthcare algorithm that underestimates Black patients’ needs leads to fewer services being
provided, worsening health outcomes.
 Bias Testing to Enhance Fairness
• Types of Bias (Friedman & Nissenbaum)
• Pre-existing Bias – Reflects real-world social inequalities (e.g., loan rejection in low-income
areas).
•Comes from social inequalities, stereotypes, or discrimination that already exist in
society.
•These biases get baked into data used to train algorithms.
Example: A loan approval system trained on historical data may continue rejecting applicants
from low-income neighborhoods—even if their current financial status qualifies them—because
it reflects past biased decisions.
• Technical Bias – Introduced by design limitations (e.g., interface lists sorted alphabetically
may disadvantage urgent cases).
•Arises due to limitations or assumptions in how a system is built.
•Happens when software or hardware unintentionally favors certain users over others.
Example: A hospital organ donor list sorted alphabetically, instead of by medical urgency, can
delay care for patients with names later in the alphabet—resulting in unfair treatment.
 Bias Testing to Enhance Fairness
• Types of Bias (Friedman & Nissenbaum)

• Emergent Bias – Arises when technology is used in new contexts (e.g., education apps designed for
high-literacy cultures fail in low-literacy settings).
•Shows up when technology is used in ways or settings the designers didn’t anticipate.
•Happens when cultural, social, or environmental contexts change, but the system stays the same.
Example: An education app designed for students in high-literacy countries might confuse or alienate
users in low-literacy countries, because it assumes familiarity with language, reading habits, or cultural
norms.
 Bias Testing to Enhance Fairness
• Other Sources of Bias
• Geographic, cultural, and language bias (Baeza-Yates): Systems trained on dominant cultural values
often ignore minority worldviews.
(Based on work by Ricardo Baeza-Yates)
•Many AI systems are trained on data from Western, English-speaking, urban populations.
•This leads to algorithms that reflect dominant cultural norms and miss or misrepresent minority
worldviews.
Example: A voice assistant may struggle with non-Western accents or cultural references, making it
less useful or even offensive in some parts of the world.
• Disability bias (Meredith Ringel Morris): AI often overlooks users with physical or cognitive disabilities,
despite its potential to help them.
(Studied by Meredith Ringel Morris)
•AI design often fails to consider users with physical, sensory, or cognitive disabilities.
•This happens despite AI’s great potential to assist these groups.
Example: Image-recognition tools that don’t support screen readers or speech-to-text systems that
ignore speech differences in users with neurological disorders.
 Bias Testing to Enhance Fairness
• Other Sources of Bias

• Healthcare bias: An algorithm spent less on Black patients due to flawed assumptions, missing out on
care for ~30% of them [(Obermeyer et al., 2019)].
• A widely used healthcare algorithm used health costs as a proxy for need.
• Because Black patients historically receive less access to care, the algorithm assumed they were
healthier—when in fact, they needed more help.
Impact: This led to ~30% fewer Black patients receiving the care they needed
 Bias Testing to Enhance Fairness
• How to Do Bias Testing
• Audit the training dataset:
•Check for demographic representation: Make sure the data includes people of different races,
genders, ages, regions, and abilities.
•Update regularly: Populations change over time—so should your data.
•Biased data leads to biased decisions, especially if one group is over- or under-represented.

• USE FAIRNESS-ENHANCING INTERVENTIONS:

• Remove or de-emphasize sensitive features like race or gender when they’re not legally or
ethically justified in decision-making.
• Tools like IBM’s Fairness 360 Toolkit can:
• Detect bias in datasets and models.
• Apply algorithms to correct or reduce bias.
Why it matters: Even well-intentioned models can develop hidden biases during training.
Interventions help keep things in check.
 Bias Testing to Enhance Fairness
• How to Do Bias Testing
• Assign a bias testing leader:
•A dedicated group should oversee fairness audits, test systems across different user profiles, and track
outcomes over time.
•Keep a library of test cases (e.g., different user scenarios) to check whether the AI responds fairly
across demographics.
Without ownership, bias testing gets overlooked. Accountability helps maintain fairness.

• Involve external reviewers:

• Independent monitors help uncover hidden biases that internal teams may ignore or resist.
• Bring in independent experts or auditors to review the system.
• External reviewers often catch problems that internal teams miss or unconsciously ignore.
Why it matters: Objectivity is crucial—especially when decisions can impact health, freedom, or
opportunity.
 Explainable User Interfaces

• Designers of Human-Centered AI (HCAI) systems increasingly recognize the need for

explainable user interfaces, particularly for consequential decisions like loan approvals or
parole rulings. These explanations are not only ethically and socially important but also
increasingly legally mandated, especially under the European Union’s General Data
Protection Regulation (GDPR), which supports a “right to explanation.”
 Explainable User Interfaces
•Legal and Ethical Need for Explanability
GDPR and similar regulations necessitate clear, actionable explanations for algorithmic
decisions. Reports from the UK Information Commissioner’s Office and Alan Turing
Institute emphasize that explainability aids legal compliance, improves user trust, and
enhances governance.
•Practical Recommendations
These reports recommend that explanations should include:
•The reason for the decision
•Accountability (who is responsible)
•How the decision can be contested
•Benefits Beyond Compliance
Researchers like Weld and Bansal argue that explainability:
•Enhances model correctness
•Identifies flawed training data
•Supports dynamic, real-world adaptation
•Increases user control and acceptance
Explainable User Interfaces
 Preventing the Need for Explanations

• Rather than relying solely on post-hoc explanations after a system makes a decision, many
designers have turned to strategies that reduce or eliminate the need for explanations in
the first place. These preventive approaches aim to help users understand systems as they
use them—before confusion arises.
 Preventing the Need for Explanations
• Key Strategies and Insights:
• Step-by-Step, Transparent Processes
• Early expert systems like MYCIN faced challenges with post-hoc
explanations. Researchers like William Clancey advocated for
step-by-step graphical processes to help users understand each stage of
decision-making before a final outcome.
• This approach enables users to track and influence the outcome,
reducing later confusion.
• Prospective Over Retrospective Design
• Business rule-based systems evolved from explaining decisions after the
fact to showing users each step as it happens. This prospective design
promotes real-time understanding and reduces errors.
 Preventing the Need for Explanations
• Key Strategies and Insights:
•User-Control in Educational Interfaces
•Intelligent tutoring systems initially used avatars to explain concepts but shifted toward
giving users control over their learning pace and content focus.
•MOOCs (Massive Open Online Courses) adopted this model by offering immediate, clear
feedback and empowering learners, which minimized the need for extra explanations.

•Proactive Error Prevention in UI Design

•Instead of explaining errors after they occur, systems now aim to prevent errors altogether.
For example:
•Replacing typed dates (e.g., MMDDYYYY) with calendar pickers.
•Guiding users through progressive disclosure, like Amazon’s checkout process or
TurboTax’s step-by-step tax guidance.

•Incremental Guidance and Flexibility

•Progressive UIs lead users through decisions in small, manageable steps, while explaining
terms and allowing them to revisit previous choices. This approach builds confidence and
minimizes misunderstandings.
Prospective Visual Designs for Exploratory User Interfaces

• AI-based designs give users choices to select from before they initiate action, as in
spelling correctors, text message autocompletion, and search query completion.
• University o fColorado professor DanielS zafir and his collaborators for robot
operation showed that previews of the path and goals for intended actions of a
dexterous robot arm resulted in improved task completion and increased
satisfaction.
• Navigation systems adhere to the predictability principle when they apply AI-based
algorithms to find routes based on current traffic data. Users are given two to four
choices of routes with estimated times for driving, biking, walking, and public
transportation, from which they select the one they want. T henth is supertool
provides visual, textual, and speech-generated instructions
Safety Culture through Business
Management Strategies
Safety Culture through Business Management
Strategies
• As modern societies rely on increasingly complex, interconnected, and
technology-driven systems, the potential for catastrophic failures—such as
pandemics, infrastructure collapse, or technological disasters—has grown. To
mitigate these risks, Human-Centered AI (HCAI) emphasizes building
organizations that prepare for failure rather than simply trying to prevent it. This
preparation is accomplished through four major organizational strategies:
• Normal Accident Theory (Charles Perrow)
• High-Reliability Organizations (HROs)
• Resilience Engineering
• Safety Cultures
 Safety Culture through Business Management
Strategies
• Normal Accident Theory (Charles Perrow)
• Core Idea: In highly complex and tightly coupled systems, accidents are inevitable.
• Tight coupling means events happen quickly, with little time to respond, and small failures cascade
rapidly.
• Too much centralization and not enough redundancy (backup staff or systems) leave organizations
vulnerable.
• Example: If three top executives all travel together and perish in an accident, the organization suffers a
severe blow. Perrow suggests spreading risk—e.g., not flying key staff on the same plane.
• Critiques:
• Gary Klein extended the theory to emphasize decision-making under stress.
• Andrew Hopkins criticized the lack of measurable indicators for tight coupling and insufficient redundancy.
• Some believe Perrow is too fatalistic, assuming failure is always bound to occur.
 Safety Culture through Business Management Strategies
• High-Reliability Organizations (HROs)
• Core Idea: Disasters can be avoided through a culture of vigilance and preparedness.
• HROs have a "preoccupation with failure"—constantly studying past errors, near misses, and weak
signals.
• They practice regular simulations and distributed decision-making, allowing fast, adaptive responses.
• Key Features:
• Redundancy (extra staff or backups)
• Frequent training
• Emphasis on individual responsibility at all levels
• Example: Power grid management where real-time decisions and simulations are vital.
• Contrast with Perrow: HRO advocates are more optimistic, believing failures can be prevented with
the right organizational culture.
 Safety Culture through Business Management Strategies
• Resilience Engineering
• Core Idea: Instead of just avoiding failure, organizations must be designed to adapt and recover when
disruptions happen.
• Originating from cognitive science and human factors, resilience engineering highlights the need for
flexibility and adaptive capacity.
• David Woods emphasizes learning from biology and technology to create systems that are robust and
agile.
• Types of threats considered:
• Natural (earthquakes, storms)
• Technological (system outages)
• Adversarial (cyberattacks, terrorism)
• Design-based (bugs, poor planning)

• Example: Hospitals that can reroute resources and personnel during unexpected patient surges.
 Safety Culture through Business Management Strategies
• Safety Cultures
• Core Idea: True safety comes from organizational mindset and values, not just systems or rules.
• Inspired by major disasters like Chernobyl and the Challenger explosion, which weren’t caused by one
error but by systemic cultural issues.
• Emphasis is on:
• Open communication
• Empowered staff
• Long-term commitment to learning and transparency

• Nancy Leveson (MIT) applies systems engineering to develop safe organizational structures.
• She distinguishes between safety (preventing harm) and reliability (system performance)—two separate goals
requiring different strategies.

• Example: NASA changing its internal culture after the Challenger disaster to encourage engineers to
speak up about safety concerns.
 Building on the safety culture approach

• the ways managers can support HCAI:

(1) leadership commitment to safety,
(2) hiring and training oriented to safety,
(3) extensive reporting of failures and near misses,
(4) internal review boards for problems and future plans, and
(5) alignment with industry standards and accepted best practices.
 Leadership Commitment to Safety in HCAI Systems

• In organizations that develop and deploy Human-Centered AI (HCAI) systems—especially those impacting critical
areas like healthcare, public safety, or justice—leadership commitment to safety is not optional. It must be explicit,
proactive, and systemic. This commitment influences the entire safety culture, which is essential to protecting both
users and those affected by AI decisions.
1. Clear Commitment from the Top
• Vision and Values: Executives must articulate safety as a core organizational value through vision statements,
strategic goals, and mission alignment.
• Ongoing Visibility: Leaders must reaffirm this commitment regularly—not just in policies but also through
everyday practices such as hiring criteria, safety-focused training, and transparency around failures or near misses.
• Example: Hospitals that hold monthly safety review board meetings foster accountability and learning. In one Ohio
hospital, such efforts led to measurable improvements in patient safety and reduced harm [(Berry et al., 2013)].
 Leadership Commitment to Safety in HCAI Systems
2. Distinction Between Safety Culture and Climate
• Safety Culture: Refers to enduring organizational values and norms that prioritize safety across all levels.
• Safety Climate: Refers to the current mood or atmosphere—which can change due to internal issues (e.g., staffing
conflicts) or external pressures (e.g., a cyberattack).
• Leadership must monitor and reinforce culture even when the climate is strained.

3. Board and Budget Support

•Board Involvement: Ensures that executives are held accountable for safety performance—not just profitability.
•Resource Allocation: Leaders must provide adequate staffing, diversity, training time, and financial support. This
includes allowing redundancy—extra personnel ready for emergencies.
Although expensive, investing in safety pays off through:
•Lower failure rates
•Reduced injury-related costs
•Operational savings and higher productivity
 Leadership Commitment to Safety in HCAI Systems

4. Safety in HCAI: A User-Centered Challenge

• Traditional safety culture often focuses on employees and facilities, but HCAI systems affect external
users—patients, judges, parolees, citizens.
• User-centered safety culture must include:
• Outreach and two-way communication with affected communities
• Continuous monitoring of usage data
• Easy and accessible mechanisms to report adverse events or concerns
 Leadership Commitment to Safety in HCAI Systems
5. Technical Responsibilities for Leaders
• While safety culture is a management issue, leaders must also understand the technical risks and ensure they are
addressed:
• Choosing the right objective functions for AI systems
• Avoiding distributional shift, where algorithms fail because the real-world context differs from the training data
• Implementing supervisory controls to stop unsafe AI actions
• Preventing adversarial attacks
• Ongoing testing: Leaders must verify that AI models are not only tested during development but also continuously
monitored post-deployment.
6. Skepticism and the Risk of Safety-Washing
• Critics warn that safety declarations can become mere PR, especially in high-risk industries like nuclear energy,
healthcare, and social media.
• Failures are often blamed on operators rather than organizational flaws like poor preparation or inadequate
 Leadership Commitment to Safety in HCAI Systems

7. Mechanisms for Accountability and Reporting

• To strengthen real commitment:
• Appoint an ombudsperson to confidentially handle safety concerns
• Protect whistleblowers who expose genuine threats
• Foster a culture of openness and fairness, where staff and users feel safe to speak up.
• For HCAI systems, safety is a strategic leadership responsibility that combines cultural,
operational, and technical dimensions. Genuine leadership commitment—backed by resources,
clear communication, and robust oversight—builds trust, enhances system integrity, and
reduces the likelihood of harm to people affected by AI-driven decisions.
 Hiring and Training Oriented to Safety in HCAI Systems
• A critical part of building a strong safety culture in organizations—especially those implementing
Human-Centered AI (HCAI) systems—is ensuring that hiring practices and training programs are
aligned with safety goals. This alignment creates a workforce that is prepared, informed, and capable of
responding effectively to both routine tasks and unexpected challenges.
• 1. Hiring for Safety Commitment
• Visible Values: Including safety as a core value in job descriptions and recruitment materials signals to
both current and potential employees that safety is a priority.
• Diversity Matters: Hiring staff with diverse backgrounds in skills, roles, and demographics helps ensure
that various safety concerns are considered and that solutions are inclusive and comprehensive.
• Specialized Expertise: Organizations need experienced professionals in safety-related fields, such as:
• Health and safety
• Human resources
• Organizational design
• Ethnography (to understand user culture and behavior)
• Forensics (to analyze and learn from incidents)
 Hiring and Training Oriented to Safety in HCAI Systems
• 2. Training Focused on Safety and Preparedness
• Routine Emergency Drills: Safety-first organizations conduct regular simulations and emergency
preparedness exercises tailored to their environment:
• Control room operators simulate shutdowns or failures.
• Pilots use flight simulators for emergency scenarios.
• Hospital staff conduct multi-day disaster response exercises (e.g., for pandemics or mass casualty events).

• Integrated Routines: When normal operations mirror emergency procedures, employees are more
likely to react correctly under pressure.
• Emergency Prioritization:
• Organizations rank possible emergencies by frequency and severity.
• They estimate how many responders are needed and plan collaboration with external agencies (e.g., fire,
EMS, cybersecurity teams).
• Checklists are crucial tools—they reduce human error and help staff stay focused during high-stress
situations.
 Hiring and Training Oriented to Safety in HCAI Systems
• 3. Design Training for Technical Roles
• Designers and engineers need training aligned with user interface guidelines and AI-human
interaction principles:
• Apple’s Human Interface Guidelines
• Google’s Material Design
• Microsoft’s 18 Guidelines for Human-AI Interaction
• IBM’s Design for AI

• These frameworks offer best practices for predictability, transparency, and control in HCAI systems,
but need to be actively taught and enforced within organizations.
 Hiring and Training Oriented to Safety in HCAI Systems
4. Broader Organizational Support
• Guidelines alone are not enough—they must be:
• Taught systematically to developers, designers, and managers.
• Supported by organizational structures that enforce compliance, allow justified exceptions, and evolve the
guidelines over time.
• Embedded into policies and project reviews to ensure consistency.
• 5. User Training for HCAI Systems
• As HCAI technologies expand, non-technical users like patients, drivers, and frontline workers must also be
trained:
• Self-driving car users must understand what actions the system can and cannot take.
• Clinicians using electronic health records (EHRs) must know how AI makes suggestions and how to override or
verify them.
• Operators of industrial systems need to be aware of how algorithms could fail and what signals indicate
malfunction.
 Hiring and Training Oriented to Safety in HCAI Systems
5. User Training for HCAI Systems
• As HCAI technologies expand, non-technical users like patients, drivers, and frontline workers must also be
trained:
• Self-driving car users must understand what actions the system can and cannot take.
• Clinicians using electronic health records (EHRs) must know how AI makes suggestions and how to override or
verify them.
• Operators of industrial systems need to be aware of how algorithms could fail and what signals indicate
malfunction.
• These users must gain basic literacy in AI, including:
• Machine learning's strengths and limitations
• Probabilistic predictions
• The meaning of alerts and error signals
• Steps for manual control in emergencies
 Hiring and Training Oriented to Safety in HCAI Systems

6. Training as Strategic Investment

•As noted by Paul R. Daugherty and H. James Wilson in Human + Machine, AI adoption demands massive investment
in workforce training.
“Companies must provide the employee training and retraining required so that people will be prepared and ready
to assume any new roles… Investing in people must be the core part of any company’s AI strategy.”
Without proper training, the most sophisticated AI systems can become dangerous liabilities rather than
productive tools.

A safety culture cannot exist without intentional hiring and continuous, role-specific training. Whether through
structured emergency drills, design guideline education, or user literacy efforts, investing in people is key to safely
managing and scaling HCAI systems.
 Extensive Reporting of Failures and Near Misses
• One of the most powerful tools for creating a safety culture in organizations—especially
those using Human-Centered AI (HCAI) systems—is the systematic, transparent
reporting of failures and near misses. Rather than hiding or minimizing problems,
successful safety-focused organizations encourage detailed reporting, learn from it, and
use the insights to prevent future harm.
• 1. Why Reporting Near Misses Matters
• Near misses, or “close calls,” are incidents that almost lead to harm but are averted in time.
These events are more frequent than actual failures and provide rich data for identifying
patterns of risk.
• Examples: water leaks, equipment restarts, operator slips, or brief power outages.
• If these events are documented, patterns can be analyzed, prompting preventive action
(e.g., redesigns, updated protocols, or training).
 Extensive Reporting of Failures and Near Misses
• 2. Rewarding Reporting, Not Silence
• The US National Safety Council recommends not rewarding managers for having “no reported failures,”
as this can encourage underreporting or cover-ups.
• Instead, reward those who actively report near misses, which indicates a culture of vigilance and
honesty.
• 3. Successful Reporting Models from Aviation and Healthcare
• a. Aviation Industry
• Uses multiple reporting systems:
• FAA Hotlines (public and anonymous)
• The Aviation Safety Reporting System (ASRS), which aggregates, analyzes, and shares safety data
• Investigations by the NTSB that are transparent and influential
 Extensive Reporting of Failures and Near Misses
• b. FDA Adverse Event Reporting Systems
• Invites input from health professionals, manufacturers, and consumers
• Web-based forms walk users through data collection steps
• Tracks millions of reports annually
• MAUDE system tracks robotic surgical device malfunctions, injuries, and deaths, offering critical insights to
improve human-machine interfaces and team training
• 4. Cybersecurity and Software Reporting as Models
• MITRE’s Common Vulnerabilities and Exposures (CVE) and the National Vulnerability Database help
track and resolve security threats.
• Bug tracking tools like Bugzilla and platforms like GitHub allow developers to log, trace, and resolve issues
collaboratively.
• Bug bounty and bias bounty programs (e.g., at Google, HackerOne) pay individuals to report vulnerabilities or
algorithmic biases, encouraging transparency and rapid improvement.
 Extensive Reporting of Failures and Near Misses
• 5. Challenges in HCAI Reporting
• Complexity and opacity in AI systems can make error reporting difficult, especially in highly automated tools like
self-driving cars or industrial AI systems.
• Many systems lack adequate status displays or diagnostic tools that help users pinpoint issues (e.g., unreliable
home internet with unclear causes).

• 6. After-Action Reviews (AARs)

• Originating in the US Army, After-Action Reviews involve structured reflection post-incident:
• What was supposed to happen?
• What actually happened?
• What can be done better?

• Used across domains—healthcare, firefighting, transportation—and a valuable template for analyzing

HCAI failures.
 Extensive Reporting of Failures and Near Misses
• 7. HCAI-Specific Reporting Initiatives
• Early efforts to document HCAI failures include:
• Roman Yampolskiy’s incident list, expanded by the Partnership on AI
• Sean McGregor’s AI Incident Database, with over 1,000 entries searchable by keyword (e.g., “facial
recognition,” “mortgage”)

• While these datasets are promising, comprehensive reports and policy uptake are still lacking
• 8. Case Study: Tesla’s Autopilot Incidents
• Karl Hansen, a former Tesla investigator, compiled reports of over 200 deaths involving Tesla
vehicles—often underreported in public perception.
• Issues include:
• Sudden unintended acceleration (SUA)
• Crashes involving first responders or parked vehicles

• Tesla often attributes failures to driver error, but critics argue that safety-first systems should prevent
these events, regardless of driver input.
 Extensive Reporting of Failures and Near Misses

• To foster safety in HCAI systems, organizations must create a culture that embraces
reporting—not punishes it. Learning from near misses, rewarding transparency, and
building robust public reporting systems (as seen in aviation, medicine, and
cybersecurity) are crucial steps toward accountable, trustworthy, and
user-protective AI systems.
 Internal Review Boards for Problems and Future Plans
• A foundational component of Human-Centered AI (HCAI) safety culture is the establishment of
internal review boards. These bodies help maintain safety and accountability by providing
regular, structured opportunities to review problems, track performance, and plan for future
improvements.
• 1. Role and Purpose of Internal Review Boards
• Monthly meetings serve as forums to:
• Discuss recent failures and near misses
• Analyze patterns and emerging risks
• Celebrate resilience and successful responses to challenges
• These boards provide a continuous improvement loop—evaluating what happened, learning
from it, and adapting practices and systems accordingly.
 Internal Review Boards for Problems and Future Plans
• 2. Standardized Reporting and Metrics
• Data-driven analysis is key. Statistical reports of incidents allow teams to:
• Monitor trends
• Identify problem areas
• Propose new or improved metrics

• Public summaries or transparency reports help promote accountability and trust—internally

and externally.
• 3. Cross-Disciplinary Participation and Benchmarking
• Effective review boards include a diverse mix of stakeholders—managers, frontline staff,
engineers, domain experts, and even external advisors.
• In industries like aviation and healthcare, internal metrics (like on-time flight data or surgery
success rates) are often made public, encouraging transparency and even healthy competition
among organizations.
 Internal Review Boards for Problems and Future Plans
• 4. Disclosure, Apology, and Offer Programs
• In healthcare, a novel approach involves full disclosure to patients and
families when mistakes occur, paired with:
• A clear apology
• Offers of corrective treatment and/or financial compensation
• Impact:
• Fewer lawsuits
• Increased trust and professional integrity
• Lower overall error rates due to higher awareness
• Enhanced organizational morale and pride
 Internal Review Boards for Problems and Future Plans

• 5. Application to HCAI Systems

• HCAI systems—especially those used in sensitive domains like healthcare,
criminal justice, and finance—benefit from internal audits and ethical
reviews.
• These should cover both technical risks and societal impacts.
 Internal Review Boards for Problems and Future Plans
6. Google’s Five-Stage Internal AI Auditing Framework
Google’s internal algorithmic audit framework is a practical model for others building HCAI systems:
1.Scoping
•Define project and audit boundaries
•Identify potential risks
2.Mapping
•Build a stakeholder map and list collaborators
•Interview stakeholders
•Select evaluation metrics
3.Artifact Collection
•Document:
•System design
•Datasets
•Machine learning models used
4.Testing
•Conduct adversarial testing to explore vulnerabilities and edge cases
5.Reflection
•Analyze risks and propose failure remediation strategies
•Archive the design decisions and review process
A post-audit report supports self-assessment and includes mechanisms for tracking whether corrective actions are
implemented.
 Internal Review Boards for Problems and Future Plans

• 7. Corporate Examples of Oversight and Ethical Governance

• Facebook’s Oversight Board: Established to make content moderation decisions and provide
transparency around governance issues on its platform.
• Microsoft’s AETHER Committee (AI and Ethics in Engineering and Research):
• Advises on responsible AI policies, standards, and sensitive use cases.
• Works alongside the Office of Responsible AI, which enforces company-wide rules and helps shape laws and
norms for responsible AI.
 Internal Review Boards for Problems and Future Plans
8. Importance of Institutionalizing Review Processes
• While internal reviews are powerful, they are only one part of a broader quality assurance
ecosystem. They need to be:
• Backed by enforceable governance policies
• Embedded in daily operations
• Supported by leadership commitment
• Informed by user and stakeholder feedback

Internal review boards are vital for maintaining safety, transparency, and accountability in HCAI
systems. When paired with robust metrics, open disclosure, and cross-functional input, these boards
foster a learning culture that adapts, evolves, and earns public trust.
 Alignment with Industry Standard Practices
• These standards help ensure that systems are developed, tested, and maintained according
to proven guidelines that promote safety, fairness, transparency, and
accountability—particularly critical in life-critical applications like healthcare,
autonomous driving, and cybersecurity.
• 1. Industry Standards for Automation and AI Safety
• a. Association for Advancing Automation (AAA)
• Formerly the Robotics Industries Association (RIA).
• Collaborates with ANSI (American National Standards Institute) to create voluntary safety
standards for robotics and automation.
• Promotes innovation, growth, and safety, serving as a model for HCAI domains.
 Alignment with Industry Standard Practices
b. International Standards Organization (ISO)
• Has a Technical Committee on Robotics that issues global safety standards for:
• Industrial robots
• Service robots
• Installation protocols
• Performance benchmarks

• ISO’s work helps standardize safety practices and improve cross-border compatibility.
c. IEEE P7000 Standards
• Address core HCAI concerns: transparency, bias, accountability, and trustworthiness.
• Provide more precise definitions and assessments for ethical and safe AI.
d. Open Community for Ethics in Autonomous and Intelligent Systems (OCEANIS)
• A meta-coordination effort that brings together various global standard-setting bodies to
ensure alignment and reduce duplication.
 Alignment with Industry Standard Practices
• 2. Standards for Accessibility and Inclusivity
• W3C’s Web Content Accessibility Guidelines (WCAG) and the US Access Board’s Section 508 standards help
ensure that HCAI systems are usable by all, including those with disabilities.
• These guidelines are key models for making AI systems universal in design, supporting equity and inclusion.
• 3. Capability Maturity Models (CMMs)
• a. CMMI from the Software Engineering Institute (SEI)
• A process maturity framework with five levels:
• Initial – Reactive and inconsistent
• Managed – Project-level process control
• Defined – Organization-wide process standardization
• Quantitatively Managed – Measured and controlled processes
• Optimizing – Continuous improvement and innovation
• Many U.S. government contracts, especially defense-related, require specific maturity levels for vendors.
 Alignment with Industry Standard Practices
• b. HCAI Capability Maturity Models (Emerging)
• Aim to apply this structure to AI domains like healthcare, transportation, and
cybersecurity.
• Focus on:
• Bias assessment
• Data validation
• User testing
• Audit trails and error analysis
• Continuous performance improvement
 Alignment with Industry Standard Practices
• 4. Trustworthiness Maturity Models (TMMs)
• Evolve the CMM concept to focus on earning and sustaining trust in HCAI systems.
• Levels describe the evolution from:
• Ad-hoc, individual-driven approaches
• To standardized, measured, and audited trust-building practices

• Include checks for:

• Algorithmic fairness
• System explainability
• User experience quality
• Responsiveness to complaints
 Alignment with Industry Standard Practices
• 5. Documentation Standards: Transparency in Development
• To support transparency and accountability, HCAI systems are now expected to be documented as
thoroughly as traditional software:
a. Datasheets for Datasets
• Introduced in 2018 to provide structured descriptions of:
• Dataset origins
• Intended uses and limitations
• Ethical/legal concerns

b. Model Cards (Google)

• Document model behavior across different populations or contexts.
 Alignment with Industry Standard Practices

c. FactSheets (IBM)
• Comprehensive reports about AI systems, covering:
• Purpose and performance
• Safety mechanisms
• Transparency and ethics

• Evaluated through user studies for clarity and effectiveness.

d. Explainability Fact Sheets (Sokol & Flach)
• Focused specifically on how explanations are generated and whether they meet users' needs and
expectations.
 Alignment with Industry Standard Practices
• Criticisms and Considerations
• Skeptics warn of “corporate capture”—companies influencing voluntary standards to weaken
regulations or avoid more expensive compliance.
• However, industry involvement can improve the relevance and adoption of standards, ensuring they
reflect real-world constraints and capabilities.
• Agile development proponents fear that maturity models can lead to bureaucracy, slowing innovation.
Still, many maturity models are evolving to be modular and adaptable, supporting leaner
implementations.
Aligning with industry standards and adopting maturity models signals a serious commitment to
building safe, ethical, and trustworthy HCAI systems. While voluntary standards and models must be
carefully managed to avoid being toothless, they remain essential tools for guiding development, improving
quality, and gaining public trust in high-stakes AI technologies.
Trustworthy Certification by
Independent Oversight
CHAPTER 21
 Independent Oversight in HCAI Governance

Independent oversight represents the third layer of governance in

Human-Centered AI (HCAI) systems—complementing technical engineering
practices and internal organizational safety strategies. It is essential for
promoting trustworthiness, accountability, and public confidence,
especially as HCAI systems increasingly affect consequential, life-critical
domains like healthcare, law, finance, and transportation.
 Independent Oversight in HCAI Governance

• 1. Purpose of Independent Oversight

• The goal is to ensure that organizations building or deploying HCAI systems
are legally, morally, and ethically accountable for the outcomes of those
systems.
• Oversight reviews plans, monitors processes, and analyzes failures to
promote continuous improvement and public transparency.
 Independent Oversight in HCAI Governance

2. Clarifying Responsibility and Liability

•Oversight helps define who is accountable:
•Designers
•Developers
•Operators
•Maintenance teams
•Legal liability in HCAI systems is evolving, but existing laws are largely seen as
sufficient, especially when audit trails are implemented to determine where
responsibility lies.
Example: Facebook was held accountable for allowing discriminatory housing ads
via its AI. Legal action under existing discrimination laws forced them to reform
their algorithms and practices.
 Independent Oversight in HCAI Governance

• 3. Professional Responsibility in Tech

• Fields like medicine, law, and aviation already have strong norms for individual
professional accountability.
• Software development lacks equivalent professional certification and liability
structures, often using "hold harmless" clauses.
• HCAI calls for algorithmic accountability, transparency, and ethical design,
supported by organizations like ACM, IEEE, and AAAI, though enforcement is still
weak.
Three Common Forms of Independent Oversight
• a. Planning Oversight
• Involves pre-implementation reviews of new systems or upgrades.
• Analogous to zoning boards for construction or environmental impact
assessments.
• Ensures stakeholders can provide input before harm occurs.
• May include Algorithmic Impact Assessments (AIAs)—structured tools to
predict and address risks before deployment.
• Effective only if accompanied by follow-up reviews to ensure plans were
carried out.
Three Common Forms of Independent Oversight
• b. Continuous Monitoring
• Involves ongoing inspections and audits (e.g., quarterly, annually).
• Expensive but effective in other domains:
• FDA inspectors in pharmaceutical plants
• Federal Reserve monitoring large banks
• Could be adapted to HCAI systems in parole decisions, mortgage
approvals, or autonomous systems, especially as contextual conditions
evolve (e.g., during COVID-19).
Three Common Forms of Independent Oversight
• c. Retrospective Analysis of Disasters
• Post-incident investigations reveal root causes and system flaws.
• NTSB crash investigations are a gold standard.
• Emerging applications in HCAI:
• FCC reviews of social media
• Audit trails for algorithmic decisions
• Promote accountability and learning from failures.
Accounting Firms Conduct External Audits for HCAI
Systems
• As Human-Centered AI (HCAI) systems become more integrated into high-stakes domains
like healthcare, finance, and governance, there's growing interest in leveraging
independent accounting and auditing firms to evaluate these systems—just as they
already do for financial statements. This approach aims to increase trust, transparency,
and accountability, especially among investors, regulators, and the public.
 Accounting Firms Conduct External Audits for HCAI
Systems
The SEC Model: Financial Audits as a Precedent
•The U.S. Securities and Exchange Commission (SEC) mandates annual
internal and external audits for publicly traded companies.
•These audits follow Generally Accepted Accounting Principles (GAAP) and
are published to enhance investor confidence and deter fraud.
•Despite historic failures (e.g., Enron, WorldCom), the SEC model led to the
Sarbanes–Oxley Act of 2002, which strengthened audit integrity and
accountability.
Parallel for HCAI: Introducing similar audit requirements for corporate HCAI
projects (e.g., fairness reports, usability studies) could standardize reporting
and make performance across companies comparable.
 Accounting Firms Conduct External Audits for HCAI
Systems
Expanding the Role of Independent Auditors
•Independent financial auditors are well-positioned to develop frameworks for
evaluating HCAI systems, focusing on:
•Fairness
•Privacy
•Bias
•User experience
•Transparency
•They already have established relationships with internal audit committees,
making their recommendations more likely to be implemented.
These firms could issue public HCAI audit reports that assure investors of a
company's commitment to ethical and safe AI deployment.
Accounting Firms Conduct External Audits for HCAI Systems
The Big Four Accounting Firms and HCAI
•PricewaterhouseCoopers (PwC)
•Deloitte
•Ernst & Young (EY)
•KPMG
All claim expertise in AI and already consult on AI ethics, governance, and risk
management. Deloitte, for instance, highlights the importance of pairing AI
tools with human-centered design—aligning closely with HCAI principles.
Separation of Duties: Under Sarbanes–Oxley, consulting and auditing functions
must be kept separate to avoid conflicts of interest.
Accounting Firms Conduct External Audits for HCAI Systems
Use Case Example: COVID-19 Contact Tracing
•Apple and Google’s COVID-19 contact tracing apps prompted privacy and
surveillance concerns.
•Public pressure led to calls for independent oversight and detailed
governance frameworks.
•One proposal included over 200 criteria for oversight, showing how thorough
audit protocols can be designed for AI systems handling sensitive data.
This case illustrates how public trust can be reinforced by independent
boards or audits in controversial or high-risk projects.
Accounting Firms Conduct External Audits for HCAI
Systems
COSO: A Model for HCAI Auditing
•The Committee of Sponsoring Organizations (COSO)—a coalition of top
accounting associations—developed frameworks for:
•Enterprise risk management
•Internal controls
•Fraud prevention
A similar HCAI audit consortium could help guide responsible AI development
and potentially reduce the need for heavy government regulation by
promoting industry self-governance.
 Accounting Firms Conduct External Audits for HCAI Systems
• Role of Consulting Firms
• Firms like Accenture, McKinsey, and Boston Consulting Group are increasingly advising
companies on:
• Responsible AI practices
• Bias mitigation
• Trust-building strategies

• They publish white papers and best-practice guides to help organizations implement safe
and effective HCAI systems, even if they do not perform independent audits.
• Just as financial audits foster investor trust and regulatory compliance, external HCAI audits
by trusted accounting and consulting firms could strengthen ethical standards, reduce bias,
and enhance transparency in AI systems. If these efforts gain traction, they could preempt
stricter government regulation and position audited companies as leaders in responsible AI.
 The Role of Insurance Companies in Compensating
for Failures in HCAI Systems
• Insurance as a Guarantor of Trust and Safety
• Insurance companies can drive safety standards by requiring that HCAI systems meet specific
criteria to be eligible for coverage.
• This mirrors how insurers enforce building codes for structures or safety requirements for
medical devices.
• Carl Landwehr’s proposal for “a building code for building code” suggests that just as engineers
must meet fire and electrical codes, software engineers should meet AI development
standards—especially in domains like avionics, cybersecurity, and healthcare.
 The Role of Insurance Companies in Compensating
for Failures in HCAI Systems
Enabling Risk Assessment and Coverage
To insure HCAI systems effectively, insurers would require:
•Detailed documentation of software design, testing, and certification.
•Audit trails to understand and analyze system behavior and failures.
•Regular reporting on failures and near misses to develop actuarial data.
Insurers could then create risk profiles for different applications, industries,
and system types—laying the groundwork for pricing, underwriting, and
compensation structures.
 The Role of Insurance Companies in Compensating
for Failures in HCAI Systems
Driving Standards Through Data Collection
•Insurance companies can aggregate industry-wide data on failures and near
misses to:
•Develop risk metrics
•Educate developers
•Influence the creation of HCAI-specific codes and best practices
As more HCAI applications are deployed, this growing pool of incident data will
improve insurers’ ability to model risk and ensure accountability.
 The Role of Insurance Companies in Compensating for
Failures in HCAI Systems

Models for Who Pays and Who Benefits

•In industries like healthcare and transportation, it is common for providers
(e.g., hospitals, airlines) to purchase insurance that protects consumers.
•A similar model is logical for HCAI:
•AI system providers or operators would purchase insurance.
•Consumers who suffer harm—due to algorithmic bias, misdiagnosis, or
wrongful denial—would be eligible for compensation.
This would increase product costs, but history shows that consumers often
accept such costs for greater peace of mind and protection.
 The Role of Insurance Companies in Compensating for
Failures in HCAI Systems

Case Study: Insuring Autonomous Vehicles

•In 2018, Travelers Insurance released a paper titled “Insuring Autonomy”:
•It called for a framework to encourage innovation while also protecting
drivers and consumers.
•It predicted that claims may increase due to the expensive technology in
autonomous vehicles, even as overall accidents decline.
•The report highlighted the need for:
•Clear safety data
•Manufacturer transparency
•Regulatory standards
To date, many car manufacturers have resisted public data sharing, and
federal efforts to regulate self-driving cars remain limited.
 The Role of Insurance Companies in Compensating for
Failures in HCAI Systems

• Challenges and Skepticism

• Critics worry that:
• Insurers may prioritize profit over public safety.
• Claimants could struggle to prove fault—especially with opaque AI systems involved in
healthcare decisions, hiring, or parole assessments.

• However, having an insurance system, even with limitations, is often better

than leaving victims with no recourse.
 The Role of Insurance Companies in Compensating for
Failures in HCAI Systems
Alternative Models: No-Fault Compensation and Victim Funds
To reduce litigation and improve accessibility, governments or industries could establish
no-fault compensation programs, such as:
•9/11 Victim Compensation Fund (terrorism)
•Gulf Coast Claims Facility (Deepwater Horizon oil spill)
These programs:
•Pay injured parties promptly
•Avoid the cost and complexity of lawsuits
•Use independent oversight boards to determine eligibility and payouts
While proposals exist to replicate this model for HCAI failures, no large-scale initiative has yet
been implemented.
 Non-governmental and Civil Society Organizations

• In addition to government regulation, corporate audits, and

insurance-based accountability, non-governmental organizations
(NGOs) and civil society groups play a vital role in shaping the ethical,
safe, and transparent development of Human-Centered AI (HCAI)
systems. These groups act as watchdogs, educators, and innovators,
pushing for reform, transparency, and fairness where official oversight may
fall short.
 Non-governmental and Civil Society Organizations
Independent and Agile Advocacy
•NGOs are often funded by:
•Wealthy individuals
•Private foundations
•Ethical-minded corporations
•Unlike government or corporate institutions, NGOs have greater freedom to:
•Explore novel ideas
•Launch public campaigns
•Conduct early investigations into ethical, legal, or social issues
Their independence allows them to raise red flags quickly and stimulate
public dialogue around the risks and misuse of AI technologies.
 Non-governmental and Civil Society Organizations
Independent and Agile Advocacy
•NGOs are often funded by:
•Wealthy individuals
•Private foundations
•Ethical-minded corporations
•Unlike government or corporate institutions, NGOs have greater freedom to:
•Explore novel ideas
•Launch public campaigns
•Conduct early investigations into ethical, legal, or social issues
Their independence allows them to raise red flags quickly and stimulate
public dialogue around the risks and misuse of AI technologies.
 Non-governmental and Civil Society Organizations
Successful Example: The Algorithmic Justice League
•This civil society organization exposed racial and gender bias in facial
recognition software.
•Their activism pushed major tech companies to:
•Improve their algorithms
•Pause or end sales to law enforcement agencies
•Their work contributed to a broader public reckoning on algorithmic bias,
especially during the 2020 movement to limit police abuses.
This shows NGOs can achieve real policy and product changes by applying
public pressure backed by credible research.
 Non-governmental and Civil Society Organizations
NGOs as Leaders in Ethics and Principles
•NGOs were among the first to frame AI development in terms of:
•Fairness
•Transparency
•Human rights
•Now, as HCAI systems become more entrenched, NGOs are shifting toward:
•Influencing software engineering practices
•Advising on business management strategies
•Shaping governance and oversight mechanisms
To be effective, NGOs must collaborate more actively with policymakers, lawyers, insurers, and
auditing firms.
 Non-governmental and Civil Society Organizations
• Limitations of NGOs

• NGOs have no formal authority to enforce rules or penalize violations.

• Their impact is indirect, based on:

• Public persuasion

• Media coverage

• Informing regulatory agendas

• They play a catalytic role, encouraging:

• Governments to set new rules

• Auditors to revise evaluation methods

• Insurers to adapt risk models for AI technologies

 Non-governmental and Civil Society Organizations

New Opportunities: NGO-led Oversight and Reporting

•NGOs can extend their influence by:
•Conducting independent audits or assessments of high-impact HCAI systems
•Publishing transparency reports
•Creating scorecards for ethical compliance across industries
•These reports could provide:
•Fresh insights into industry-specific challenges
•Frameworks for accountability tuned to different sectors (e.g., finance vs. health)
Well-researched NGO reports can inform regulation, pressure corporations, and educate the
public on HCAI risks and best practices.
 Non-governmental and Civil Society Organizations

Non-governmental and civil society organizations are crucial players in the ecosystem of HCAI
governance. While they lack enforcement power, they lead in setting ethical standards, raising
awareness, and influencing policy. As the HCAI field matures, NGOs will be essential in bridging
technical, legal, and social dimensions of responsible AI—and ensuring that public interest
remains at the center of AI innovation.
Professional Organizations and Research Institutes

• The Role of Professional Organizations and Research Institutes in

Advancing Responsible HCAI Systems
• Professional organizations and academic research institutions are key players
in shaping the ethical, technical, and governance foundations of
Human-Centered AI (HCAI). They contribute by developing standards,
producing research, training future professionals, and influencing public
policy, but their influence is not without challenges and limitations.
 Professional Organizations and Research Institutes
Professional Organizations: Setting Guidelines and Standards
Professional societies like IEEE and ACM play a central role in creating voluntary frameworks that guide the
ethical development and deployment of AI systems.
a. IEEE Initiatives
•The IEEE P7000 series addresses:
•Transparency in autonomous systems
•Algorithmic bias
•Fail-safe design
•Trustworthiness of information sources
•IEEE’s Ethics Certification Program for Autonomous and Intelligent Systems is working to develop:
•Measurable metrics
•Certification procedures for bias, accountability, and transparency
 Professional Organizations and Research Institutes
b. ACM’s US Technology Policy Committee
•Focus areas include:
•Accessibility
•AI accountability
•Digital governance
•Privacy protection
Challenge: These organizations often face low engagement from their members
and criticism of corporate capture, where powerful tech companies influence
standards to suit their business interests rather than the public good.
 Professional Organizations and Research Institutes
Academic Research Institutes: Ethical and Technical Innovation
Many universities have launched dedicated HCAI research centers, combining AI
development with human-centered ethics, governance, and societal impacts.
Examples of Leading Institutions:
•Stanford HAI (Human-Centered AI Institute)
•UC Berkeley’s Center for Human-Compatible AI
•Oxford’s Future of Humanity Institute
•Harvard’s Berkman Klein Center
•NYU’s Center for Responsible AI
•University of Cambridge’s Leverhulme Centre
These centers focus on research topics like AI ethics, safety, policy, and
long-term impacts, helping bridge academic theory with practical
implementation.
 Professional Organizations and Research Institutes
Bridging Research and Practice
•The challenge for academic institutions is not just generating research, but
translating it into usable frameworks, engineering practices, and oversight
systems.
•Strong university–industry–government partnerships can help scale
innovations and influence regulation.
These partnerships can drive interdisciplinary solutions that combine technical
robustness, ethical foresight, and social accountability.
 Professional Organizations and Research Institutes
Corporate Commitments: Aspirations vs. Accountability
a. Microsoft’s Six Principles (Satya Nadella)
•Human assistance, transparency, dignity, privacy, accountability, and fairness
b. Google’s Seven Objectives (Sundar Pichai)
•Social benefit, fairness, safety, accountability, privacy, scientific excellence, and
responsible use
Criticism: These principles, while promising, are often seen as corporate PR rather
than enforceable commitments. For example, Google’s attempt to form an AI
ethics board in 2019 failed within a week due to controversy over member
selections.
 Professional Organizations and Research Institutes
Need for External Oversight
While corporate statements can influence internal company culture and public
expectations, they must not replace external review. External oversight ensures:
•Accountability beyond profit motives
•Transparency in processes
•Credibility in risk mitigation
The public interest is best served when independent experts, journalists, and
civil society groups hold companies and institutions accountable.
 Professional Organizations and Research Institutes

• Professional societies and research institutions are critical engines for ethical AI
development, offering the knowledge, standards, and frameworks needed to
guide HCAI forward. However, their effectiveness hinges on:
• Inclusive participation
• Balanced governance (free from corporate dominance)
• Bridging theory with real-world practices
While corporate ethics statements can spark progress, they must be backed by
meaningful external oversight to ensure that HCAI systems are safe, fair, and
accountable for everyone.