HMN 2026: How AI is making life-changing decisions—researchers say we need a better way to keep it fair

Every day, algorithms make consequential decisions about millions of people’s lives—who gets approved for a mortgage, who is called back for a job interview, who receives priority care in a hospital emergency room. Most people have no idea these decisions are being made by software, let alone that the software might be treating them unfairly.

A growing body of evidence suggests this is a serious problem. Organ-transplant algorithms have been shown to encode racial disparities. Mortgage-lending software has produced discriminatory outcomes. Hiring tools have been scrapped after demonstrating bias against women. And these are just the cases that made headlines.

Now, a team of researchers from Georgia State University, Penn State, and the University of Georgia has published a sweeping new design theory aimed at giving organizations a practical roadmap for tackling the problem. Their theory, called FAIR (Fairness Adaptation through AI-augmented Responsiveness), argues that the standard response to AI bias has been fundamentally misguided.

“Most organizations treat AI fairness as a one-time fix,” said Arun Rai, director of the Center for Digital Innovation and professor of computer information systems at Georgia State’s Robinson College of Business, and lead author of the paper published in MIS Quarterly.

“They discover a problem, patch it, and move on. But the tensions that cause unfair AI are persistent, interconnected, and constantly evolving. Making an AI system fairer in one sense can make it less fair in another or bring it into conflict with an organizational goal. It’s complex. You can’t solve them once and walk away. It must be consistently managed.”

Why ‘fair’ is so hard to define

Part of what makes AI fairness so difficult is that “fair” means different things to different people—and those meanings often conflict with each other in ways that have no clean solution.

Consider a hospital emergency department. A physician might want an AI triage system that applies the same criteria to every patient equally. A patient advocate might want the system to give extra weight to patients from underserved communities who historically receive worse care. A hospital administrator needs the system to maximize the number of patients treated. A regulator demands that its decision-making be transparent and non-discriminatory.

These goals are all legitimate and they frequently pull in opposite directions. The researchers synthesized four widely accepted, diverse perspectives on fairness and found that each perspective makes different assumptions and reveals a different set of tensions baked into AI design.

Overall, the tensions arise from conflicts among different fairness principles, other organizational goals, and the needs of individuals versus the collective. For example, making a system more equitable for disadvantaged groups can reduce its overall efficiency. Maximizing transparency can conflict with protecting patient privacy. Standardizing decisions for consistency can make the system less responsive to individual circumstances.

The researchers call this tangle of ongoing competing fairness demands a “sociotechnical paradox“—a set of contradictions that cannot be permanently resolved, only continuously managed. As laws evolve, social expectations shift, and new data flows in, the tensions keep changing shape. A decision that looks fair today may seem discriminatory next year.

The proposed solution: Treat fairness like safety

Like safety, fairness is not a problem that organizations ever fully “solve.” Safety standards evolve as technologies change, risks shift, and new failures emerge—and managing safety means continually balancing protection, performance, and efficiency, often among stakeholders who define “safety” differently. FAIR argues that AI fairness works the same way: it involves persistent, competing demands over what fairness should mean and that cannot be eliminated, only monitored and managed over time.

The FAIR theory proposes two interlocking mechanisms to help organizations develop the capability to manage these tensions proactively on an ongoing basis.

The first operates at the level of individual AI systems that automate decisions. Rather than treating an AI decision-making system as a single black box, FAIR breaks it into an AI pipeline—the input data (which may harbor historical biases), the model that learns from that data to make predictions, and the policy layer that determines whether and how the final decision diverges from those predictions. Fairness tensions are monitored and resolved at each layer of the AI pipeline through a continuous adaptive cycle of “surfacing” (finding problems) and “resolving” (fixing them).

Crucially, this process depends on teamwork between AI agents and humans. AI agents can scan data and flag patterns of unfairness across the AI pipeline at a speed and scale no human team could match. But human experts—clinicians, ethicists, compliance officers—are needed to interpret what those flags mean, make judgment calls, and give final approval for any changes. Neither alone is sufficient, the researchers contend. An AI agent left to fix the pipeline by itself may optimize the wrong definition of fairness or embed new problems it cannot recognize. Humans left to do it alone will be too slow and will miss too much.

The challenge isn’t just making one AI decision-making system fair, but keeping multiple systems fair across different risks and contexts. The second mechanism operates at the organizational level, spanning all of an organization’s AI-automated decision systems.

FAIR proposes a federated governance structure in which central AI leadership sets minimum standards for fairness—non-negotiable baselines that apply everywhere—while local teams have the flexibility to adapt those standards to their specific context. How much independence an AI agent is allowed to exercise within the AI pipeline depends on the stakes involved: a low-stakes product recommendation might run largely on autopilot; a decision about who gets a medical procedure demands close human oversight.

Why it matters—and why it will be hard

The researchers are candid that adopting this approach will not be easy or cheap. Building governance councils, training staff to work alongside AI monitoring tools, and creating genuine feedback channels with affected communities all represent significant investments. The theory predicts organizations will likely see limited improvement—or even temporary setbacks—before those investments pay off. They call this a “J-curve”: short-term pain followed by long-term gains.

But advocates warn that the cost of inaction is already being felt. Unfair AI decisions can deny people access to housing, health care, credit, and employment, often reinforcing disadvantages that already fall disproportionately on communities of color, women, and people with disabilities. When trust in AI systems erodes, it can trigger legal backlash, regulatory crackdowns, and lasting damage to the organizations that deploy them.

Regulators are already paying attention. The European Union’s AI Act, which came into force last year, requires organizations deploying high-risk AI systems to demonstrate ongoing oversight and accountability—a requirement that maps closely to what FAIR prescribes. In the United States, federal agencies have issued guidance signaling increased scrutiny of algorithmic discrimination.

The researchers hope their framework will help organizations move from a defensive posture of fixing bias only after being caught to a proactive one of building fairness into the fabric of how AI systems are designed, monitored, and governed from the start.

“The question is not whether AI will make high-stakes decisions about people’s lives,” Rai said. “It already does. The question is whether we build the institutional capacity to make sure AI makes those decisions fairly—and keeps doing so as the world changes around it.”

The content is provided for information purposes only.