Two Early 2026 AI Exposures: Lessons for the Future of AI and Data Governance

It has been a difficult few weeks for AI and data governance in the business world.

In February and March of 2026, two major organizations—one a legacy retailer and the other a global consulting powerhouse—faced serious data exposures tied directly to their AI-powered chat systems. These aren’t futuristic hypotheticals; they’re real-world demonstrations that deploying generative AI at scale introduces new risks that go beyond what traditional security practices can address.

While the breaches differ in scope and impact—one exposed millions of customer interactions publicly, the other allowed an AI agent to compromise an internal knowledge system—they share a common thread: rushed or misconfigured AI deployments turning sensitive conversational data into easy targets.

Case 1: Sears Home Services – Public Exposure of Customer Chats and Voice Recordings

Earlier this month, cybersecurity researcher Jeremiah Fowler discovered three completely unprotected databases—no passwords, no encryption—containing massive amounts of data from Sears’ customer-facing AI chatbots and voice agents. These included 3.7 million chat-log transcripts, 1.4 million audio recordings (some running hours long and even capturing household noise after the calls ended), and over 4 TB of plaintext data. The data included names, phone numbers, home addresses, emails, appliance details, and repair and delivery information.

The databases were secured shortly after Fowler reported his discovery, but the exposure creates risks of identity theft, phishing, social engineering, and even voice-cloning attacks using the biometric audio.

This is a classic “left the door open” misconfiguration—a familiar data security failure—but amplified by the practice of logging every interaction for training or quality purposes.

Case 2: McKinsey’s Lilli – An Autonomous AI Agent Breaches the Prompt Layer

On February 28, 2026, CodeWall’s autonomous offensive AI agent (operating without human intervention or credentials) targeted McKinsey’s internal generative AI platform, Lilli, used by tens of thousands of consultants for strategy, research, and client work.

The agent found 22 unauthenticated API endpoints, then exploited a SQL injection vulnerability to gain full read-write access to the production database. It took less than two hours for the agent to gain access to 46.5 million plaintext chat messages (covering topics such as strategy, M&A, and client engagements), 728,000 confidential files, 57,000 user accounts, 3.68 million RAG document chunks, and, crucially, 95 writable system prompts that control how Lilli behaves for consultants throughout the organization. Had this been malicious, an attacker could have poisoned responses firm-wide—all without changing a single line of code.

McKinsey responded quickly and responsibly and reported no evidence of client data access beyond the researcher’s test. Still, the incident highlights how prompts and RAG data are becoming crown-jewel assets, vulnerable in ways traditional code never was.

Insights and Lessons for AI and Data Governance

While these two incidents were unrelated, they reflect a shared pattern. Both involve enterprise AI systems whose back-end databases were left wide open to basic and well-known types of attacks. Both stored massive volumes of sensitive conversational data in plain text, and both surfaced within days of each other. This demonstrates how quickly AI deployment can outrun security controls.

Several commonalities are worth highlighting in particular:

• Both involve AI chat interfaces backed by databases storing sensitive plaintext conversations.
• Decades-old flaws (misconfiguration and SQL injection) succeeded because AI layers add complexity without equivalent hardening.
• Exposures happened at machine speed—one by simple discovery, the other by an autonomous agent chaining vulnerabilities.
• Risks extend beyond data theft to include voice biometrics (Sears), model poisoning, prompt tampering, regulatory scrutiny, and erosion of trust.

For business leaders, these incidents highlight the gap between simply using AI and governing it responsibly. AI isn’t just another application. It ingests, retains, and can sometimes even rewrite valuable data at scale.

The unique benefits of AI come with new and unique risks, and in many cases can amplify old risks. Those who hope to leverage AI successfully—and to protect their customers—must understand why they can’t just bolt AI onto existing processes.

Back-end governance

The attack on McKinsey’s infrastructure bears an especially important lesson for the future of AI data security. It’s easy to focus on the scale of the exposure: tens of millions of chat messages and hundreds of thousands of confidential files laid bare. Those numbers are alarming, but they can distract from a different, equally serious concern.

The McKinsey database that CodeWall’s AI agent accessed reportedly held 95 system prompts—the hidden instructions that shape how Lilli interprets questions, reasons, evaluates risk, analyzes deals, and delivers advice. Those prompts were fully writeable. With one carefully crafted database update, a malicious actor could quietly modify Lilli’s core behavior. Consultants would continue using and trusting the system without realizing that its mind had been altered.

This goes beyond a conventional data breach. It instead represents a direct attack vector on the integrity of high-stakes corporate decision making—a kind of supply-chain compromise at the level of reasoning itself.

No organization can guarantee flawless execution across every endpoint, every release, every developer. Strong application-layer defenses are a good start, but for even the most sophisticated organizations, the absence of compensating controls at other layers can turn their most powerful decision-support tools into the weakest link in their strategic chain.

Actionable Governance Steps for Business Leaders

The Sears and McKinsey incidents are early warning signals. For organizations seeking to prepare themselves for the future of AI security, there are a few clear takeaways:

1. Conduct AI-specific risk assessments. Map every deployed AI system: data flows, prompt storage, logging practices, and third-party dependencies. Be proactive about asking what will happen if those systems or their databases are exposed.
2. Enforce zero-trust principles on AI backends. Application-level fixes aren’t enough. Serious AI governance requires infrastructure-level controls. Require authentication for all APIs and endpoints, segment AI infrastructure from core systems, and engage in continuous monitoring to detect anomalies and tampering.
3. Treat prompts and RAG as crown jewels. Implement versioning, integrity checks, access controls, and audit trails. Ensure the ability to detect and roll back unauthorized changes.
4. Build AI-tailored incident response. Require regular reporting on AI exposures and controls. Go beyond traditional breach questions. Ask “Was any model behavior altered?” and “Do we need to re-validate outputs post-incident?”
5. Red-team with modern tools. Consider conducting tests using autonomous agents to simulate fast, chained attacks—not just traditional vulnerability scans.

The Bottom Line

The Sears and McKinsey incidents aren’t anomalies. As AI systems become more deeply embedded in core business processes, risk is expanding from data exposure to decision integrity. With agentic AI on the rise (and attackers increasingly incorporating AI agents into their techniques), the window for securing data used to power AI systems is closing fast.

Beyond the media’s focus on the data that was leaked, these incidents point to deeper structural gaps in how enterprises govern AI systems. These weren’t exotic attacks, they were basic failures at AI scale. These incidents illustrate the new reality of business security practices. AI systems draw on sensitive data, but they also shape decisions, and that means that both layers are now attack surfaces.

Perhaps the most important takeaway is the need for a shift in mindset. AI governance is no longer just an IT concern. It is a core enterprise risk function. Organizations that treat it as such will be the ones that can scale AI successfully and safely—and earn the trust required to sustain it.

Written by Jon Iwry, Fellow, Wharton Accountable AI Lab