Most Organizations Don’t Have the Consent They Think They Have
This post is inspired by a recent workshop I led on AI and HIPAA, where I focused on the dual challenges LLM AI generates: consent and impact. There’s a moment in almost every new software deployment conversation where businesses assume user and employee consent to the new tool has been covered. The organization might think a terms-of-service checkbox, an employee acknowledgment form, or a general authorization signature on a TOS no one reads is enough to count as informed consent, so they move forward, confident that the people most affected by the software agreed to the impact of its use. Frankly, this was not enough explicit consent prior to LLM AI hitting the scene, and now, with the higher stakes and increased consequences of AI use to the customer and the employee, it doesn’t come close to the level of consent a business needs for these tools.
Consent is a technical term with specific requirements that a simple checkbox doesn’t satisfy. Impact is what happens when the tool is running and something goes wrong, creating unacceptable consequences for unsuspecting users and employees (and in the case of LLM AI, often broader consequences for people that aren’t even using the tool as well). Most organizations focus on getting “consent” via a simple signature and don’t think about the impact of their choices.
This matters because the strength of that consent determines what the organization is actually responsible for when the impact hits. If the consent was real, there’s a genuine agreement between informed parties to point to and a path to resolution. If the consent was a formality that produced a signature without producing understanding, that signature offers far less protection than the organization assumes. And if nobody assessed the potential impact before deployment, there’s no framework for responding when the consequences arrive.
What consent actually requires
Our current framework for informed consent came out of biomedical ethics research in the 1970s and 1980s. Thomas Beauchamp and Ruth Faden, building on work from the U.S. National Commission for the Protection of Human Subjects, identified five elements that need to be present for consent to be meaningful:
- Disclosure: the person received relevant information about what they were agreeing to.
- Comprehension: they understood it.
- Voluntariness: they were free from coercion, manipulation, or structural pressure that made refusal impractical.
- Competence: they had the capacity to understand and evaluate what was being proposed.
- And consent itself: the act of agreement.
A terms-of-service checkbox usually only satisfies one of these, at best. The person clicked “agree,” certainly, but whether they understood the technical system they were agreeing to have their data entered into, whether they had a realistic option to decline, or whether the information was presented in a way that made comprehension possible are all questions that largely go unanswered. The organization treats the click as sufficient because the click is what the usual compliance process measures.
Research backs this undertanding of consent up
Borberg, Hougaard, Rafnsson, and Kulyk published a study in 2022 examining how people interact with cookie consent interfaces four years into the EU’s General Data Protection Regulation. They combined quantitative measurement of consent rates with qualitative interviews. and their findings were clear. Participants could identify when an interface was steering them, they were aware of dark patterns, and they reported discomfort with standard data practices. Yet they consented anyway!
The researchers attributed this to decision fatigue, meaning the dark patterns in the system produced agreement without producing consent. They determined that while the signature existed, the necessary understanding didn’t.
If that’s true for cookie banners, consider what it means for generative/LLM and agentic AI tools that process employee communications, generate clinical notes, analyze customer behavior or make recommendations that affect people’s careers, health, or finances. The complexity of these systems exceeds what most users are able to evaluate, and the structural pressure to agree (e.g. your employer adopted this tool, your healthcare provider uses this platform, this service requires opt-in to function) removes the voluntariness that consent requires.
Why this hits harder with AI
There is so much research out there about consent, however; I chose to focus on the cookie consent research because it’s an analogy businesses can understand and because AI tools raise the stakes in two specific ways:
Scope: A cookie tracks browsing behavior on one site, but an AI tool integrated into a workplace platform can process emails, calendar data, meeting transcripts, documents, and interpersonal communications. The surface area of data exposure is larger by orders of magnitude. It pulls in people who never consented at all, who may simply be on an email chain or in a contact list, and the person being asked to consent often has no way to know what the tool can access or who may get swept up in its net.
Agency: Most of the generative AI tools deployed in the last two years are reactive: they process data when asked. The next generation of AI is agentic, meaning it takes actions autonomously. It sends messages, schedules meetings, compiles information, reaches out to contacts. Meta’s Muse AI, launched in 2026, demonstrated what this looks like when it goes wrong: it built dossiers on users, bought things users hadn’t asked it to (costing them real money!) shared home addresses without permission, and contacted people on users’ behalf without authorization. When a tool acts on its own like that, the question moves from “did the user agree to this processing” to “did the user agree to this action.” In many cases, the answer is no, because the user didn’t know the action was possible
What this means for your organization
Three practical implications of lackadaisical consent practices come to mind for your organization.
First, your legal cover may be shakier than your legal team believes. A consent process that fails on comprehension, voluntariness, and competence will not hold up well in a regulatory environment that is increasingly interested in whether consent was meaningful. The EU’s AI Act, state-level privacy legislation in the U.S., and sector-specific regulations (HIPAA in healthcare, 42 CFR Part 2 for substance use records, FERPA in education) are all moving toward standards that treat a checkbox as insufficient.
Just having an organization’s paperwork in order doesn’t shift the ethical weight of consent requirements. If the people affected by your AI deployment agreed to something they didn’t understand under conditions where declining wasn’t a realistic option, the organization still carries responsibility for what happens next. This is true legally in a growing number of jurisdictions, and it’s also true reputationally in almost all of them.
Building real consent is possible, and making the extra effort to do so not only produces better results, it creates better rapport and trust with your customer base. People who understand what they’re agreeing to behave differently once they’re using the tool. They catch problems earlier, engage more productively, and trust the organization more. Goodwill is worth the extra effort.
Three questions to ask before deploying AI
Before deploying the next AI tool or evaluating one that’s already deployed, work through these three questions:
1. Can the people being affected describe, in their own words, what the tool does with their data? If the answer is no, disclosure and comprehension failed, regardless of what the forms say.
2. Could they decline without professional, financial, or social cost? If the tool is mandatory, or if opting out creates a visible disadvantage, voluntariness is absent. The consent form may be signed, but the consent isn’t real (we see this one often with healthcare AI deployment).
3. Do they have the technical background to evaluate what they’re agreeing to? If the system is complex enough that evaluation requires specialized knowledge, it changes who should be making the decision and how the consent process needs to be structured.
The answers to these questions show you where your organization is carrying risk that your compliance process may not have revealed.
Where to start
Pick your highest-stakes AI deployment and audit it against Beauchamp and Faden’s requirements: disclosure, comprehension, voluntariness, competence, and consent. Not the compliance version of the audit, though, the psychological one. Ask whether the people affected actually understood what they agreed to and whether they had a genuine choice. If you find gaps, you have two paths forward: redesign the consent process so it works, or redesign the deployment so it centers customer privacy and care and doesn’t require consent that can’t be meaningfully given.
The organizations that build real consent into their AI strategy make better decisions, surface problems earlier, and absorb less reputational damage when things go wrong. The ones that treat consent as a compliance task eventually learn why that approach was always going to fail.













0 Comments