Everyone Uses LLM Dashboards. Nobody Asked Where the Data Goes.

12 min readOct 27, 2026

What security teams need to stop doing, and what to do instead.

Everyone uses LLM. For emails, for reports, for dashboards, for “just quickly checking something.” It’s fast, it’s impressive, and almost nobody stops to ask what happens to the data after they hit enter.

Here’s how it usually goes. Someone exports a spreadsheet, maybe customer records, support tickets, or last quarter’s numbers. They upload it and type “build me a dashboard.” Ninety seconds later there are charts. Everyone is impressed.

Nobody asks which account that was,

what plan it was on,

or whether that file was allowed to leave the building in the first place.

Nobody is being reckless on purpose. They’re being productive. That’s exactly what makes this hard to fix.

It’s not Just One Office

The numbers say this is everywhere. Verizon’s 2026 Data Breach Investigations Report found that the share of employees regularly using AI tools on corporate devices tripled in one year, from 15% to 45%. And that roughly two-thirds of those users reach AI services through personal accounts rather than corporate ones (Kiteworks summary of the DBIR). Source code was the most common type of data going into unauthorized AI systems (ManageEngine). Shadow AI now ranks among the most common non-malicious insider actions in the report’s DLP data.

The cost side is just as clear. IBM’s 2026 Cost of a Data Breach report gave shadow AI its own analysis for the first time. Shadow AI incidents more than doubled from 20% to 43% year over year, and breaches involving it averaged $5.39 million, up from $4.63 million the year before. IBM also found that 68% of breached organizations lacked governance able to manage AI use or detect shadow AI, and only 19% coordinated AI governance with security (SureCloud’s breakdown of the IBM report). Only 32% of breached organizations had AI policies actually in place (Kiteworks).

One fair warning

much of the shadow AI data in circulation comes from vendors and one-off surveys, and one analysis of the field points out that no government statistics office measures it (StationX). Treat the exact percentages as direction, not precision. The direction is not in doubt.

This has also already happened to a household name. In 2023, Samsung engineers pasted source code and internal meeting notes into ChatGPT while trying to get work done, and the AI Incident Database logged at least three separate instances (Ainora).

Why nobody cares (and why that’s the real problem)

People don’t ignore the risk out of malice.

They ignore it because,

It works → The tool saves hours. The risk is invisible and the, benefit is immediate.

There’s no friction → Nothing in the browser warns you. No approval step, no pop-up.

They don’t know the rules differ by plan → OpenAI says that by default it doesn’t use data from ChatGPT Business, Enterprise, or its API for training (OpenAI). Consumer Free and Plus accounts are different, conversations may be used to improve models unless the user opts out (Strac). Paying for Plus out of your own pocket is not a privacy upgrade.

That last point matters most. When an employee uses a personal account for work, company data lands on the weakest privacy tier, and the company never knew it happened.

Beyond pasting | what can go wrong technically, and why it’s so dangerous

So far I’ve talked about what people put into these tools. The second problem is what the tools, and the dashboards and apps people build with them, can do with that data. These six risks show up again and again in real incidents and in OWASP’s Top 10 for LLM applications.

1. Indirect prompt injection and data exfiltration

An AI assistant can’t reliably tell the difference between data it is reading and instructions it should follow. So an attacker doesn’t need to talk to your AI at all. They hide instructions inside something it will read such as an email, a shared document, a web page, a support ticket. When an employee asks the assistant to summarize or analyze that content, the hidden instructions run with that employee’s access. OWASP has ranked prompt injection as the number one LLM risk across two editions of its list (Aembit).

This isn’t theoretical. In June 2025, Aim Security researchers disclosed EchoLeak (CVE-2025–32711, CVSS 9.3), showing that a single crafted email could make Microsoft 365 Copilot leak data from the victim’s context with no click from the user (SecurityWeek). Microsoft patched it, and there’s no evidence it was exploited in the wild (Startup Defense).

Why it’s so dangerous ?

The victim doesn’t have to make a mistake, the attacker borrows the victim’s own permissions, and patching one flaw doesn’t remove the attack class. It exists anywhere an AI can read untrusted content and reach internal data (Sentra).

2. Broken object- and function-level authorization (IDOR)

IDOR, which OWASP’s API list calls broken object-level authorization, is the classic flaw where changing one number in a URL shows you someone else’s record. Its sibling, broken function-level authorization, is when an ordinary user can call an admin function. AI-built dashboards and apps are prone to both, because the AI builds what you asked for (“show customer data”) and not what you forgot to ask for (“and only to the right person”).

A widely reported example is CVE-2025–48757, tied to apps built with the AI app builder Lovable. A researcher scanned 1,645 apps and found about 170 of them, 10.3%, with critical row-level security failures, exposing data such as home addresses, financial records, and API keys (RockingTech). The CVE entry is marked as disputed because the vendor argues customers are responsible for protecting their own application data (Medium analysis). That dispute is the point, when an employee builds the dashboard, nobody has clearly accepted responsibility for its access control.

Why it’s so dangerous?

the dashboard works perfectly in the demo. The flaw only appears when someone tries something nobody tested, and when it does, it exposes records in bulk, not one at a time.

3. Data leakage through over-sharing and hallucination

Assistants connected to company data inherit the access of whoever asks. Anything the user can technically open, the assistant can find and summarize. Years of loose permissions, such as “share with everyone in the company,” old HR folders, and former employees or guests who never lost access, are suddenly one natural-language question away. As one commentator put it, this is effectively an internal chatbot over everything people technically have access to (Simon Willison). The AI doesn’t create the over-sharing, it makes years of it instantly discoverable (Strac). OWASP moved sensitive information disclosure from sixth to second place in its 2025 list (Invicti).

Hallucination is the other half. The model can produce confident, wrong, or invented details, which then get pasted into a client email, a report, or a dashboard that leadership trusts.

Why it’s so dangerous ?

Obscurity used to protect badly set permissions. Because nobody went looking. AI removes that protection overnight. And a wrong number on a dashboard looks identical to a right one.

4. Vulnerable code generation (XSS and insecure deserialization)

People who build dashboards with AI are often not developers, and AI-written code can run perfectly while being unsafe. Veracode’s 2025 study tested more than 100 LLMs on 80 coding tasks and found that the generated code introduced a security vulnerability 45% of the time. Models failed to defend against cross-site scripting (XSS) in 86% of relevant cases and against log injection in 88%, and security performance had not improved even as the code became more functionally correct (SD Times, Help Net Security). Veracode sells security tools, so read it as one vendor’s study, but other research points the same way.

In plain terms → XSS lets an attacker’s script run in another user’s browser, stealing sessions or data. Insecure deserialization means an app rebuilds objects from untrusted data, which can let an attacker run their own code on the server. The underlying habit is what OWASP calls improper output handling And treating what the model outputs as trusted, for example by rendering it as raw HTML or executing it (OWASP LLM Top 10 explainer).

Why it’s so dangerous ?

Because the code works, nobody reviews it. And internal dashboards usually sit close to real data, so a flaw there is a doorway into it.

5. No audit trail, and shadow IT

If nobody knows which AI tools are in use, there are no logs of what was uploaded, who asked what, or which data left. After an incident you can’t answer the first question customers and regulators ask , what exactly was exposed?

Remember the earlier numbers. Two-thirds of AI use on corporate devices goes through personal accounts, and 68% of breached organizations lacked the governance to detect shadow AI.

Why it’s so dangerous ?

You can’t investigate, contain, or properly report what You can’t see. “We don’t know” is the worst possible answer in a breach.

6. Model inversion and membership inference

These are privacy attacks on the model itself. Membership inference works out whether a particular person’s record was in the data a model was trained on. Model inversion, also called data reconstruction, tries to rebuild pieces of the training data from the model’s outputs. NIST’s adversarial machine learning taxonomy covers both under privacy attacks (NIST AI 100–2 E2025).

To be honest about the odds. If your staff only use a hosted chat tool, this is the least likely risk on the list. It matters when your organization trains or fine-tunes a model on its own data, or exposes an analytics model or dashboard that returns detailed scores to a wide audience.

Why it’s so dangerous ?

The attacker doesn’t break in, they just ask questions. And even confirming that a person appears in a sensitive dataset, such as a patient list or a customer segment, can itself be a privacy breach.

The common thread. AI rarely invents a brand-new kind of weakness. It takes old ones, loose permissions, untrusted input, insecure code, and missing logs, and makes them faster to trigger and harder to see. And the person using the tool is usually not the person accountable for security.

What we need to stop

1. Stop uploading raw data to personal AI accounts. Customer records, credentials, source code, contracts, unreleased financials, internal security findings. If it would be a problem on a USB stick left in a taxi, it’s a problem in a chat window.

2. Stop relying on a ban. Blocking AI sites feels decisive. But one compiled study reports that 41% of employees find a way around blocked AI apps (dope.security, citing UpGuard). A ban without an approved alternative just moves the behavior out of sight.

3. Stop installing AI browser extensions and connectors without review. Verizon found the average organization has more than 15% of users running unauthorized AI extensions in their browsers (see the Kiteworks summary above). Extensions and connectors can see far more than a pasted paragraph.

Get Eng. Naveen Ganeshan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

4. Stop writing policies nobody can enforce. IBM’s finding is blunt in spirit , organizations are producing AI policies faster than they build any way to apply them. A PDF in the intranet is not a control.

5. Stop shipping AI-built dashboards and code unreviewed. An AI-generated chart can be confidently wrong, and AI-generated code can be insecure while working fine. If a number goes to leadership, someone should be able to trace it to the source. If a dashboard touches company data, someone should test who can see what.

What we need to do instead

Find out what’s actually in use. You can’t govern what you can’t see.

Look at proxy and DNS logs,

browser telemetry,

SSO logs,

and expense reports for AI subscriptions.

Ask people openly. An amnesty-style “tell us what you use” conversation gets better answers than an investigation.

2. Give people a sanctioned way to do it. Provide a business-tier account with SSO and admin controls. So the fast path is also the safe path. This does not make everything safe. Business-tier data still goes to a third party, so classification still matters. But it removes the worst default.

3. Write a one-page rule people can remember. Use three buckets,

Green Bucket → public or non-sensitive content. Use any approved tool.
Amber Bucket → internal but low-impact. Approved business account only.
Red Bucket → customer personal data, credentials, source code, legal and security material. Not in external AI tools, or only through an explicitly approved, reviewed setup.

4. Add technical guardrails and logging. Data loss prevention that understands AI sites, browser controls, an extension allow-list, and SSO enforcement for approved tools. Keep logs of who used which AI tool and what was uploaded, so you can answer questions after an incident. Policy tells people what to do and guardrails catch the moment they forget.

5. Clean up permissions and limit what AI can do. Before connecting any assistant to company data, remove “everyone” access, stale guest accounts, and old shares. Because the assistant will inherit all of it. Then apply least privilege to the AI itself, read-only access where possible, only the tools it needs, and a human approval step for anything irreversible. Be especially wary of any setup where one AI can read untrusted content, reach internal data, and send information outward. Because that combination is what prompt injection exploits.

6. Treat AI output as untrusted. Review AI-built dashboards and code like any other code. Peer review, automated security scanning, and a simple access test with two different user accounts to confirm one can’t see the other’s data. Never render or execute raw model output without checking it first.

7. Treat AI vendors like any other supplier. Check the data processing terms, retention, where data is processed, and who can access it. If you hold ISO/IEC 27001 certification, the hooks already exist in Annex A of the 2022 edition, acceptable use of information (A.5.10), information access restriction (A.8.3), logging (A.8.15), data leakage prevention (A.8.12), secure coding (A.8.28), supplier relationships (A.5.19 to A.5.22), and security awareness and training (A.6.3). The usual gap is that nobody has pointed those controls at AI yet.

8. Train with real stories, not generic slides. Show the Samsung case. Show EchoLeak. Show what a pasted spreadsheet looks like from the other side. People change behavior when they can picture the consequence.

9. Build it into how the organization already works. AI incidents should land in the same register as every other incident and corrective action. And reported to the same management review, owned by the same process owners. Security culture grows when it travels through channels people already trust, rather than arriving as a separate program. (I explore this idea in more depth in my paper on aligning ISO 27001 with ISO 9001- 10.5281/zenodo.23054363.)

If you only do five things this week

List the AI tools your teams are using, officially or not.
Publish the one-page green, amber, red rule ( One of the Buckets )
Give people one approved account to use instead.
Turn on monitoring for uploads to AI sites, and check what any AI assistant connected to company data can actually reach.
Tell managers, not just IT, what the rule is and why.

The bottom line

The question isn’t whether your colleagues will use AI. They already do, and they’re getting real value from it. The question is whether you’ll know where your data goes when they do, and who can reach it once it’s there.

Right now, in most offices, the honest answer is no. That’s fixable. But only if security stops being the department that says no and becomes the one that makes the safe path the easy one.

Is your organization blocking AI tools, allowing them, or somewhere in between? I’d like to hear what’s working, and what backfired, in the comments.