The question I get asked most often before an AI agent goes live is the one that should be asked: “what can it actually see?” In the last six months I have watched two small businesses hand an agent full read and write access to their entire Google Workspace because that was the default scope the install screen requested, and nobody read it. Neither was breached. Both were one stolen token away from a very bad week. This post is the access checklist I now run before connecting any agent to anything.
It is not a reason to avoid AI agents. It is the difference between an agent that can summarise your enquiries and one that can, if its credentials leak, read every contract, invoice and customer record you own.
The mistake almost everyone makes
When you connect an AI agent to Gmail, Outlook, a CRM or your accounting software, you are handed a permissions screen. Most of these screens default to the widest scope the product can technically use, because that guarantees nothing breaks in the demo. People click “Allow” because the alternative looks like effort.
The result is over-provisioning: the agent has permission to do far more than the job needs. An email triage agent that only needs to read and label messages often ends up with permission to send mail, delete mail and read your full contact list as well. None of that is required for the task. All of it is exposure if the token is compromised.
The fix is a principle borrowed straight from security engineering, and it costs you nothing but a careful read of the consent screen.
Least privilege, in plain English
Least privilege means the agent gets the narrowest access that lets it do its actual job, and not one permission more. If the agent drafts replies but you press send, it does not need send permission. If it triages your inbox, it does not need access to your Drive. If it reads bookings, it does not need write access to your customer database.
Here is how the common scopes map to real tasks.
| What the agent does | Access it genuinely needs | Access to refuse |
|---|---|---|
| Triages and labels email | Read messages, modify labels | Send, delete, full contacts |
| Drafts replies for you to send | Read messages, create drafts | Send on its own |
| Summarises calendar | Read-only calendar | Write, delete events |
| Pulls booking data into a report | Read-only on the booking table | Write, delete, billing fields |
| Files documents | Write to one named folder | Access to the whole Drive |
The pattern is the same every time: prefer read over write, prefer one folder over the whole account, and never grant send or delete until you have watched the agent work in draft mode for a fortnight. I covered the two-week draft period in detail in the practical email agent setup guide; the same discipline applies to every system an agent touches, not just email.
OAuth tokens are the thing that actually leaks
Agents do not store your password. They log in with OAuth, which hands the tool a token: a long string that proves “this app is allowed to act as you, within these scopes.” That token is the asset an attacker wants. If it leaks, the password and even multi-factor authentication on your account do not help, because the token already represents an approved login.
Three things follow from that.
First, scope matters more than you think. A leaked read-only token is an irritation. A leaked send-and-delete token is an incident. The narrower the scope you granted, the smaller the blast radius.
Second, tokens accumulate. Every tool you ever trialled and forgot to disconnect still holds a token. I have audited Google accounts with eleven connected apps, of which the owner recognised four. Each of the other seven was a standing risk for no benefit.
Third, you can revoke instantly. In Google or Microsoft account settings there is a connected-apps list. Removing an app kills its token immediately, with no need to involve the vendor. That is your emergency stop, and you should know where it is before you need it.
The data an agent should never touch
Some categories of data do not belong in an AI agent’s reach at all, regardless of scope. Keep these out:
- Card and bank details. No legitimate agent task needs raw payment data. If a tool asks for it, that alone is a reason to walk away.
- Passwords and secrets. Agents that scrape inboxes will happily ingest password-reset emails. Filter these out of any mailbox the agent reads, or exclude that label.
- Special category data. Health, ethnicity, and similar data under UK GDPR carry extra obligations. If your inbox or records contain them, an agent reading that mailbox needs a far higher bar of justification and documentation.
- Anything you would not put in a third party’s filing cabinet. That is the honest test. The agent is a third party. Its vendor’s staff can, under some plans, see flagged content for support and model-quality reasons. Read the data processing terms before you assume otherwise.
The GDPR position, briefly
If you are a UK business and the agent processes personal data, the agent’s vendor is a data processor and you are the controller. That is a real legal relationship, not a formality. Three things make it compliant rather than risky:
- A signed data processing agreement. Every reputable vendor offers one. No DPA, no deal.
- A data region you can name. Pick a tool that processes in the UK or EU if any customer is based there. “We use AWS, it is fine” is not an answer.
- An updated privacy notice. One sentence stating you use a third-party AI tool to assist with a named task, linking to its privacy policy.
The ICO’s guidance on AI and data protection ↗ is the authoritative reference here and is updated as the regulator’s position moves. Read it once before your first agent, not after your first complaint.
A quarterly access audit you can run in ten minutes
I do this for every managed customer, and you can do it yourself. Once a quarter:
- Open the connected-apps list in Google or Microsoft, and your CRM and accounting admin panels.
- For each connected app, ask: do we still use this, and does its access still match the job? Revoke anything you do not recognise or no longer need.
- Check that multi-factor authentication is on for every account an agent connects to. The agent’s token sits on top of that account; if the account falls, the token falls with it.
- Confirm send and delete permissions are still limited to the agents that have earned them.
- Write the date in a note. Next quarter, you will be glad you did.
The NCSC’s small business guide ↗ is the right baseline for the wider cyber hygiene this audit sits inside. Ten minutes a quarter is cheaper than one breach notification.
If an agency runs your agents, demand answers
A growing number of small businesses do not configure agents themselves; they pay an “AI agents as a service” provider. That can be the right call, but it concentrates the access question in someone else’s hands. Before you sign, ask the provider these five questions and expect crisp answers.
- What scopes will you request, and why does each one map to a task?
- Where is our data processed, and will you sign a DPA?
- Who at your company can see our data, and under what circumstances?
- How do we revoke your access in an emergency without phoning you?
- When we leave, what happens to our data and your tokens?
If a provider is vague on any of these, that is not a communication problem, it is a security one. I make the same argument about ongoing ownership in who manages your app after launch: the install is the easy part, and the access you grant lives on long after the novelty wears off. If you are still deciding whether to build, buy or hire an agent at all, the build, buy or hire decision guide is the right place to start.
Frequently asked questions
Is it safe to give an AI agent access to my email?
Yes, with least privilege. Grant read and label permissions only, keep send and delete off until you have watched it work in draft mode for two weeks, turn on multi-factor authentication on the email account, and sign the vendor’s data processing agreement. The risk is over-provisioning, not the agent itself.
What is the single most important security step?
Granting the narrowest scope the task needs. A leaked read-only token is an annoyance; a leaked send-and-delete token is an incident. Everything else is secondary to getting the scope right at the consent screen.
How do I remove an AI agent’s access quickly?
In your Google or Microsoft account settings there is a connected-apps list. Removing the app revokes its OAuth token immediately, with no need to contact the vendor. Do the same in your CRM and accounting admin panels for any agent connected there.
Does an AI agent reading customer email breach GDPR?
Not by itself. The vendor becomes a data processor and you remain the controller. Sign a data processing agreement, pick a tool with a UK or EU data region, and add a line to your privacy notice. Done properly, it is compliant.
How often should I review agent access?
Once a quarter. Open every connected-apps list, revoke what you no longer use, confirm multi-factor authentication is on, and check that send and delete permissions still match the job. It takes about ten minutes.
Want a second opinion before you connect it?
If you are about to give an AI agent access to live email, customer records or accounting data and you want someone to read the consent screen with you, get in touch. I will look at the scopes the tool is asking for, your data protection position, and whether the access matches the job, then give you a straight answer on what to grant and what to refuse. No charge for the first conversation. For the wider picture on what these agents are and where they help, start with the plain English explainer on AI agents for small businesses.