These assistants could live on desktop, in-browser, or in ambient/connected devices — so the context varies.
A few challenges I keep running into:
> Unknown tools – I don’t know ahead of time which apps the user uses (Gmail vs Outlook, Google Calendar vs iCloud). But with MCP this should be OK as long as each tool is set up.
> Auth fragmentation – Each tool has different auth flows (some sync, some async), and different scopes.
> Granularity – I want to limit what the assistant can do (e.g., just read subject lines, not full emails).
> Shared permissions – Ideally, multiple agents should be able to re-use the same connection with scoped permissions.
> User control – Users need a clear view of what each agent can do, and a way to revoke or update it.
I’ve started sketching an infra layer to help solve this — something like “Auth0 for AI agents” — with:
> Categorised integrations (email, calendar, etc.)
> Method-level permissioning
> Agent-specific permissions + scopes
> Async-friendly auth flows
> End-user permission dashboard
Before building too far, I’d love to hear how others are solving this. Are you working on similar AI assistants or tools? How are you handling integrations, auth, and control?
Checkout the step-by-step quickstart [1] if you want to go through calling the Google Calendar API from an AI agent (Vercel AI SDK based in this case). There are also how-tos for other frameworks like LangGraph, GenKit, LlamaIndex, etc. Async authorization is also supported via CIBA (Client-Initiated Backchannel Authentication).
You can also secure remote MCP servers [1] with Auth0.
[0] https://auth0.com/ai/docs/call-others-apis-on-users-behalf [1] https://auth0.com/blog/secure-and-deploy-remote-mcp-servers-...
Disclosure: I work for Auth0.