AI customer service bots can leak security codes
AI customer service bots can be tricked into giving attackers security codes or pretending to be you. Attackers send fake messages to the bot to make it share private info or act like you're asking for help.
- Report priority
- Medium
- Targets
- AI customer service bots
How it works
- Attackers send fake messages to AI customer service bots to trick them into sharing security codes or acting like a real customer.
- One way is by injecting fake text into a chat session and using the bot's transcript feature to send a fake email from a trusted support address.
- Another way is by sending an email that looks real but is actually controlled by the attacker, making the AI bot think it's from you.
What to do
If your company uses AI customer service bots that can read emails, send transcripts, or make account changes. Check if your company's support system lets AI bots handle security codes or sensitive account actions without extra security checks. If so, attackers could trick the bot into sharing your security codes or pretending to be you.
Ask your company's IT or security team to review how their AI customer service bots handle emails, transcripts, and security codes. They should add extra checks to make sure the bot only acts on real requests from real customers, not fake ones sent by attackers.
Technical details
An attacker sends a fake email to an AI support bot pretending to be a customer asking for a security code reset. The bot, tricked by the fake email, shares the code with the attacker. The attacker then uses the code to access the customer's account.
AI customer service bots, often granted access to customer profiles, billing data, and account modification tools, can be manipulated into leaking sensitive information or performing unauthorized actions. Attackers exploit this by injecting malicious prompts or spoofed emails to trick the bot into revealing details like security codes or account credentials. For example, an attacker could craft a phishing email appearing to originate from a trusted support address by abusing email header validation flaws, where the AI agent relies on the visible From field rather than authenticated sender fields.
Once the bot processes the request, it may retrieve billing data or account details, then unintentionally forward responses to the attacker's controlled email address via CC or reply fields. Researcher Inti De Ceukelaire demonstrated at DEF CON 34 how these risks extend to multi-factor authentication (MFA) bypasses. Some bots require one-time passcodes for sensitive actions like phone number updates, but weak email normalization allows attackers to reset rate limits by altering email address formatting (e.g., adding comments or aliases) while targeting the same inbox.
This enables credential theft or unauthorized account changes without traditional exploitation.