ChatGPT Leak: When AI Conversations Become a Google Privacy Trap

In recent days, a privacy scandal surrounding the ChatGPT platform has caused significant uproar. More than 4,500 – and according to the latest reports, up to 100,000 publicly indexed chats – were findable via the Google search engine, even though many users considered them private.

What happened?  

OpenAI had introduced a feature called “Make this chat discoverable” – a checkbox option when sharing a conversation. Users could share chats publicly, expecting them to be accessible only via a direct link. In reality, these conversations could be crawled and indexed by Google, making confidential information such as employee data, job applications, mental health content, or internal corporate strategies publicly visible. A simple search using the URL structure “chatgpt.com/share/…” alone revealed many of these chats.

OpenAI reacted quickly: CISO Dane Stuckey announced on X (Twitter) that the feature was being discontinued – within just one day, it was deactivated and cooperation with search engines to de-index the content had begun. However, cached versions can still remain visible.

Risks for IT and OT Security  

For companies with critical infrastructure (OT) and IT environments, this leak poses an additional serious attack surface:

  • Exposure of trade secrets or technical documentation inadvertently entered into ChatGPT.
  • Compliance risks due to uncontrolled data disclosure (e.g., GDPR, ISO 27001, NIS2).
  • Offboarded prompts: Employees sharing code, access credentials, or configuration details that become indexable.
  • Prompt injection / jailbreak techniques: Security researchers have already manipulated GPT-4 and other systems to extract sensitive data or product keys, for example through specially crafted “Guessing Game” prompts.

From our project experience, we know how often ChatGPT and other AI tools are used for creating or revising materials in companies’ IT and OT areas – often without control and with insufficient user awareness. 

What companies can do – Immediate measures  

  1. Audit your AI usage & shared links 
  • In ChatGPT/Settings under “Shared Links,” check whether conversations were accidentally shared.
  • Immediately delete all “discoverable” links and request de-indexing of open URLs.
  • Search Google with site:chatgpt.com/share/* combined with company-specific terms to identify possible exposures. 
  1. Security policies & awareness training 
  • Develop clear guidelines on what employees may enter into generative AI (no-go: trade secrets, PII, OT details). 
  • Conduct regular awareness training to strengthen knowledge of AI privacy, prompt injection, social engineering risks, and data classification.
  • Our training programs specifically raise employee awareness – we teach how to use AI safely and what risks exist. 
  1. Technical safeguards & governance 
  • Implement AI governance, e.g., use internally hosted AI systems or secured enterprise AI gateways. In many cases, existing licenses can already cover much of this.
  • Use standardized input/output filtering, prompt evaluation, logging, and monitoring for threat detection.
  • Enforce least privilege access, role-based access, and whitelists for external content in the context of RAG architectures.
  1. Incident response & threat modeling 
  • Establish processes for detecting and responding to AI-related privacy incidents.
  • Use tools to assess AI use cases, including adversarial testing and simulation of potential attacker scenarios.

What we offer as an OT/IT security partner  

Our company specializes, among other things, in training and awareness programs for employees in companies – precisely on this topic:

  • AI Threat Awareness Training, including scenarios that explain ChatGPT leaks and prompt attacks in detail.
  • Workshops on security culture, specifically for OT employees and technical teams, focusing on data protection, compliance (NIS2, GDPR, ISO 27001), and insider risk.
  • Training modules on governance, policies, and incident response in the context of enterprise AI.
  • Training for engineers and decision-makers on the secure use of AI (also with certification through one of our partners). 

Conclusion  

The current ChatGPT leak is no longer a theoretical danger but a concrete reality: Thousands of conversations – including business-critical content – were discoverable via Google, often without users explicitly intending this. Companies must respond immediately – through technical controls, training, policies, and governance. Our training specifically raises employee awareness of the risks of generative AI applications and fosters a sustainable security culture – from staff members to the executive level.

If you are interested in raising your employees’ awareness of these topics, please feel free to contact us. 

An den Anfang scrollen