Artificial intelligence applications are no longer just tools to assist in learning, work, or creativity. Rather, they have gradually begun to enter the scene of digital and criminal crimes, whether as an indirect instigator or as digital evidence used by the authorities to uncover those involved.
A 13-year-old boy in DeLand, Florida, learned a harsh lesson about the privacy risks of using ChatGPT, after police stopped him for asking an artificial intelligence question about how to kill his friend, according to Phandroid.
The boy had used a laptop computer provided to him by the school, which contained a monitoring program known as Gaggle. He immediately monitored the inquiry and informed the authorities, who rushed to his location and arrested him within hours.
The friendly nature of artificial intelligence chatbots deceives users into treating them as if they are close friends or trusted advisors, as ChatGPT answers in a warm tone and appears understanding and personal, but in the end it is just a computer program, not a psychotherapist bound to confidentiality, nor a friend who keeps secrets, but rather a machine that processes what is written to it. In many cases, these conversations are recorded, monitored or reported.
Although this incident was uncovered through a school monitoring program, concerns about ChatGPT’s privacy extend beyond the educational environment, as companies can monitor their employees’ use of smart tools, parents can see their children’s chat logs, and law enforcement agencies can request access to the data when needed. Thus, the friendly conversational interface gives users the illusion of privacy that does not actually exist.
Investigations revealed that the boy tried to justify his behavior by saying that he was joking after a dispute with one of his colleagues, but the police explained that such actions are not considered joking in the age of intelligent surveillance, as such inquiries are classified as serious threats.
This incident highlights a fundamental gap in people’s understanding of how to interact with AI-powered chatbots. The friendly and helpful answers provided by ChatGPT create the illusion of a private conversation, when the reality is quite different.
Privacy risks are not limited to school devices, as any device in a work environment, shared computer, or controlled network can reveal the content of conversations with artificial intelligence. This technology may appear personal, but it is devoid of any legal protection of confidentiality. Users should be aware that every request, question, or conversation with AI is treated as data that can be accessed, reviewed, or used against them.
In another incident, the US authorities announced the arrest of a man from Florida named Jonathan Rinderknecht on charges of starting the catastrophic fire that destroyed large parts of California in early 2025, killing 12 people and destroying thousands of homes.
What is noteworthy in the case is that ChatGPT played a pivotal role in the investigations. According to what was revealed by the US Department of Justice, the accused used the application to directly ask: “Will I be held responsible if a fire starts because of my cigarette?” And ChatGPT’s response was: “Yes”.
But what is more dangerous is that the digital devices that the authorities confiscated from the accused contained an image he created via ChatGPT showing a burning forest and crowds fleeing from the fire, an image he created months before the fire.
Judicial documents showed that the accused used the application to imagine fire scenes, and that this digital data helped investigators prove his direct connection to the accident. Rinderknecht now faces a charge of “destruction of property by arson,” a federal crime that carries a penalty of up to 20 years in prison.
Although the two incidents are different, they reveal a new trend in the relationship between artificial intelligence and crime. In the first, ChatGPT was a tool used in the context of a threat, while in the second it became part of the incriminating evidence itself.
Experts stressed that the danger does not lie in the technology itself, but in the way it is used and users’ misunderstanding of its limits.
Interacting with AI is not a private conversation, but rather a digital activity that is recorded, stored and can be referred to when legally necessary.
As individuals increasingly rely on artificial intelligence applications in various aspects of their lives, analysts believe that there is an urgent need to establish clear legal frameworks that regulate the use of these tools and protect users from the consequences of misuse or unintentional involvement in digital crimes.
