Every few months, a new online trend convinces ordinary people to hand over personal information without thinking twice. Recently, many South Africans have been uploading photos to create AI caricatures and profile pictures, often using tools linked to ChatGPT or similar platforms. At the same time, employees are pasting contracts, medical records, and voice notes into these systems because it feels easier than drafting from scratch. However, most users do not stop to ask what happens to that information after the image is created or the answer is generated.
In this blog, we will walk you through what data is collected when a person uploads information to ChatGPT, how that data can be used afterwards, and what privacy controls really mean in practice. We will also look at the risks of sharing details about a criminal act and how South African law may respond in such situations. The aim is to give practitioners clear language to advise anxious clients who want practical steps rather than theory. We begin with the basics, then move on to real-life South African examples and the legal tools that are available today.
What data is collected when you upload anything to ChatGPT?
The recent caricature trend has given us a simple way to see how much information people are willing to hand over without realising it. Users type a prompt asking ChatGPT to create a caricature based on “everything you know about me,” and then add photos, job descriptions, family details, and hobbies. What feels like a bit of fun is a moment where a very detailed personal profile is being built in one place. Many South Africans have shared these images on LinkedIn, Instagram, and Facebook, often with captions inviting others to do the same.
When we look at what happens behind the screen, a few categories of data usually come into play:
- Account and device information. This includes the email address used to sign up, the IP address, location information, and details about the device or browser being used.
- Content that the user actively provides. Every message typed into the chat, any document uploaded, and any photo shared becomes part of the interaction.
- Behavioural information. The system can record how often a person uses the tool, the kinds of topics they ask about, and how they phrase their questions.
The caricature example shows how quickly this can grow. If the first image does not look accurate, users tend to add more context about where they work, what they studied, or what their family looks like. Step by step, the system receives richer material. None of this feels sensitive in isolation, but together it can reveal far more than people intended. That is usually the moment a client realises they have shared much more than a simple photo, and many people assume the tool must have gone off to search for them somewhere on the internet to produce such a personal result.
It is usually much simpler than that. The system works with what the user has already provided, and with general knowledge it was trained on long before. The output feels personal, not because the tool has investigated anyone, but because the user has slowly painted a detailed picture without noticing. Then they add a line about their job, so the caricature looks right. They mention a hobby or two. Each step feels like a minor edit, the kind of thing you would tell a graphic designer sitting next to you. Only later do they realise that all those edits now live together in one digital space.
How your data is used after you have uploaded it
Once the information has been shared, the platform begins to use it in a few predictable ways. Most users picture a private conversation that ends when the window closes, but the reality is more layered than that.
1) To generate the response you asked for
The first and most obvious use is to make the tool work. The text, photo, or document is sent to servers so the model can read it and produce an answer. If someone pastes a draft contract and asks for more explicit language, that contract is processed outside their own computer. The same happens when a photo is uploaded for a caricature. The system cannot create anything without first handling the original material.
2) To improve how the system performs
In many cases, conversations can help the model learn. This means examples of how people write, the kinds of questions they ask, and the corrections they make may shape future versions of the tool. A person might only want a quick summary of a medical report, yet the wording of that report could still influence how the model responds to other users later on. This is the part that surprises people who believed the chat was only for their own benefit.
3) To monitor safety and misuse
Platforms also keep information for security reasons. They need to check for harmful behaviour, fraud, or technical problems. Even if a user deletes a conversation on their screen, the data may remain for a period so those checks can happen. Someone who thinks they have removed a sensitive file may discover it is not gone as quickly as they imagined.
4) To support different versions of the service
The way data is treated often depends on which account is being used. Personal plans usually allow broader use of content, while business or enterprise plans are designed to keep material separate from training. The difficulty is that many people use a personal account for work tasks, especially after hours. A staff member might upload a client list at home, not realising the organisation would never have approved that version.
5) To connect across multiple interactions
When the same person returns to the tool, earlier conversations can shape new ones if memory features are switched on. That can be helpful for convenience, but it also means details shared over weeks slowly join. A hobby mentioned in January can sit next to a workplace document uploaded in March, creating a fuller picture than either item alone.
Implications if a user shares information about a criminal act with ChatGPT
This is not about the platform acting as a police officer or secretly reporting every message to authorities. ChatGPT is not a law-enforcement tool, nor is it designed to investigate crimes, nor does it create legal privilege simply because someone typed their confession into a chatbot. The real issue is what happens when a person places details of unlawful conduct into a system that stores and processes information in ways they cannot control.
From a legal perspective, two key takeaways matter most:
- Typing crime details into a chatbot does not make those details protected or confidential in the way a conversation with an attorney would.
- Once that information is uploaded, it may exist outside the user’s control and could become relevant in later investigations or civil disputes.
In South Africa, this becomes serious because our law already treats digital communications as objective evidence. A WhatsApp message can be used in court, and the same principle applies to AI chats if they are obtained through proper legal channels.
The platform is not a safe space for confessions.
People sometimes treat ChatGPT like a private diary and ask questions such as how to hide money, alter documents, or avoid detection. The tool may refuse to help with illegal conduct, but the questions themselves can still be recorded. If those records are later requested in an investigation, the user cannot rely on any automatic right of secrecy.
There is no attorney-client privilege.
South African law protects communication between a lawyer and a client because it serves the interests of justice. A chatbot does not enjoy that status. Even if a person is seeking legal information, the conversation remains an ordinary digital record, not privileged advice.
Admissions can have consequences beyond criminal law.
A person might describe paying a bribe, avoiding tax, or sharing intimate images without consent while asking for “general advice.” Those statements could later affect employment hearings, disciplinary processes, or civil claims. The law does not require a formal confession in writing.
Requests for harmful actions can trigger monitoring.
Platforms are expected to prevent misuse, such as threats, harassment, or the distribution of illegal material. When someone asks for help to commit violence or cybercrime, the system may block the request and retain the interaction for safety reasons. That record can exist even if no crime ultimately occurs.
The Cybercrimes Act and other South African laws still apply.
Sending threats, inciting violence, or sharing intimate images without consent are offences whether they are typed to a person or to a chatbot. Asking the tool to draft such a message does not remove liability. The digital trail may even make proof easier. The uncomfortable truth is that many users speak to AI more freely than they would speak to another human. They test ideas, admit mistakes, and ask how to “fix” situations that involve unlawful behaviour. In South Africa, those words can matter later, and the platform is not built to protect them from themselves.
South Africa has already seen how quickly AI chats can move from private experimentation to public legal trouble. The problem is not only that the technology can be wrong. The deeper issue is that people trust what feels informal and then treat it as if it carries no consequences. In one matter, legal representatives relied on case references generated by an AI tool that were later shown to be fictitious. The court made it clear that time pressure does not excuse weak verification. If a legal argument can collapse because an AI produced convincing but invented authorities, then a person’s own words about unlawful conduct can also create problems later if they are recorded and compared with other evidence.
How to protect yourself and your organisation: Practical steps for the South African context
The safest approach is to assume that anything typed into an AI tool could one day be seen by someone else. That does not mean the technology cannot be used; instead, it means the use must be deliberate and cautious, especially where personal or business information is involved.
Decide first what should never be uploaded.
Organisations need a clear line about documents that stay off AI platforms altogether. Client files, medical records, identity numbers, bank details, and internal investigations should not be copied into a chatbot. Individuals should follow the same rule for their own sensitive information, even when they feel stressed or rushed.
Use the correct version for the right task.
If a business allows AI tools, it should use enterprise plans that limit how data is reused. Staff should not switch to personal accounts to finish work at night. That one shortcut can move protected information into a space the organisation cannot control.
Remove identifying details before asking for help.
When someone needs assistance with a document, they can first change names, addresses, and reference numbers. A contract can be turned into a neutral example instead of the real agreement. This small step reduces the risk of the text being stored.
Treat AI output as a draft, not a decision.
Every response should be checked by a human who understands the context. This is especially important for legal, financial, and medical matters. The tool can help organise thoughts, but it should not replace judgment.
Create internal rules and training.
Workplaces should explain to staff what they may and may not paste into AI systems. A short policy is better than silence. People often make mistakes simply because no one told them the boundaries.
Keep evidence when something goes wrong.
If harmful content is created or sensitive information is shared by mistake, take screenshots and record dates immediately. Those details matter if a complaint, criminal case, or civil claim follows.
Think before asking for help with anything illegal.
Questions about hiding assets, altering records, or threatening someone can leave a digital trail. The safest option is to speak to a qualified professional in a confidential setting rather than experimenting with a chatbot.
These steps do not require technical expertise; instead, they need the same caution South Africans already use when sending emails or posting on social media.
Conclusion
AI tools have become part of daily life faster than most laws and policies could adapt. For many South Africans, it feels friendly and private, like a helpful assistant sitting next to them. The reality is more complicated. Information shared in these chats can be stored, reused, and sometimes retrieved in ways users never expected. The best protection is a mix of common sense and clear boundaries. People should pause before uploading real names, real documents, or real confessions. Organisations should treat AI platforms the same way they treat any other third-party service that handles personal data. South African law already recognises digital messages as evidence, and AI conversations are no different.
For practitioners advising clients, the message is simple. Use the technology, but do not trust it with what should stay private. Ask what problem needs solving before deciding to paste anything into a chatbot. That one question can prevent years of difficulty later.
About the author
Theshaya Naidoo is a PhD (Law) Candidate and Canon Collins Scholar. Her research focuses on 'The Legal & Ethical Implications of Neurotechnology on the South African Criminal Justice System'. She holds an LLM in Medical Law, where her thesis focused on 'The necessity of sui generis AI regulation in South Africa'. Theshaya is a Gawie le Roux Institute of Law alumna and top achiever in the 400-hour GLR PVT School.
Last updated on 12 February 2026.
(Main blog image for illustration only.)