The Weekly Breach Breakdown: Fake It Til You Make It: ChatGPT’s New Image Generator Creates More Problems For Businesses

Summary 

• This episode of the ITRC’s Weekly Breach Breakdown focuses on the risks and potential misuse of OpenAI’s new image generator, part of the ChatGPT platform.
• While the tool can generate impressive, creative images (like anime-style art or Bitcoin ads), it can also be exploited to create realistic fake documents, such as forged receipts and job offers.
• Tests showed the image generator producing increasingly convincing fake receipts and scammy cryptocurrency ads, raising concerns about fraud and deception.
• OpenAI has implemented restrictions to prevent abuse, such as blocking fake ID creation, but bad actors are finding ways to work around them.
• This development highlights how generative AI tools, while powerful, pose new threats in the realm of digital security and fraud.
• To learn about the latest data compromises, consumers and businesses should visit the Identity Theft Resource Center’s data breach tracking tool. If you believe you are the victim of an identity crime, call or text toll-free at 888.400.5530 or live chat on our website, idtheftcenter.org.

Full Transcript

Welcome to the Identity Theft Resource Center’s (ITRC) Weekly Breach Breakdown for April 11, 2025. I’m Timothy Walden. Thanks to SentiLink for their support of the ITRC and this podcast. Each week, we bring you the latest developments in data security and privacy. Today, we’re going to discuss something that is gaining more traction in the digital world: Chat GPT. More specifically, OpenAI’s new image generator, which is part of the ChatGPT family.

If you haven’t heard, this tool can create everything from art in the style of the famous Japanese anime maker Studio Ghibli to images promoting Bitcoin investments. While the tool has gone viral for its creativity, it’s also raising some serious concerns about its potential for misuse, particularly in the hands of scammers.

The big news is that ChatGPT’s new image generator, which was recently made available for free to users, has already demonstrated the ability to create highly convincing fake documents. There have also been several cases where reporters created fake receipts, forged employment offers, and even social media ads promoting cryptocurrency investments—all within minutes and with minimal effort.

While this isn’t inherently a privacy breach or a data security incident, the implications are clear: these generated images can be weaponized to create realistic, fraudulent materials. Whether it’s fake receipts for fraudulent refunds or job offers from nonexistent companies, the possibilities for exploitation are endless.

Let’s examine some examples that illustrate the risks. First, in one test, the new image generator created a fake receipt for two coffees from a popular coffee shop. At first, the result was obviously fake — no logo, incorrect address and no real coffee names. However, after some prompting, it quickly produced a much more convincing version. ChatGPT even incorporated the coffee shop’s real logo. The result looked plausible enough that a bad actor could use it to trick someone into believing they were owed an business expense reimbursement, for example.

The tool was also used to generate a social media ad promoting an investment in Bitcoin — a scam that we’ve seen time and time again. While the tool itself doesn’t seem to allow the creation of everything (such as an ID card for a New Jersey driver’s license), there’s still enough flexibility for scammers to work around restrictions and create fake documents that could pass for legitimate.

This raises an obvious question: how does ChatGPT know what’s legitimate and what’s not? OpenAI has built some guardrails to prevent the most harmful types of content from generating — things like fake IDs or obvious fraud. However, these safeguards are not perfect. Scammers are already finding ways to bypass them. It’s a problem that many AI models face: balancing creative freedom with ensuring that the technology isn’t exploited.

OpenAI has acknowledged these risks, saying that while it wants to give users as much creative freedom as possible, it monitors the use of its tools for any violations of its policies. According to OpenAI, it’s committed to refining these guardrails as it gathers more real-world feedback. However, as we’ve seen with other AI-generated content, bad actors will always look for ways to take advantage.

What can we take away from all of this? For one, the rise of generative AI tools like this reminds us that new technologies always come as a double-edged sword. While they can open doors for creativity, they can also be turned against us. From a security standpoint, it’s clear that AI-driven image generation could make it easier for bad actors to trick people with more convincing, realistic fraud.

If you have concerns about how your personal information may be at risk or if you suspect you’ve fallen victim to fraud or identity theft, the ITRC is here to help. You can speak with an expert ITRC advisor by phone or text, chat live on the web, or exchange emails during our regular business hours (6 a.m. – 5 p.m. PT). Just visit idtheftcenter.org to get started.

Thanks to SentiLink for their support of the ITRC and this podcast. Please hit the like button for this episode and subscribe wherever you listen to your podcasts. We will return next week with another episode of the Weekly Breach Breakdown. I’m Tim Walden; until then, thanks for listening