The Evolution of Phishing

AIOP: “AI-Oriented Phishing”

Yoav Sakal & Wild Pointer Team

 

 

Intro

 

• Nowadays, Phishing became huge part of cyber-crime, where around 90% of cyber-attacks starts with a phishing scam & more than 30% of phishing emails are opened, The phishing attack vector has become a huge cyber-threat for most organizations.
• If that’s not enough, ChatGPT has born, offering extensive features for attackers with much more sophisticated attack vectors at-hand.
• Consequently, attackers started offering services to bypass ChatGPT restrictions to create malware, phishing campaigns, and much more.
• In this blogpost, we did a small research focused on the impact- how much damage & scale could be done through ChatGPT oriented-phishing? The results were astonishing.

 

 

AI & Phishing – what’s new?

 

Along our research, we observed many techniques to bypass ChatGPT safety protocols, hence giving attackers believable phishing emails at scale, tailor-made in instant to the victim of their choice, inventing background stories on its own without any intervention.

If attackers would use that (and they are starting to), the statistic of “30% of phishing emails are opened” would easily become much more, giving attackers more opportunities to breach organizations, and consequently we expect to see much more breaches.

 

 

The new characteristics of phishing:

 

• Believable and Sophisticated – mails are written in a professional and believable manner (no more spelling mistakes).
• Completely Tailored – phishing mails are specifically tailored to the victim by using known public-information.
• Scalable & Autonomous – phishing campaigns generated in a matter of seconds, Managing complete phishing campaigns with thousands of victims can be done almost instantly.

 

 

Bypassing ChatGPT’s Safety Protocols

 

For security reasons, we won’t expose all the tricks we’ve found out to bypass ChatGPT’s safety features, but we will say there are many.

 

 

Yes, it’s hard to make ChatGPT completely safe, as it is widely based on structured text and tends to more “probable” text in some sense. As we are not living in a completely safe-world where all text doesn’t contain any “bad-words” / “bad-sentences” according to some specific context, it would make it really hard to disable completely attacker-based queries that wants to get some innocent looking emails according to its context (although they would being used for vicious purposes).

 

Phishing emails by ChatGPT

 

“Creativity” was all over ChatGPT emails – we were astonished how both-unique & creative stories could have been generated through it all along without us instructing it for any specific story just giving it some basic info.

 

Along the way we’ve also observed emails contain:

1. ChatGPT Emails that also offered Fictional Representative Phone-Number (to fill-out) for the victim to call & ask questions so it won’t suspect it is vicious.
2. ChatGPT Emails that invented a new background story about new tv-series in the making that needs to collect some basic survey-data about your family (it was tv-series about family!) in order to create more belivable story line! (In the background it could be used to bypass several user account security questions)

 

New TV-Series phishing email:

 

Product-Experience Survey Phishing mail:

 

Suspicious Activity phishing email:

 

 

PoC – Ongoing & Scalable ChatGPT-Phishing Campaigns

 

We are expecting a rise of campaigns scaling up spear-phishing attacks with unique ChatGPT possibilities.

To make a PoC on ourselves – we have devised simple system that tries to phish ourselves continuously, from asking us various information into answering our own security hesitations.

 

PoC Video:

 

PoC Results*:

 

1. In couple of minutes – Hundreds of uniquely tailor-made mails were generated & sent.
2. Saved ChatGPT-Context – we’ve sent continuously messages from the attacker-side to victims using previous context, and along the way tried to re-assure victims the mails are legit & just “trying to prove their reliability” along the way and get the data.
3.  Bypass ChatGPT query checks at scale – without seemingly any problems at all.

 

*Disclaimer: Our goal here it to protect users and raise awareness about these new attack vectors. We chose to expose only basic techniques we’ve found (publicly-known techniques) & system’s results without exposing extra-tools. We’ve also showed here only a small part of what we’ve fully did, which was tested only on ourselves while keeping it completely private. Usage of  Phishing for attacking targets without prior mutual consent is illegal. It’s the end user’s responsibility to obey all applicable local, state and federal laws. Authors & Wild Pointer Team assume no liability and are not responsible for any misuse or damage caused by this. Only use for educational purposes.