AI, Deepfakes, and Phishing

A round-up of AI and LLMs being applied to deepfakes and phishing

Soon I’ll write a nice narrative tying everything together and add more links, but for now here’s just my notes.

Deepfake Calls

Deepfake Images

Warning: adult stuff here

  • An underground website called OnlyFake is generating realistic looking photos of fake IDs for just $15. This could streamline everything from bank fraud to laundering stolen funds.

  • AI-generated explicit photos of Taylor Swift went viral on X. Post-Elon X did a great job of cracking down on it (not).

  • More than 40 high school girls in New Jersey were similarly victims.

  • OctoML, the engine that powers a16z funded Civitai, knew that much of the images on Civitai were NSFW (including nonconsensual images of female celebrities and images that could qualify as child pornography), but ultimately decided to keep working with Civitai anyway

  • Fortunately there are some laws being proposed that would establish civil and criminal penalties for the creation and sharing of nonconsensual explicit deepfake images.