• tl;dr sec
  • Posts
  • [tl;dr sec] #248 - 10X Your Cloud Security, Secure by Design, AI OffSec Benchmarks

[tl;dr sec] #248 - 10X Your Cloud Security, Secure by Design, AI OffSec Benchmarks

Excellent scaling CloudSec resources, SANS + AWS white paper, two OSS benchmarks for measuring the offensive capabilities of AI models

Hey there,

I hope you’ve been doing well!

I recently had an awesome chat with my friend Alex Smolen, who is the Director of Security at LaunchDarkly.

His talks and posts have been featured in tl;dr sec many a time over the years, as Alex is experienced at building engineering-forward modern security teams.

In this podcast, we discussed:

  • Security Principles - Like how to enable the business at an acceptable level of risk

  • What not to focus on (cough security questionnaires cough)

  • Eliminating classes of vulnerabilities with Invariants

  • Building an “asset security data lake”

  • A 🌶️ take on the subject of some conference talks…

👉️ Watch it here 👈️ 

🍸️ Security Speakeasy @ Global AppSec SF

If you’re in San Francisco next week, feel free to come say hi at a Security Speakeasy with Jit, Oligo, and Semgrep. 7pm-10pm Wednesday September 25th.

Note that Wednesday is tl;dr sec night for me.

Will I be responsible and finish the newsletter ahead of time? Will I finish it in the wee hours of the night after this event? Only time and a typo-filled newsletter will tell.

P.S. There are still a handful of spots to get free security awareness or secure coding training from Tanya Janca (She Hacks Purple).

Sponsor

📣 Identity Threat Detection and Response (ITDR) Guide

Identity Threat Detection and Response has emerged as a critical component to effectively detect and respond to identity-based attacks for both human and non-human identities. Threat actors have shown their ability to purchase credentials, compromise the identity infrastructure and move laterally across an environment. Download this comprehensive ITDR Solution Guide to learn: 

  • How to secure both human and non-human identities in an environment

  • The most common identity threat use cases 

  • How identity-based attacks are commonly orchestrated against environments

👉 Get The Guide 👈

So many breaches start with some sort of compromised identity. Being able to detect and respond to identity compromises is 👌

AppSec

1Password/onepassword-sdk-python
The 1Password Python SDK allows programmatic access to secrets stored in 1Password, creating/updating/deleting vaults, managing users and groups, etc.

Building security from the ground up with Secure by Design
New SANS <> AWS collab whitepaper by Eric Johnson, Bertram Dorn, and Internet OG Paul Vixie, covering: integrating Secure by Design into the SDLC, supporting it with automation, reinforcing defense-in-depth, applying it to AI, and more.

We Spent $20 To Achieve RCE And Accidentally Became The Admins Of .MOBI
watchTowr Labs’ Benjamin Harris and Aliz Hammond share a rollercoaster of a tale in which they inadvertently undermined the CA process for the entire .mobi TLD by registering an expired WHOIS server domain ( dotmobiregistry.net) and responding to queries. They found 135,000+ unique systems still querying the old server, including government/military mail servers and security companies. Some Certificate Authorities were using their rogue WHOIS server responses for domain ownership verification, potentially allowing them to issue fraudulent TLS certificates for any .mobi domain.

💡 Great example of threat modeling a space (WHOIS clients), researching prior related work, hunting for potential impact on different systems, testing hypotheses, dank memes, etc.

Sponsor

📣 Turbocharge FedRAMP compliance with modern infrastructure access

FedRAMP Authorization can elevate a company’s growth by unlocking lucrative federal contracts - but achieving and maintaining FedRAMP compliance is a massive investment. 

Teleport’s infrastructure access platform addresses many of the toughest controls in the FedRAMP process. Download this white paper to learn practical steps that engineering, compliance, and security leaders can take to implement and enforce FedRAMP security controls.

I’ve heard from a number of friends that FedRAMP is rough. Anything that makes FedRAMP easier could be a big win 👍️ 

Cloud Security

lusingander/stu
A text-based terminal UI for browsing Amazon S3 buckets, by Kyosuke Fujimoto et al.

Azure Config Review - Nuclei Templates v10.0.0
Project Discovery’s Prince Chaddha shares newly added Azure Config Review Nuclei templates, automating azure cloud misconfiguration review, and creating custom Azure checks.

Introducing Azure Activity Log Axe
Permiso’s Nathan Eades announces Azure Activity Log Axe, an open-source tool designed to simplify and improve the analysis of Azure Activity logs. It uses the "Axe Key" concept to group related events more reliably than OperationId or CorrelationId, focusing on the "Administrative" log category.

Highlights from fwd:cloudsec Europe 2024
Datadog’s Christophe Tafani-Dereeper shares a few key takeaways from each of the 11 fwd:cloudsec EU talks. What a madman, awesome!

DataDog/undocumented-aws-api-hunter
Nick Frichette has released, along with his fwd:cloudsec EU talk (slides), a tool to uncover undocumented APIs from the AWS Console. The research has already lead to some useful tradecraft, and even two cross-tenant vulnerabilities.

The tool works by using Selenium to pilot a headless instance of Google Chrome to crawl the AWS console and extract routes defined by JavaScript loaded on each page, and then compares the results to the “official” API, e.g. what’s exposed via the AWS CLI / botocore.

How to 10X Your Cloud Security (Without the Series D)
Fantastic fwd:cloudsec EU talk by Rami McCarthy with an epic distillation of useful resources and ideas across building a security program, invariants, vulnerability and asset management, identity and access management, detection engineering, deployment, and more. With so many great links and ideas distilled in one place, Rami knows how to my speak my love language 🥰, and I’m honored that his talk was inspired by my BSidesSF talk.

💡 Also, Rami has moved to Sweden and is open to new opportunities, after writing roughly 10 blog posts per hour on his sabbatical. I’m a HUGE fan of Rami’s work, and I know he’d be an incredible asset on any team. Feel free to slide into his DMs on LinkedIn.

The Cloud is Darker and More Full of Terrors
The blog version of Chris Farris’ Sec-T talk walking through a number of major cloud security incidents, highlighting both customer failures and cloud provider shortcomings. I’ve read about many of these incidents before, but the narrative walking through all of them in one place + historical context was nice. Chris argues that cloud providers should take more responsibility for customer security, and offers some nice 🌶️ throughout.

At AWS, Security is Job Zero, but:

It’s not job zero to ensure that their software is compatible with the new security features that are released. (Cloudwatch logs didn’t support IMDSv2 for several years).

It’s not job zero to set reasonable secure defaults for customers. Not until enough customers have high-profile security incidents, then it becomes Job Zero to protect AWS’s image.

Blue Team

Acquiring Malicious Browser Extension Samples on a Shoestring Budget
Neat walk through by Pepe Berba on finding malware samples and live infrastructure: you can search the hashes of samples (as listed on write-ups) on MalwareBazaar or VirusTotal, use features to search for similar samples (like file names or directory structure), if a URL that used to serve the malware is no longer live, you can see if it’s cached via a site like urlscan, or if it has a unique extension (e.g. .bs64) you can also search urlscan for that.

Pepe then walks through reversing acquired samples to find URLs, live infrastructure, and other useful indicators. A fun read 👍️ 

Prioritizing Detection Engineering
“Detection is a problem I describe as deceptively tractable.” I like Ryan McGeehan’s (aka Magoo) opinionated take on how and when to build out a detection engineering program, with a focus on not creating too much work for your or other teams, and fitting in within the broader eng team. Ryan’s proposed implementation order:

  1. Get logging in order, focusing on query-ability and minimum viable logs.

  2. Spend time on hardening before formalizing detection.

  3. Introduce high-quality detections and alerts, starting with a reference alert and focusing on invariants.

  4. Address management challenges before scaling detection efforts.

  5. Fully embrace an engineering approach to detection, with the ability to throttle or accelerate work as needed.

Red Team

Meckazin/ChromeKatz
A toolset for dumping cookies and credentials directly from the memory of Chrome and Edge browsers, bypassing the need for DPAPI keys or touch on-disk database files.

shaddy43/BrowserSnatch
By Shayan Ahmed Khan: A tool to steal saved passwords, cookies, bookmarks, and browser history from Chromium-based browsers (Edge, Chrome, Opera) and Gecko-based browsers (Firefox, Thunderbird).

JohnHammond/recaptcha-phish
PoC project by John Hammond of a phishing technique that mimics a reCAPTCHA form to trick users into executing malicious commands via the Windows run dialog, a method seen in recent "ClickFix" or Emmenhtal campaigns.

AI + Security

Opt out from all supported AWS AI services
With one button click or by attaching the provided example policy.

LinkedIn Is Training AI on User Data Before Updating Its Terms of Service
Great reporting by 404 Media’s Joseph Cox. By default, your LinkedIn data is opted in for training their content creation AI models. Disable it by: Settings → Data Privacy → turn off “Data for Generative AI Improvement.”

CyBench
A benchmark for evaluating the cybersecurity capabilities and risks of language models, by Stanford’s Andy Zhang et al. The benchmark includes includes 40 CTF tasks from 4 distinct recent competitions, spanning a range of difficulties, and includes subtasks, which break down a task into intermediary steps for more gradated evaluation. The tasks cover crypto, web, reverse engineering, forensics, exploitation, and misc areas.

Overall the models performed mediocre, peaking at Claude 3.5 Sonnet solving 17.5% without subtask guidance. Subtasks % solved reached up to 44% for Sonnet and 47% for OpenAI’s o1-preview.

XBOW releases a unique set of benchmarks to test AI offensive capabilities
Nico Waisman announces XBOW’s release of 104 novel benchmarks they had created by several pen testing companies, covering a range of vulnerability classes (SQL injection, IDOR, SSRF, etc.), for the purposes of testing AI’s offensive capabilities.

💡 I love that security researchers and companies are creating and sharing benchmarks. That goes a long way in proving an AI system (or any security tool) Actually Works™️ . Of course, no company is going to release a benchmark they perform poorly on, and I could write a 10+ page post on nuances in benchmarks, but it’s certainly directionally a good thing.

I’d be interested to see if Company A and Company B both create benchmarks, and they point their systems at the other company’s benchmarks, how well do they perform? Do the systems generalize or are they somewhat “tuned” for their own benchmark?

AI

Learning to Reason with LLMs
OpenAI has launched o1, a new model that produces a long internal chain of thought before answering. It seems to perform better on math and coding, not necessarily in all areas.

💡 It’ll be interesting to see the impact of various improvements beyond just “train on more data and longer and build bigger models.”

Detailed thread by Terence Tao, widely regarded as one of the greatest living mathematicians, on GPT-o1’s reasoning ability.

Pietro Schirano shares some prompting techniques around guiding a model’s reasoning.

Misc

Breaking Down OnlyFans’ Stunning Economics
In 2024, OnlyFans generated $6.3 billion in gross revenues, up from $300 million five years earlier. In 2023, payments to creators exceeded $5.3B (8% more than total NBA payroll). The company counted an average of only 42 employees (+ hundreds of contractors) in 2023, generating $31MM in net revenue per employee (13-28x that of Amazon, Apple, Google, and Microsoft). Catering to loneliness is profitable 😢 

✉️ Wrapping Up

Have questions, comments, or feedback? Just reply directly, I’d love to hear from you.

If you find this newsletter useful and know other people who would too, I'd really appreciate if you'd forward it to them 🙏

Thanks for reading!

Cheers,
Clint
@clintgibler