• tl;dr sec
  • Posts
  • [tl;dr sec] #162 - Meaningful Security Product Metrics, Vulnerability Inbox Zero, Joe Sullivan Trial Deep Dive

[tl;dr sec] #162 - Meaningful Security Product Metrics, Vulnerability Inbox Zero, Joe Sullivan Trial Deep Dive

How to justify the value of your security team's investments and prioritize, how to build an Inbox Zero vulnerability management approach, Magoo's detailed blameless post-mortem of USA vs Joe Sullivan.

Author

Clint Gibler
December 14, 2022

Hey there,

I hope you’ve been doing well!

🎄 Greetings of the Season

I’m back in Cincinnati for the holidays, where I mostly grew up.

For non-American (and East or West Coast) readers: Cincinnati is in Ohio, which is part of the “Midwest.” Feel free to share that fun fact over the dinner table and impress your friends.

I’m looking forward to going to the gym with my brother, the boxing club with my sister, and eating some high protein desserts (#Swolemas2022). And making cookies from scratch of course.

We also have an annual tradition of buying a slightly too big Christmas tree and putting on ornaments together.

I hope you have the opportunity to relax, recharge, and spend some quality time with family and friends.

Next tl;dr sec will be in 2023

I’m taking the next two weeks off- this will be the last tl;dr sec issue until the new year.

Dear security researchers and newsletter authors, if you want to also take a break, so I don’t feel compelled to read what you put out, feel free 😅

Also, I have a few things I’m excited to launch next year that I can’t wait to share with you 🎁 Keep an eye out.

Happy holidays!

Sponsor

📢 Forget everything you know about SSH

Say hello to Tailscale SSH — and say goodbye to managing SSH keys, setting up bastion jump boxes, and unnecessarily exposing your private production devices to the open internet. Never deploy an infrastructure bastion again.

SSH from mobile devices, and across OSes. Tailscale SSH works where Tailscale works. Code from an iPad to your Linux workstation, without having to figure out how to get your private SSH key onto it. Answer an on-call emergency from anywhere, which means you can leave your desk now.

📜 In this newsletter...

  • Conferences: Black Hat Europe 2022 slides, Global AppSec EU 2022 Virtual playlist, BSidesSF CFP is open

  • Podcasts: Jason Haddix's pen test stories on Darknet Diaries, I join 404 - Security Not Found episode 2

  • AppSec: A Rust CTF, lessons learned from Tailscale's Soc 2 Type II audit, reaching vulnerability inbox zero

  • Infrastructure as Code (IaC): A tool for determining the permissions or policy required for IaC code, infrastructure diagrams as code

  • Politics / Privacy: Apple's new advanced security features, introducing passkeys in Chrome

  • Cloud Security: GCPGoat/AWSGoat/AzureGoat, 2022 Hacking the Cloud wrap-up, a tenant isolation framework for cloud apps, authenticating external apps in a machine-to-machine scenario, visualizing multi cloud IAM concepts, AWS ECR Public Vvulnerability, top security talks from AWS re:Invent 2022

  • Misc: Consoles and Competition, impactful book list, everyone is sick right now, 20 of Charlie Munger's favorite books, a blameless post-mortem of USA v. Joseph Sullivan

  • SBOM: Tool to create a CycloneDX SBOM for source and container image deps across many languages, tool to validate SBOMs against versioned JSON schemas, GitHub Action to create an SBOM, the SBOM lifecycle, big tech vendors object to US gov't SBOM mandate

  • Tracking Meaningful Security Product Metrics: How to justify the value of your security team's investments and prioritize appropriately

Conferences

Black Hat Europe 2022
Some slides, whitepapers, and source code posted.

BSidesSF 2023 CFP is Open
BSidesSF will take place from April 22-23, 2023 and the CFP for talks and workshops will on January 8, 2023. The CFP for Workshops and Villages is also open.

  • Topics: All topic areas related to reliability, application security, web security, network security, privacy, cryptography, and information security are of interest and in scope.

  • Theme: Space: Putting the Cyber in Space!, to commemorate the new James Webb Space Telescope while paying homage to “cyberspace”.

BSidesSF is one of my favorite cons- I highly encourage you to submit! I’ll almost definitely attend, hope to see you there 🙌

Podcasts

Darknet Diaries Ep130: Jason’s Pen Test
Jason Haddix joins Jack Rhysider to share his funny and enlightening stories about breaking into buildings and computers, and talks about the time he discovered a major security flaw in a popular mobile banking app.

404 - Security Not Found: Episode 2
I joined my buds Travis McPeak, Anna Westelius, and Leif Dreizler on the second episode of the hit podcast. We discussed the Joe Sullivan trial, Rob Joyce memes, the recent OpenSSL bug, what’s happening at Twitter, and FTC goings-on.

AppSec

CTF - Puget Sound
A challenging Rust CTF by Nathanial Lattimer.

I wanted to set out to not only build a challenging Rust CTF challenge, I wanted to defy expectations, teach something novel, and reshape a participant’s understanding of Rust even for long-time Rustaceans.

 

What we learned (and can share) from passing our SOC 2 Type II audit
Tailscale’s David Anderson, Denton Gentry, and Maya Kaczorowski describe the challenges they faced with their audit, open source tools they’d like to share, and how they think their SOC 2 compliance efforts can be improved. See also:

Vulnerability Inbox Zero
A summary of LaunchDarkly’s Alex Smolen and Jake Mertz’ great LocoMocoSec 2022 and QCon SF conference talks. Excellent thoughts on scaling vulnerability management, building an Inbox Zero pipeline, asset inventory, and even automatically generating required documentation for FedRAMP. Noice.

Infrastructure as Code (IaC)

JamesWoolfenden/pike
By James Woolfenden: A tool for determining the permissions or policy required for IaC code. Currently supports Terraform and multiple providers (AWS, GCP, Azure).

Terrastruct
A diagramming tool crafted to visualize software architecture. Proprietary, but uses the open source D2, a diagram scripting language that turns text to diagrams.

Politics / Privacy

Apple’s New Advanced Security Features Protect Your Sensitive Data
Features: iMessage Contact Key Verification (allows users to verify the identity of a contact before sharing sensitive information or engaging in secure communications), Security Keys for Apple ID, and Advanced Data Protection for iCloud.

Introducing passkeys in Chrome
With the latest version of Chrome, passkeys are enabled on Windows 11, macOS, and Android. On a desktop device you can also choose to use a passkey from your nearby mobile device (Android or iOS).

Sponsor

📢 Relying on Sampling to Classify Unstructured Cloud Data Creates Security Risk

When classifying unstructured cloud data, many solutions employ data sampling techniques to improve efficiency while decreasing accuracy, potentially leaving hidden sensitive information silently exposed to cyber threats.

Read our blog and learn how Open Raven performs complete scans of unstructured data efficiently and at scale without sacrificing accuracy and discovers sensitive data wherever it hides.

Cloud Security

ine-labs/GCPGoat
By INE Lab Infrastructure: A vulnerable by design infrastructure on GCP featuring the latest released OWASP Top 10 web application security risks (2021) and other misconfiguration based on services such as IAM, Storage Bucket, Cloud Functions and Compute Engine.

Note that INE also released AWSGoat and AzureGoat at BlackHat USA this year.

2022 Wrap-up - Hacking The Cloud
Nick Frichette reflects on some accomplishments for the site this year, along with some noteworthy updates. This is such an excellent resource, keep up the great work Nick et al!

PEACH - Tenant Isolation Framework for Cloud Apps
A new framework that can help companies ensure the security of their multi-tenant cloud apps, by Wiz’s Amitai Cohen et al, cloud providers, and industry colleagues. PEACH stands for Privilege Hardening, Encryption Hardening, Authentication Hardening, Connectivity Hardening, and Hygiene.

Amitai has a nice thread about it, and there’s also a whitepaper.

Approaches for authenticating external applications in a machine-to-machine scenario
The pros and cons of a number of approaches, including AWS Signature v4, mutual TLS, OpenID Connect, SAML 2.0, Kerberos, Active Directory, and IAM Roles Anywhere.

Visualizing Multi Cloud IAM Concepts
Julian Wiegmann created several detailed diagrams about key AWS, Azure and GCP IAM concepts and terminology.

AWS ECR Public Vulnerability
Lightspin’s Gafnit Amiga discovered a critical AWS Elastic Container Registry Public (ECR Public) vulnerability that allowed external actors to delete, update, and create ECR Public images, layers, and tags in registries and repositories that belong to other AWS Accounts, by abusing undocumented internal ECR Public API actions.

Top Security Talks from AWS re:Invent 2022
By Wiz’s Scott Piper (congrats on the new job Scott!).

  • Reimagining multi-account deployments for security and speed

  • Accelerate insights using AWS SDK instrumentation

  • Zero-privilege operations: Running services without access to data

  • When security, safety, and urgency all matter: Handling Log4Shell

  • Context is everything: CNAPP revolution to secure AWS deployments

Misc

Consoles and Competition
Some fascinating history about the video game industry by Stratechery’s Ben Thompson, to put FTC vs Microsoft in context.

My Must Read List
Mike Privette’s continuously updated list of books that have helped him the most throughout his career and life (so far).

Everyone Is Sick Right Now
For the past two years, social distancing kept seasonal viruses at bay. Now they’re roaring back.

20 of Charlie Munger’s favorite books
Charlie Munger is a modern-day polymath and one of the most respected investors of all time.

A blameless post-mortem of USA v. Joseph Sullivan
Epic in-depth analysis by Ryan McGeehan, in which he combed through every word of testimony in the trial, and offers a blameless, objective description of what exactly happened and lessons that can be learned- technically, organizationally, and legally. This blog should be considered the definitive description of what happened.

SBOM

AppThreat/cdxgen
Creates a CycloneDX SBOM containing an aggregate of all project dependencies from source and container images. Currently supports C/C++, Node.js, PHP, Python, Ruby, Rust, Java, .Net, Dart, Haskell, Elixir, and Go projects in XML and JSON format. Integrate in your CI/CD pipeline with automatic submission to Dependency Track server.

IBM/sbom-utility
Utility that provides an API platform for validating, querying, updating and managing standardized SBOMs. Can be used to validate CycloneDX or SPDX SBOMs (encoded in JSON format) against versioned JSON schemas as published by their respective organizations.

SecureStackCo/actions-sbom
A GitHub Action that creates an SBOM from your application so you can meet compliance and security requirements. It can also include cloud resources, vendor dependencies and partner APIs you call. Note: requires a SecureStack API key.

The SBOM Lifecycle
SecureStack’s Paul McCarty describes the SBOM lifecycle as having 5 stages: asset discovery, application data analysis, SBOM creation, SBOM storage and SBOM searchability.

Big Tech Vendors Object to US Gov SBOM Mandate
By Ryan Naraine: Basically the big tech vendors are arguing that there needs to be more standardization and other maturing before SBOMs can be a reasonable contractual requirement, and agencies aren’t really ready to consume them anyway.

…practical challenges related to implementation, including naming, identification, scalability, delivery and access, the linking to vulnerability information, as well as the applicability to cloud services, platforms and legacy software.

The tech vendors also flagged concerns around the security of sensitive proprietary information that may be collected via SBOMs and held by federal agencies and called for clarifications around the definition of artifacts and what protections will be afforded to safeguard sensitive information.

Great post by Segment’s Leif Dreizler on why security product (things your security team builds) metrics are important, example metrics to track, and some concrete examples.

  • Tracking metrics helps you track improvements over time and/or rationalize maintenance costs.

  • Helps you know which initiatives or bug fixes to prioritize.

  • For health metrics, consider stealing what other teams in your company are already doing.

  • Put on your product manager hat and think about what indicators of success you want to track. Every service your team owns should have at least one product metric.

  • Track time savings for previously manual tasks you’ve automated (how long it took to do manually x number of times it occurred)

Example metrics:

  • Customer annual recurring revenue (ARR) associated with a feature

  • % or total number of customers using a feature

  • Weekly users of feature or internal tool

  • Number of times a tool completed a task

For better or worse, part of everyone’s job is marketing. Having even basic numbers really makes your work stand out during quarterly check-ins, performance reviews, or on your resume.

Always tie metrics to customer ARR when available. This type of thinking helps you break people out of the “security is a cost center” mindset.

✉️ Wrapping Up

Have questions, comments, or feedback? Just reply directly, I'd love to hear from you.

If you find this newsletter useful and know other people who would too, I'd really appreciate if you'd forward it to them 🙏

Thanks for reading!

Cheers,

Clint