[tl;dr sec] #22 - Post AppSec Cali, K8s Security Monitoring at Scale

Hey,

AppSec Cali 2020 was a blast! I met some new cool people, caught up with friends, and got asked more than a few times if I was going to write summaries for all of the talks next year 😅 We’ll see, no promises.

People at the conference as well as on Twitter and LinkedIn after were quite kind and positive about my talk. Like the Grinch, my heart grew three sizes. You can check out my slides here and a Twitter overview here.

🗣️ Feedback Requested: High Leverage Security Engineering Projects?

One of my favorite topics is hearing about neat systems and processes people have built that have had huge security ROI. For example, RepoKid for automating IAM least privilege and Dropbox’s proxy to protect internal web apps.

I’m giving talks at BSidesSF and RSAC in a few weeks that are a continuation of my effort to aggregate and distill scalable modern security best practices.

If your company has built some baller systems or processes that have prevented or mitigated classes of vulnerabilities, or otherwise significantly raised your security posture, I’d really appreciate you emailing me a description or linking me to a talk or blog post about it! 🙏

I’ll cite you and your company if it gets included in the talk, and I’m planning to write a blog post that describes all these projects in one place, like a cookbook manual of cool, impactful things you can build.

AppSec Cali 2020

Over the next few weeks and months more slides and videos will be released. Here are some I’ve come across so far.

DevSecOps Talks

• Purple is the new black: Modern Approaches to Application Security by Tanya Janca

• The Security Phoenix raises from DEV-OPS ashes by Francesco Cipollone

• An Opinionated Guide to Scaling Your Company’s Security by yours truly 😀

Tools & Misc

Introducing OWASP Nettacker by Sam StepanyanHere’s a demo video of it detecting Citrix CVE-2019-19781 and a list of features on its repo on GitHub:

• Python Multi Thread & Multi Process Network Information Gathering Vulnerability Scanner

• Service and Device Detection ( SCADA, Restricted Areas, Routers, HTTP Servers, Logins and Authentications, None-Indexed HTTP, Paradox System, Cameras, Firewalls, UTM, WebMails, VPN, RDP, SSH, FTP, TELNET Services, Proxy Servers and Many Devices like Juniper, Cisco, Switches and many more… )

• Asset Discovery & Network Service Analysis

• Services Brute Force Testing

• Services Vulnerability Testing

• HTTP/HTTPS Crawling, Fuzzing, Information Gathering and more

Kompar.tools
A site that aims to provide a data-driven, detailed break down of static analysis tools based on cost, license, supported languages and frameworks, performance, and so forth. Described by Chris Horn in his talk Choosing the right static code analyzers based on hard data.

videos.owasp.org
A more streamlined interface to OWASP videos, built by Jonathan Marcil and others. You can also contribute to the source code here.

Cloud / Containers

Enumerating Docker Registries with go-pillage-registries
Tool release by NCC Group’s Josh Makinen that, given a Docker registry, will enumerate all repos and their tags using Docker’s /v2/_catalog and /v2//tags/list APIs. pilreg was built because on a number of engagements, Josh gained read access to private Docker registries, from which he could extract useful info from images, such as secrets compiled into the container during build time, app logic, config files, etc. For each image pilreg finds, it automatically requests their manifest and configuration and outputs the results code 

KubeCon NA 2019 CTF
Materials for “Attacking and Defending Kubernetes Clusters: A Guided Tour Walkthrough Guide” that was presented at KubeCon NA 2019. There are several challenges that alternate between an attack and defense perspective. (Thx Marco Lancini)

Aligning to the NIST Cybersecurity Framework in the AWS Cloud
Whitepaper from AWS on AWS services that enable alignment with the CSF core functions: Identify, Protect, Detect, Respond, and Recover. Dropbox’s Hongyi Hu called out the NIST CSF as being a really useful framework for planning your security engineering investments during his AppSec Cali 2019 talk. See here for more details.

AWS Security Documentation
A link to AWS’s security documentation for various services all on one page.

Kubernetes Security monitoring at scale with Sysdig Falco

Nice blog post from Christian Martorella and Nacho Rasche about how Skyscanner used Falco to monitor >2,000 nodes across 30 different clusters. “Falco is a container Native Runtime Security solution focused in Intrusion and abnormality detection which uses the open source linux Kernel tooling built by Sysdig to generate alerts based on a custom rules and macros engine.”

Falco comes with powerful built-in default rules that can detect things such as a shell being run in a container, a server process spawning a child process of an unexpected type, and a sensitive file, like /etc/shadow, being unexpectedly read.

The post has some nice details about how and why they deployed Falco the way they did. Once they tuned down the initial noisy alerts, they started observing interesting behavior about their environment, such as some containers running as root.

To determine the owner of a pod when alerts fire, they use a system they’ve built that collects ownership and resource information about each service, obtained by scanning the metadata available in the project’s repo, their AWS accounts and a few other places, such as data received from their deployment pipeline.

Skyscanner has over 5,000 services and the security team is 22 people- how do you scale?

They have a ChatSecOps bot that scans received data (in this case, Falco findings + info added by their ownership mapping tool) and contacts the resource owner to inform them of the event. They’re working on having the bot ask the owner if they indeed initiated the action, and if the owner says ‘No,’ it’ll trigger their incident response process.

Web Security

huntr.dev
“The first bug bounty board for securing open-source code,” launches publicly Feb 1, 2020. I like the idea of bounties for fixing, not just finding bugs, but I’m not sure how effective this will be in practice for a few reasons: a) finding and fixing require different skills, b) fixing well can require significant contextual understanding of a code base, that a non core dev won’t have, and c) will programs be willing to pay an amount that will incentivize well-written fixes (vs. just band-aids)? Still, I think this is exciting and I’m looking forward to seeing where it goes.

Use GitHub actions at your own risk
Any repo maintainer of a popular GitHub action can change what a branch or tag points to in the future, for example, to insert malicious functionality, such as stealing provided API tokens. To prevent this, specify the exact version you mean (action@sha1). (Thx Marco Lancini)

Exploiting email address parsing with AWS SES
There are certain ambiguities in email specification RFCs such that depending on the language and parsing function used, different email addresses may be returned (e.g. ([email protected])[email protected]). This blog post shows how in some (infrequent) situations you can bypass account email validation/verification by providing an email address that a frontend web app interprets to have one domain while a backend service returns a different one (e.g. AWS SES).

Static Analysis (Python)

pschanely / CrossHair
If you have functions with type annotations and you add some checks in the (defunct) PEP 316 syntax, CrossHair will attempt to find counterexamples by repeatedly calling functions with symbolic inputs and using an SMT solver to explore viable execution paths.

Pyre / Pysa from Facebook
There are now some docs on using Pysa for taint analysis (data flow analysis) to find when user input reaches dangerous functions (e.g. SQL injection, command injection, etc.) The docs describe how you can specify the sources and sinks you care about, as well as sanitizers to reduce false positives. There’s also a Pysa tutorial here.

Misc

Starlink is a Big Deal
SpaceX being able to cheaply launch low orbiting satellites may allow Starlink (a subsidiary of SpaceX) to provide fast Internet (600 MBPs is referenced) globally, putting 4 billion more people online. Some interesting predictions about the outcomes of this at the end.

InfoSec Skills Matrix
A chart that visually shows what skills you need (e.g. presentation skills, network fundamentals, web app basics) for which security roles (e.g. pen tester, infrastructure specialist). Useful resource for people thinking about getting in to security or switching to a different speciality. (Thx Daniel Miessler)

VirusTotal Graph++
Blog post on updates to the VirusTotal Graph, a “tool which understands the relationship between files, URLs, domains and IP addresses, and an easy interface to pivot and navigate over them.” Documentation for the API can be seen here and a Python client for the API here. (Thx Marco Lancini)

Bezos’s Phone

As mentioned in tl;dr sec 21, there was quite the hullabaloo about accusations of the Saudi crown prince (MBS) hacking Bezos’s phone by sending him a malicious file over WhatsApp. The initial forensic investigation said, “due to end-to-end encryption employed by WhatsApp, it is virtually impossible to decrypt the contents of the downloader…” Predictably, the Internet said, “Hold my beer.”

Dino Dai Zovi released some PoC scripts that can do just that, and Rob Graham wrote a nice blog post walkthrough of how to do it with PoC code as well. Rob also wrote an overview article for non technical users.

Politics / Privacy

NIST releases version 1.0 of the Privacy Framework
The framework has 3 main parts:

• The Core enables communications within organizations about privacy protection activities and desired goals.

• Profiles allow organizations to prioritize the outcomes and activities according to privacy values, the business mission, and risks.

• Implementation tiers help organizations to optimize the resources that are necessary to manage the risk.

Read the full document here. (Thx Daniel Miessler)

Apple reveals worldwide national security requests for customer data

Full transparency report here 

How to Influence Engineering Teams

Great thread by Scott Berkun on influencing people as a designer. Try reading it and replacing “designer” with “security engineer,” as in both cases we’re often trying to influence engineering teams:

✉️ Wrapping Up

Have questions, comments, or feedback? Just reply directly, I'd love to hear from you.

If you find this newsletter useful and know other people who would too, I'd really appreciate if you'd forward it to them 🙏

Thanks for reading!

Cheers,

Clint