• tl;dr sec
  • Posts
  • [tl;dr sec] #17 - AWS Re:Invent, Supply Chain Security, Absolute AppSec

[tl;dr sec] #17 - AWS Re:Invent, Supply Chain Security, Absolute AppSec

A number of interesting new AWS services, backdoors on PyPI, I had a blast on the Absolute AppSec podcast, and tl;dr sec hits 500 subscribers! 🚀

Hey there,

I hope you’ve been getting some nice time to relax as the holidays approach.

I have, except for when I’ve been mentally taxed by important life decisions, like which type of egg nog to buy. Should I keep it classy with original? Embrace a modern take with vanilla cinnamon? Go full basic with pumpkin spice? A single bead of sweat dropped from my face when I realized I was going to have to live with this decision.Permalink to this tl;dr sec issue on our blog here.

tl;dr sec surpasses 500 subscribers! 🚀

Whoa. I’m honored and humbled by this 🙏. Thank you.

Now time to get back to writing, until the subscriber count causes me to crush my scouter.

Interviewed on the Absolute AppSec Podcast

I had a blast being on Absolute AppSec #77 with Seth Law and Ken Johnson. We talked about how I got into security, DevSecOps, how a number of companies are scaling their security (Ken shared some neat things GitHub has done), the differences between research in industry and academia, and a number of other topics.

You can check it out on the Absolute AppSec homepage or wherever podcasts can be found.

📜 In this newsletter...

🔗 Links:

  • Tools: DNS covert channel implant for red teams, coverage-guided blackbox fuzzer based on Frida, data science framework by Netflix

  • Shiny AWS Services: RDS proxy for Lambdas, enterprise search + ML, analyze log data + ML, automatically create high quality ML models, automated code review using ML, analyze IAM policies for unintended resource accesses

  • Cloud Security: monitoring anomalous app behavior via CloudTrail logs, tool to find wasteful/unused AWS resources.

  • Guardians of the Cloud: how Auth0 uses security automation to respond to GuardDuty events at scale

  • Supply Chain Security: Two typosquatting PyPI packages, study on typospuatting various package managers

  • Misc: HackerOne leaked session cookie to researcher, the social subsidy of angel investing, practical AI walkthroughs

  • Humor: John Oliver on SLAPP suits, Warren Buffet handing out stacks of cash

  • Twitter: insightful, practical thread on the art of influencing

Tools

WEASEL - a DNS covert channel implant for red teams built by Vlad Ionescu, a friend and former colleague who’s now on Facebook’s red team. Small (5.2KB minified), uses Python 3 with no dependencies.

Frizzer - A coverage-guided blackbox fuzzer based on the Frida instrumentation framework.

Metaflow - a framework for data science by Netflix.

Shiny AWS Services

There is not a complete list of new AWS offerings, but here are a few that caught my eye.

If there’s one subtle trend that you’ll be able to see in many of the following services, it’s that Amazon is leaning hard into machine learning - applying it to new offerings and trying to make it easy for other companies to reap its benefits without requiring significant in-house expertise.

I never expected to create a file called oprah_machine_learning.jpg, but here we are.

Using Amazon RDS Proxy with AWS Lambda
To keep your Lambdas from overwhelming the relational database you’re pointing them at, you can instead have them connect to this RDS Proxy, which will manage the connection pools.

Amazon Kendra - Enterprise Search Service
“Delivers powerful natural language search capabilities to your websites and applications so your end users can more easily find the information they need within the vast amount of content spread across your company.” Lets users ask natural language questions rather than keywords, allows you to add content from file systems, SharePoint, intranet sites, file sharing services, and more so you can search all of the content in one place, and search results improve over time as Kendra’s ML algorithms learn which results users find most valuable.

Amazon Detective
“Makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities. Amazon Detective automatically collects log data from your AWS resources and uses ML, statistical analysis, and graph theory to build a linked set of data that enables you to easily conduct faster and more efficient security investigations.” Can analyze trillions of events from data sources such as VPC Flow Logs, CloudTrail, and GuardDuty, and automatically creates a unified, interactive view of your resources, users, and the interactions between them over time.

Amazon SageMaker Autopilot – Automatically Create High-Quality Machine Learning Models With Full Control And Visibility
SageMaker Autopilot first inspects your data set and runs a number of candidates to figure out the optimal combination of data preprocessing steps, machine learning algorithms and hyperparameters. Then, it uses this combination to train an Inference Pipeline, which you can easily deploy either on a real-time endpoint or for batch processing. Lastly, it also generates Python code showing you exactly how data was preprocessed, so you can understand what it did and reuse that code for further manual tuning if you want.

Potential game-changer?
If this works well in practice it would be an impressive advance into two common challenges to the widespread effective use of ML: understanding what it’s doing, and having the domain expertise to know which algorithms and hyperparameters are best for your use case.

Amazon CodeGuru
An ML service for automated code reviews and application performance recommendations. It helps you find the most expensive lines of code that hurt application performance and cause subtle errors, then gives you specific recommendations to fix or improve your code. Powered by machine learning, best practices, and hard-learned lessons across millions of code reviews and thousands of applications profiled on open source projects and internally at Amazon. Currently supports Java, more languages in the works.

CodeGuru Reviewer is trained using rule mining and supervised machine learning models that use a combination of logistic regression and neural networks. During training to detect deviation from best practices, it mines Amazon code bases for pull requests that include AWS API calls. It looks at code changes and cross-references them against documentation data, which it also mines in parallel. This creates new models for best practices that Reviewer uses when it reviews your code to provide recommendations. CodeGuru Profiler is also trained by Amazon performance engineers and used to profile tens of thousands of services used internally at Amazon.

A PR with 500 lines of code costs $3.75 to scan. CodeGuru costs $0.005 per sampling hour per application profile and $0.75 per 100 lines of code per month.

Finds issues like:

  • Excessive recreation of expensive objects, expensive deserialization, usage of inefficient libraries, and excessive logging.

  • Deviations from best practices for using AWS APIs and SDKs, flagging common issues that can lead to production issues, such as detection of missing pagination or error handling with batch operations.

  • Concurrency issues such as atomicity violations and the use of classes that are not thread-safe.

  • Identifies correct input validation, where un-sanitized inputs can lead to issues such as injection attacks or denial of service.

Security applications?
Though CodeGuru seems currently mostly focused on code robustness and performance, there’s a clear security play here. It will be interesting to see how CodeGuru progresses - is it effective on non-Amazon code? What’s the false positive rate like?

Identify Unintended Resource Access with IAM Access Analyzer
“Mathematically analyzes access control policies attached to resources and determines which resources can be accessed publicly or from other accounts. It continuously monitors all policies for S3 buckets, IAM roles, Key Management Service (KMS) keys, Lambda functions, and Simple Queue Service (SQS) queues. Gives you visibility into the aggregate impact of your access controls, so you can be confident your resources are protected from unintended access from outside of your account.”

Cloud Security

AWS re:Invent - Monitoring anomalous application behavior
Travis McPeak and Will Bengston describe Netflix’s system, using off-the-shelf AWS components, to track CloudTrail activity across multi-account environments and send alerts when applications perform anomalous actions. Specifically, it keeps track of all API actions a principal calls (that are tracked by CloudTrail) for an N day period and then alerts on new API calls after the N day period. (Source code)

similarweb/finala - A resource cloud scanner that analyzes and reports about wasteful and unused resources to cut unwanted expenses. Uses YAML definitions (no code) and currently supports AWS resources (RDS, EC2 instances, DynamoDB, ElasticCache, documentDB, ELB and etc).

In tl;dr sec #11 we saw Twilio release SOCless, their serverless framework for running SecOps runbooks at scale, and Dropbox discussed their automation around threat detection and IR built on Jupyter notebooks.

In this article, Alejandro Ortuno describes how Auth0 uses security automation to respond to GuardDuty events at scale.

  • GuardDuty is enabled at AWS account creation in all regions. Every new account becomes a member of the master GuardDuty account, and a combination of regional CloudWatch event rules and Lambda functions ship all GuardDuty events into their SIEM.

  • They monitor specific GuardDuty API calls (such as Disassociate, StopMonitoring, DeleteDetector, etc.) that could cause gaps in their alerting infrastructure. If observed, their team is paged.

  • They’ve subscribed to the GuardDuty Announcements SNS Topic and their Slack bot notifies them when there are new GuardDuty findings or updates to existing ones that require their attention.

The rest of the article describes Auth0’s Slack SecurityBot, their AWS incident response tool, a number of GuardDuty tunings they’ve made to make the alerts more tractable, and how they’ve automated their response for certain classes of findings (EC2 PortProbe alerts, IAM User alerts, and EC2 Other alerts).

Thanks Marco Lancini for the heads up on this article.

Supply Chain Security

Two malicious Python libraries caught stealing SSH and GPG keys
Two typosquatting PyPI packages were caught trying to steal SSH and GPG keys. The malicious functionality downloaded a file from a repo hosted on Gitlab, decoded it into a Python file, and then executed it.

Typosquatting programming language package managers
A 2016 blog post describing Nikolai Tschacher’s thesis in which he investigated the feasibility of typosquatting attacks on Python, NodeJS, and Ruby packages. He wrote mini backdoor PoCs that would phone home to a server he controlled, and after publishing the packages he observed requests from over 17,000 distinct IPs, and over 40% of the unique IP addresses executed the program with administrative rights.

Misc

While discussing a finding, a HackerOne analyst shared a cURL command with a researcher that included a valid session cookie for the HackerOne platform, enabling the researcher to read private reports from other companies. HackerOne now binds session cookies to IP addresses. Ars Technica | HackerOne report

This attack isn’t that interesting from a technical point of view, but the impact is. Honestly, similar to supply chain attacks, I’m surprised that attackers don’t target companies with lots of bugs more often, like bug bounty platforms or consulting firms (😅). Finding bugs can be hard work man, why not let someone else do that for you and then just steal their bugs?

The Social Subsidy of Angel Investing
This article argues that Angel Investors in the Bay area often do it more for the social status rather than the financial returns (seeing value in a company that your peers didn’t), and that this provides important early capital to founders on terms they generally couldn’t get in other regions where angels have whacky priorities, like maximizing likely financial returns.

practicalAI
A series of hands-on, practical walkthroughs that discuss a concept and then implement it using popular tools like NumPy, TensorFlow, and Keras. Uses Google Colaboratory, which is a free Jupyter notebook environment running in the cloud. Nifty!

Humor

Here are two things that made me laugh out loud this week.

SLAPP Suits: Last Week Tonight with John Oliver
SLAPP Suits are spurious lawsuits filed with the intention to intimidate a critic. John Oliver discusses a lawsuit his show had to deal with because of one of their episodes. Worth watching the whole thing, or skip to 20:45 for the epic finale.

Twitter

Another excellent thread from Phil Venables, this time on The Art of Influencing. The thread is a bit long, but has some insightful perspective, so I’ve included it in its entirety:

A critical measure of success for most security roles is the ability to influence. I’ve often found people think influence skills are innate - you have them or you don’t. But, as with most “soft skills”, they can be learnt.

  1. Be very clear on the outcome you want. Write it down in a clear way. I love the Amazon press-release technique. If you cannot clearly state what it is you want then you’ve no chance of influencing others - except in a bad way.

  2. Understand the current situation: data, motivations, people, and environment. Do some research to find out as much as you can about the “system” you’re seeking to change. Often, what you want to do will have been tried before in various ways that you can learn from.One of the anti-patterns of influence are people new to an organization who can’t believe everyone is so dumb as to not fix something without finding out people have been working like crazy to do just that - but there’s a whole iceberg of issues under the surface.

  3. Understand the “forces” that keep a situation current. Don’t be frustrated at something not changing. Instead, be amazed - in a complex world - that a situation is not already changing. Develop a better understand of the forces and counter-forces that keep it that way.Sometimes, if you want change something, the best thing is not to add more force, but rather remove a head-wind. There’s a great part of Kurt Lewin’s work in social science called Force Field Analysis that is worth reading up on.

  4. Use thinking tools to find the core of an issue - increasing the depth of your insight shows your commitment to the goal and also educates and improves those around you. They’ll remember you for that. Look for power laws (where 80% of outcome can come from 20% of work).

  5. Plan the change of mindset. To influence is to cause a change - that often needs some change of mindset or priorities. This needs planning, like a campaign with the messages and medium orchestrated across the right people, moments, venues and governance constructs.For example, if you’re going into a committee/management group with a proposal and you don’t have high confidence that all in that room are going to support you, then you’ve not done the prep and campaigning needed to get people on side.Understand the social archeology of an organization. Put yourself in the shoes of the people who will be affected by your proposal. Know who influences the decision makers, many have a trusted lieutenant, perhaps now in another part of the organization - get to them.

  6. Communicate - in person and in various internal media. Appeal to people’s motivation and objectives. Find ways such that what you want will also help provide adjacent benefits or satisfy the commercial goals of the people whose help you need.In my experience, even if your case for adjacent benefits is not totally compelling the other person will appreciate you trying and be more likely to help. Use behavioral science techniques in how you communicate (e.g. social proof, story telling, branding).

  7. Present the message clearly and effectively - be precise and capture people’s imagination. Be as simple as possible (but not dumbed down). Be persistent - some problems are not ready to be solved - so be ready to re-present when the time or circumstance is ready.Have a “library” of proposals ready to go when you’ve had close-calls or incidents. Refine/practice your pitches. There’s nothing so frustrating for a senior decision maker (or anyone for that matter) to see people who have obviously spent zero time refining/rehearsing.

  8. Execute on your commitments. Establish credibility that people can trust you to get things done, on time, in budget and with care and attention for being a good team member. This is a force multiplier for you to influence in the future - you’re a good bet.

Bottom line: if you want to drive change you need to influence people. There are many techniques to do this, but the best meta-technique is to reflect on what works on you and when others have failed with you - adapt accordingly.

✉️ Wrapping Up

Have questions, comments, or feedback? Just reply directly, I'd love to hear from you.

If you find this newsletter useful and know other people who would too, I'd really appreciate if you'd forward it to them 🙏

Thanks for reading!

Cheers,

Clint