• tl;dr sec
  • Posts
  • [tl;dr sec] #163 - Rebuilding Detection and IR at LinkedIn, CVEs and Misaligned Incentives, 2022 in Review and 2023 Predictions

[tl;dr sec] #163 - Rebuilding Detection and IR at LinkedIn, CVEs and Misaligned Incentives, 2022 in Review and 2023 Predictions

How LinkedIn scaled detection and minimized toil, why ReDoS CVEs are mostly noise, and reflecting on security in 2022 and predicting what 2023 has in store.

Author

Clint Gibler
January 03, 2023

Hey there,

I hope you’ve been doing well!

Welcome to 2023 🎉

If you’re reading this, you have successfully survived until 2023. Congratulations!

I hope you enjoyed some good food and relaxing time with friends and family over the holidays.

I did lots of working out with my siblings (including a blacklight boxing session), played the very fun card game Dutch Blitz for the first time, saw snow, learned about new words the youth are using these days (like “baddie”), and ate lots of raw cookie dough. I’ve heard it’s potentially dangerous to eat raw cookie dough, but I’ve never gotten sick*.

If your company does annual reviews in December, I hope they went well! If not**:

* tl;dr sec is not your doctor and takes no responsibility if you also eat raw cookie dough.

** This is also called, “Extreme Programming.”

Annual Review - Resources You Like?

One thing I’ve wanted to do, but haven’t really done in the past, is an annual review.

To reflect on what went well last year and create a game plan for the upcoming year.

If you have any blog posts, videos, checklists, books, or other resources you like related to doing an annual review, please send them to me!

Next week I’ll share a collection of resources based on what I’ve found and what you share. Thanks in advance!

Sponsor

📢 Tailscale, now with more SSH

Stop managing SSH keys manually, setting up bastion jump boxes, and unnecessarily exposing private production resources to the internet.

Tailscale SSH is a new way to SSH into devices in your tailnet. Simply enable it for the host and source devices, and we’ll take care of the rest — from distributing keys to authenticating connections.

Tailscale SSH works everywhere Tailscale does, so your team can code from an iPad or answer on-call emergencies from wherever they are.

📜 In this newsletter...

  • Rain in San Francisco: It be crazy out there

  • Container Security: Tool to harden AWS EKS, learning by auditing Kubernetes manifests, attacker persistence in Kubernetes using the TokenRequest API

  • AppSec: Google's OSV scanner, safeurl for Go, turning Google smart speakers into wiretaps for $100k, ReDoS "vulnerabilities" and misaligned incentives

  • Cloud Security: How to use Amazon Verified Permissions for authorization, upcoming secure defaults for S3, how cloud pen testing is different, trends from AWS security bulletins

  • Blue Team: Open source Chronicle detection rules, making an SSH client the hard way, rebuilding threat detection and incident response at LinkedIn

  • Politics / Privacy: Sudden Russian Death Syndrome, reverse engineering Tiktok's VM obfuscation, TikTok spied on Forbes journalists

  • Misc: GIF Baskets, querying the GitHub archive with the ClickHouse playground, Obsidian Canvas, the future our grandchildren deserve, what comes next for SF's emptied downtown

  • Annual Review and Predictions: The State of Cybersecurity in 2022 and Trends and Predictions for 2023, Frontview Mirror: 2023 Edition

  • Inspiration: Don't let the Resistance stop you

Rain in San Francisco

San Francisco has recently had record rainfall, which has lead to some bonkers videos.

Container Security

aws-samples/hardeneks
Runs checks to see if an EKS cluster follows EKS Best Practices.

Learning by auditing Kubernetes manifests
Nicolas Fränkel walks through running Checkov on Kubernetes manifests and highlights some of the interesting findings.

Attacker persistence in Kubernetes using the TokenRequest API: Overview, detection, and prevention
The TokenRequest API can be used to create long-lived and hard-to-detect privileged access to Kubernetes clusters. Datadog’s Rory McCune outlines how this feature works, how attackers can abuse it, and how you can detect its misuse by monitoring Kubernetes audit logs.

Sponsor

📢 71% of security teams expect container security issues to rise.

Tools and processes utilized for traditional infrastructure are not adequate for container security so you may need to consider a different approach. Even simple container environments have at least seven sources of vulnerabilities, so securing them requires a systematic and consistent approach. Check out this guide on Building Secure and Compliant Containers to help you identify those vulnerabilities, establish objectives for your container security program, and ensure application of security best practices across every phase of container security.

AppSec

google/osv-scanner
Vulnerability scanner written in Go which uses the data provided by https://osv.dev. It finds all the transitive dependencies that are being used by analyzing manifests, SBOMs, and commit hashes.

safeurl for Go
Doyensec’s Alessandro Cotto and Viktor Chuchurski announce safeurl, a one-line drop-in replacement for Go’s net/http client with built-in SSRF and DNS rebinding protection.

See also this post by Include Security that describes SafeURL libraries they created for PHP, Python, and Scala that similarly protect against SSRF.

Turning Google smart speakers into wiretaps for $100k
Matt Kunze walks through his methodology in finding security issues in the Google Home smart speaker (earning a $107,500 bounty) that allowed an attacker within wireless proximity to install a “backdoor” account on the device.

Great detailed walkthrough of understanding how a device you own works, using tools like dns-sd, nmap, intercepting the Android app’s HTTPS traffic using mitmproxy, decoding protobuf messages with protoc, aireplay-ng, and more.

ReDoS “vulnerabilities” and misaligned incentives
Trail of Bits’s William Woodruff argues that most ReDoS “vulnerabilities” are indistinguishable from malicious noise because of:

  • Misaligned incentives in the security reporting (security researchers want fame) and vulnerability reporting ecosystems (supply chain security vendors want to differentiate).

  • They produce security fatigue in engineers by making them waste time on low impact bugs.

Programmers want to be delighted by their tools: they like tools that do the right thing by default, that nudge (but don’t prod) them in the right direction, that integrate into their existing workflows and want to be integrated, rather than demanding unique treatment.

The best tools reduce the amount of work an engineer needs to do; the second best tools don’t change the volume of work but make it more enjoyable.

Cloud Security

How to use Amazon Verified Permissions for authorization
AWS’ Jeremy Ware shows how to use Amazon Verified Permissions to define permissions within custom applications using the Cedar policy language.

Heads-Up: Amazon S3 Security Changes Are Coming in April of 2023
S3 will automatically enable S3 Block Public Access and disable access control lists for all nw buckets starting in April 2023. Secure 👏 defaults 👏 for the win!

Cloud penetration testing: Not your typical internal penetration test
Bishop Fox’s Seth Art walks through how doing an internal (assume breach) cloud pen test is different than a typical pen test, with examples of increasingly useful things to look for.

AWS Security Bulletins and Cloud Security Researcher Trends
Luke Tucker shares what he learned from reviewing 10-years of AWS Security Bulletins, focusing on cloud security research trends: more security researchers are testing the security of cloud providers, and AWS is doing a better job at acknowledging contributions from the researcher community.

Blue Team

Open Sourcing Chronicle Detection Rules
Algbra’s Mikail Tunç announces the release of a collection of detection rules for Google’s cloud-native SIEM, Chronicle. The detections target GitHub, Okta, Google Workspace and Slack; with AWS, Kubernetes and others coming soon.

Making an SSH client the hard way
Tailscale’s Mihai Parparita describes building a web-based SSH client, so your browser becomes a Tailscale client. They did this by porting to WebAssembly: the Tailscale client, WireGuard, a complete userspace network stack (from gVisor), and an SSH client.

(Re)building Threat Detection and Incident Response at LinkedIn
LinkedIn’s Sagar Shah and Jeff Bollinger describes how LinkedIn was able to reduce incident investigation times by 50%, increase threat detection coverage expansion by 900%, and reduce their time to detect and contain security incidents from weeks or days to hours.

Great example of thoughtful security engineering, automation, reducing toil, and more.

Politics / Privacy

Sudden Russian Death Syndrome
A number of rich and/or important Russians have had dangerous encouters with open windows or committed “suicide,” across a number of countries.

Some two dozen notable Russians have died in 2022 in mysterious ways, some gruesomely.

Reverse Engineering Tiktok’s VM Obfuscation (Part 1)
@blastbots walks through deobfuscating and understanding some of TikTok’s JavaScript.

An internal investigation by ByteDance, the parent company of video-sharing platform TikTok, found that employees tracked multiple journalists covering the company, improperly gaining access to their IP addresses and user data in an attempt to identify whether they had been in the same locales as ByteDance employees.

Forbes first reported the surveillance tactics, which were overseen by a China-based team at ByteDance, in October. Asked for comment on that story, ByteDance and TikTok did not deny the surveillance, but took to Twitter after the story was published to say that “TikTok has never been used to ‘target’ any members of the U.S. government, activists, public figures or journalists,” and that “TikTok could not monitor U.S. users in the way the article suggested.” In the internal email, Liang acknowledged that TikTok had been used in exactly this way, as Forbes had reported.

Misc

GIF Baskets
F*ck inflation, send GIFs. This year, it’s better to GIF than to receive.

Querying the GitHub archive with the ClickHouse playground
Simon Willison walks through using the ClickHouse playground, which provides a CORS-enabled API that can query a decade of history from the GitHub events archive in less than a second.

Obsidian Canvas
A new Obsidian feature that lets you organize notes visually. Embed your notes alongside images, PDFs, videos, audio, and even fully interactive web pages. Obsidian is so cool, and it keeps getting cooler 😍 

The future our grandchildren deserve
Inspiring, detailed note from Bill Gates, reflecting on what’s been accomplished and what’s left to do, discussing investing in education in the U.S., pandemic prevention, progress on polio, saving moms and babies, curing diseases like AIDS and others with gene therapy, climate change, and more.

What Comes Next for San Francisco’s Emptied Downtown
More remote-friendly companies + high cost of living and high cost of office space = unsurprisingly, an emptier downtown. But this hits local restaurants hard, and SF relies on wealthy tech companies to bankroll its massive budget. It’ll be interesting to see what the city does about that. Cut services? Tax remaining people and businesses higher?

Today San Francisco has what is perhaps the most deserted major downtown in America. On any given week, office buildings are at about 40 percent of their prepandemic occupancy, while the vacancy rate has jumped to 24 percent from 5 percent since 2019. Occupancy of the city’s offices is roughly 7 percentage points below that of those in the average major American city, according to Kastle, the building security firm.

Annual Review and Predictions

The State of Cybersecurity in 2022 and Trends and Predictions for 2023
Mike Privette lists the largest security funding events and acquisitions in 2022, and discusses overall trends. He also makes 11 product predictions for 2023, including the importance of securing no-code and products focused on measuring a company’s cybersecurity investments and decisions.

Overall funding for 2022 is expected to come in at around ~$18.4B, with over 700 funding transactions from 642 unique companies and ~$50.1B from over 260 acquisitions and mergers from publicly available data.

Frontview Mirror: 2023 Edition
(Note: members only post) Really neat musings by Daniel Miessler on what 2023 may have in store.

One of the through themes that I personally spend a lot of time thinking about as well is the combination of:

  • Companies don’t owe people jobs- their goal is to be as profitable as possible, and only hire because they have to.

    • If a company doesn’t optimize for operational efficiency, they will likely be surpassed by a company that does.

  • Machine learning is becoming more and more effective at replicating tasks that used to require people.

There will likely be a K-shaped recovery of the economy, exacerbated by machine learning, where one segment does better (people leveraging machine learning, some segments of knowledge workers), and many will do worse.

Daniel argues that one of the keys to being on the top of the K curve is to write/learn in public and build a name for yourself in your field.

Once AI takes hold, which will happen in 2024-2026, a crucial civilizational race will start

Once jobs start getting replaced, en masse, humanity will have a race on its hands. The runners are:

1. Hundreds of millions of people becoming part of the Useless Class, not being able to generate enough money to survive, and falling into an abyss of suffering and need

2. Society figuring out how to give people a way to create value within digital worlds

The metaverse will not primarily be about fun for fun’s sake. It’s primary use will be keeping billions of people sane, and feeling useful and productive, through the creation of environments, societies, and economies within virtual worlds.

Inspiration

From Steven Pressfield’s The War of Art:

Most of us have two lives. The life we live, and the unlived life within us. Between the two stands Resistance.

Look in your own heart. Right now, a still, small voice is piping up, telling you, as it has 10,000 times before, the calling that is yours and yours alone. You know it. No one has to tell you. And unless I’m crazy, you’re no closer to taking action on it than you were yesterday or will be tomorrow—because of Resistance.

2023, let’s get after it!

✉️ Wrapping Up

Have questions, comments, or feedback? Just reply directly, I'd love to hear from you.

If you find this newsletter useful and know other people who would too, I'd really appreciate if you'd forward it to them 🙏

Thanks for reading!

Cheers,

Clint