Starting Strength for AppSec: What Mark Rippetoe can Teach You About Building AppSec Muscles

Fredrick “Flee” Lee, Head Of Information Security, Square twitter, linkedin
abstract slides video

In this talk, Flee gives some excellent, practical and actionable guidance on building an AppSec program, from the fundamentals (code reviews, secure code training, threat modeling), to prioritizing your efforts, the appropriate use of automation, and common pitfalls to avoid.

All while using building a weight lifting as an analogy.

Key Takeaways

The core points from Flee’s talk:

Start small with your program
Start with things where you can start seeing some wins on day 10, don’t only invest in things where you’ll start getting value 2 years in. You can’t be Google and Netflix tomorrow.

Specificity + Frequent Practice == Success
In times of crisis, we fall back on what we’ve practiced the most. This is what’s going to make you successful.

Measure everything!
That’s how you convince yourself you’re making progress and get buy-in from management.

You are not Ronnie Coleman - don’t use his program (yet)
If you’re just starting out or have a small program, it might not make sense to adopt what the biggest/most complex teams are doing. Pick the right thing for your company at the right time.

Everyone can do this
Your company doesn’t need an AppSec team to have a decent AppSec program, these are all things devs can do themselves.

Overview

Good AppSec is a muscle, it has to be trained and continuously practiced or else it atrophies.

If you’re just starting out, Flee believes the 3 core fundamentals for AppSec are:

• Code reviews (security deadlifts)

• Secure code training (security squats)

• Threat modeling (security bench press)

The Fundamentals

Code Review

All security critical code should receive a code review prior to deployment. For example, AuthN/AuthZ components, encryption usage, and user input/output.

The “Security-critical” caveat is there because most teams don’t have the resources to review everything. Focus on the most important parts.

Developer Training

All devs should have received language specific secure development training at least annually.

• Don’t show kernel devs the OWASP Top 10 - they’ll be bored as it’s not relevant to them.

• Rather than using generic examples, it’s much more interesting and engaging to show devs previous vulns from your company’s code bases. This makes it feel real and immediate.

• Emphasize how to do the right thing, don’t just point out what the vulnerability is.

You know you’re doing training right when attendees like it!

Threat Modeling

All new significant features should receive a design review, lead by developers with the security team assisting. AppSec should be there as a coach/assistant. This is an good way to get developer buy-in for security.

Document the results and plan mitigations for identified risks.

Getting Started - Beginner Gains!

Many teams start off trying to solve every vulnerability class. This is too much and be overwhelming. Instead, pick a handful of key vulns to focus on. This focus can enable you to make a big impact bin a relatively short amount of time.

Pick issues that are important to your business first. Are there any vuln classes that continue to pop up, from pen testing, bug bounty, and/or internal testing? If you’re not sure which vulns to focus on initially, that’s OK, you can pick a generic standard list, like the OWASP Top 10.

You’ll find that your weakest/most neglected areas improve quickly. Start small and learn!

Focus on Critical Areas

What’s most important to your business? Which teams historically have the most problems? Help them level up.

Aim your reviews at areas that return the biggest bang for the buck (e.g. high risk apps).

Get progress in a safe, refined way, for example, by defining processes or using checklists. Following the same established routine every time => get the same quality of results.

Understand the languages and frameworks your company uses. If you don’t know the stack your devs are using, you won’t be able to give them very good advice, and it damages their trust in you if you make generic or wrong recommendations.

Get help from devs - they can give you feedback on if you’re pushing security forward in a safe and sane way. Devs know the code the best, know what it’s supposed to do, etc.

Adding Automation and Tools

Adding automation helps you scale, but it’s not where you should start. Nail the fundamentals first: understand where you are and the problems you have and target them.

Static analysis can be a great tool, but it isn’t perfect and you shouldn’t start with it. Add it in after your company already has a good understanding of how to do code reviews. You should first have a good idea of the types of issues the tool should be looking for, otherwise you’ll end up wasting time.

Automation supplements humans. It doesn’t replace all manual effort: you still need a good code review program.

Every Day is Leg Day

Some activities are so useful that they should occur in every project. For example, code reviews have a huge return on risk reduction vs effort. Flee’s never had an instance where a code review wasn’t useful. They’re not quick, but they’re valuable.

Make the activity mandatory, but make reduce friction where possible to ensure the process is easy and palatable to dev teams. The security team doesn’t necessarily need to do it, these activities can be done by devs in many orgs. Have the security team engage devs: “Hey, here are some security things that you should be looking for.”

Measure and Record Your Progress

You can’t manage what you don’t measure
If you don’t measure your progress, you won’t know if you’re succeeding. What are the common vulns you’re seeing? By team / product? For example, if a team has been having trouble with SQL injection, give them focused training. Also, tune tools to target your most common vulns.

Record verything
If it’s easy to log, collect it. Track ALL defects found, and make them visible: to everyone, not just security team. Devs will change their behavior when that info is public. This doesn’t need to be done in a negative way, but rather helping people keep themselves accountable. If devs don’t see this info collected in one place, they might not know themselves.

Adopt Standards and Expert Help

Leverage what you’ve learned to start enforcing coding guidelines. As you progress, you can become stricter on what counts as a code review “violation.”

Over time, try to move the AppSec team towards being “coaches” of reviews rather than “reviewers.” The AppSec team should be personal trainers for the developers performing the reviews. Security Champions can scale AppSec efforts really well.

Refining for Specific Goals

Once you’ve mastered some fundamentals, you can tweak your review process to target specific weaknesses:

• Tune your code reviews/static analysis to find issues specific to your org (even non-security issues).

• Re-inforce good practices with automation, not just pointing out problems.

• Build/user secure frameworks and services.

Pro-tip: A powerful way to get dev buy-in for static analysis is show them how to find a non-security coding bad practice they care about. For example, foo() needs to never have a hard-coded string passed in its second argument.

Pitfall: Taking on Too Much

Don’t expect to cover every project immediately, especially if you’re just starting with a small team.

Don’t get hung up on low-risk apps and vulns and not give time to other code bases or other classes of issues that are more impactful. Some vulns matter more than others.

Give your program time to grow. You’re not going to have FAANG-level security in a year, especially if you have a small team. Overpromising what you’ll be able to accomplish in a short amount of time to other people in your org can damage your reputation and their trust in you.

Pitfall: Bad Form

No developer buy-in
Devs are incentivized to ship features. You have to be mindful of their perspective and mindset. Ideally members of the security team have some dev background, as it’s incredibly useful to be able to speak engineering’s language and communicate how they think.

Generic secure development training
Flee finds most security training is bad - it’s generic and not customized to the types of technologies one’s company uses or the situations their devs typically face. This make it much harder to get dev buy-in and interest, as the training is taking some of their time.

Using Untuned Tools
Flee has never found a security tool that you buy, run without tuning, and the output is useful. Tools always require customization.

Pitfall: Trying to Use (the wrong) Shortcuts

There are no silver bullets to create a strong AppSec program, it takes time and effort.

Skipping documentation/metrics
You can’t just find bugs and put them in Jira. You need to document what you found and your results along the way so you can look back later and hold yourself and others accountable.

Don’t over rely on tools only
Despite what’s shouted from the RSA and BlackHat vendor halls, tools won’t solve all your problems.

Avoid FUD when trying to influence devs
Using FUD to try to motivate devs undermines your credibility and makes them less likely to listen to you in the future. You need their buy-in: that’s your best ally in getting security to run and work well.

What about insert your fav activity?!?!

This talk discussed code reviews, secure code training, and threat modeling because they’re the fundamentals.

There are other things that are useful, but they don’t have to be there on day 1 (e.g. pen testing, ng(WAFs), RASP, etc.) They have their uses, but aren’t critical to AppSec.

Questions

This is hard. Partner with devs and security champions and have them help you create the training.

Rugged Software also has some useful ideas on integrating security into agile dev environments.