My friend Travis McPeak had this excellent thread on Twitter, and he gave me permission to reproduce it here.

Why Not Patch?

  1. Patching impacts stability – new libraries, packages, and operating system components cause software to fail. Companies can screen for this with automated test suites, but many bugs will only pop up under specific conditions.
  2. Companies don’t know all of the systems they have. They just don’t. In any large organization there are unknown systems that aren’t in the inventory.
  3. Keeping an updated package inventory is hard – you either need agents or scanning. Both have drawbacks. Agents introduce new attack surface and performance issues. Scanning requires updated asset lists (see #2). Scans need network access and authentication.
  4. Everybody is busy, patching takes effort. Unless your company prioritizes patching, new features come first.
  5. Patching in place causes downtime. Many companies have “pets” they are afraid to touch.

What to Do

Today I understand that patching is hard and will always be a struggle. It’s also critical for security. The best we can do is:

  • Build as much of your infrastructure as possible on top of a golden image that is regularly patched, registers the system in inventory, and collects software/package information.
  • Move to continuous builds and deployment. Shorten the deployment window. Set and track KPIs about average age and oldest age systems.
  • Set clear expectations and context about patching. Hold system owners accountable. Present patching status and issues to leadership.
  • Track issues caused by patching and use them to help fund testing initiatives.