Overcoming Analysis Paralysis

Introduction

Perhaps the earliest description of analysis paralysis begins with the story of The Fox and the Cat as told by Aesop in which the fox details the multitudes of ways to escape the hounds and hunters while the cat only has a single method. In the end, the fox, which is paralyzed while detailing the multiple ways to transition to safety, met an untimely demise while the simple cat ascended a nearby tree to safety. The moral of the story might be summarized by General Patton, “A good plan, violently executed now, is better than a perfect plan next week.”

Threat modeling can sometimes put security champions and analysts into a similar situation. Where should the threat model start? Where should it end? What does it need to contain? Who should be involved? These questions puzzle teams at one point or another and will likely be present in some fashion in most organizations.

The following sections will present some speed bumps and false statements around analysis paralysis and then propose a process and rule for overcoming those items and moving forward.

Overcoming Analysis Paralysis in Threat Modeling

Speed bump 1 - Failure to start

The most effective way to start something is to start doing something. Set expectations that the group is going to encounter some problems and that the beginning is going to be rough. Set expectations that learning and failure are a part of growth and iteration. Interestingly enough, if you are not threat modeling at all, you are already failing. Start small and fail fast with a small team of threat modeling evangelists who are willing to trade some time and energy to get a program going.

Analysis Paralysis Rule 1 - Just Start

Speed bump 2 - Failure to define scope appropriately

Ambition and motivation can be a great value add to a project but remember that all of the frozen climbers on Mount Everest were also once highly motivated individuals. Define scope early in the threat modeling process and mark off which areas are going to be “out of scope”. Out of scope does not mean that the organization won’t threat model those items but instead means that they won’t threat model them right now. Defining scope appropriately allows the team to narrow their field of vision and add clarity to which parts are being deconstructed for this analysis. If a problem or threat model seems too large, it probably is. Define layers and subprocess and break those into smaller bite sized models.

Analysis Paralysis Rule 2 - Limit Scope

Speed bump 3 - Failure to define acceptable outputs

Can your team articulate what success looks like in a threat model? Is there consensus on what success looks like for all levels in the organization? For some organizations, success means having a long list of possible threats. For others, it means having a smaller prioritized list that meaningfully improves security posture. Remember at the end of the day, a long list without priorities is better than no list at all.

Analysis Paralysis Rule 3 - Define Success

Speed bump 4 - Perfection not progress

The process is messy. The process is too manual. The process is hard to schedule. Interestingly enough, you know what other process is difficult? Depositions - where you have to answer why common security vulnerabilities were not fixed in a timely manner. Focus on forward progress, not on perfection. Iterate, iterate, iterate. Continuous improvement. Seemingly developers and teams understand how to continuously integrate new changes into code and products but stall when the team cannot decide how to phrase risk statements or craft the perfect welcome email.

Analysis Paralysis Rule 4 - Iteration leads to perfection

Speed bump 5 - Failure to define priorities (near term and long term)

A group of professionals can generate loads of threats and risk against an application and system. The next obstacle is how to navigate moving those into production systems. A threat modeling program should have some set of priorities for which controls or countermeasures should be prioritized over others. Perhaps it is based on system criticality or the financial cost of down time for those systems. Fortunately, some industries have published standards around the most common exploits per year or per system type¹. Establish priorities and then execute.

Analysis Paralysis Rule 5 - Prioritize & Execute

Speed bump 6 - We are waiting on the right product

Products do not solve process problems. A product will just get your team to analysis paralysis faster. Use a manual, a whiteboard, or paper process in the short term to start a preliminary process to get the team started and then run with it. Once the process is codified, then a tool can further sustain that process.

Analysis Paralysis Rule 6 - Process before product

Speed bump 7 - No end in sight

There is a possibility that threat modeling may generate so many threats that it will be impossible to mitigate or reduce all of them. The question might be asked, “how will we ever get through all of these security issues?” The answer, “we won’t and that's okay”. The dichotomy of finding threats and then having to fix them can create some severe angst in teams that are already overloaded with work. Communicate to the team that the goal of this process is to generate threats but only the threats that fit our priorities and match our requirements will be added to current sprints and prioritized for remediation.

Analysis Paralysis Rule 7 - Doing meaningful work

Speed bump 8 - We don’t have the right people

Not having the perfect multifaceted, cross functional team where each and every member has 15 plus years of experience in their respective roles might feel like a blocker for many organizations; however, the people that are on the team right now are the right people at the right time to start something new. The perfect team does not exist. Delaying a process while you wait for the perfect team is similar to the perfect time to plant a tree - 15 years ago and today. Start planting a new process on your team and move forward.

Analysis Paralysis Rule 8 - The RIGHT people are here RIGHT now

Conclusion

To quote the french philosopher Voltaire, “Perfect is the enemy of good” might be the final summary of successfully overcoming analysis paralysis. This problem is not unique to any single industry although it is consistently found in focused, intelligent, and purposeful individuals.

Continuously pursue good threat modeling and iterate for great threat modeling.

References: ¹ www.owasp.org/www-project-top-ten/

‍