A Vulnerability Assessment (VA) is the final step in a Security Analysis, based on information developed in all the earlier stages (Site Survey, Area Study, Threat Assessment, and External Assets Evaluation). It requires honest qualitative evaluation of strengths and weaknesses against various threats, from which we identify upgrades worth pursuing. There are several useful techniques.
We are not military staffs, federal agencies, or large corporations; we are not full-time salaried protection specialists; nor are we, for the most part, concerned with protecting large and complex targets from the sort of adversarial threats those organizations face. We won't try to teach or describe the detailed quantitative methods those people and organizations use for vulnerability assessments. We may even feel we've done enough homework already, preparing the assessments and surveys in the previous steps of our Security Analysis. But the final step requires testing or validation of assumptions, without which all we have are unfounded opinions. We recommend two categories of techniques, BOGSAT and wargaming, both of which depend on team effort, honesty, objectivity, and a touch of imagination and competitive spirit.
BOGSAT
Known by those with less humor and humility as the "seminar method," this is a technical acronym for "Bunch of Guys (and/or Gals) Sitting Around a Table." It is essentially the same method we've already used for our Threat Assessment and the other previous parts of Security Analysis. Applied to Vulnerability Assessment, it is a relatively quick and potentially useful method. Done well, it benefits from the differing perspectives and experience of participants. Done poorly, it is just a bunch of people arguing over and validating their prejudices and preconceived notions.
A facilitator/moderator is needed. He can conduct the session in an informal, freeform way or with a greater structure, but should use a light hand.
The key to its success is that everyone has a say and a vote, and most conclusions should be reached by consensus, unless the process becomes hopelessly logjammed. There should be no deference to hierarchy or authority, beyond basic good manners. Even the head of household, or the leader or manager of the enterprise, does not have a veto on topics or opinions during a BOGSAT process.
Effectiveness of security system elements, adversary strategies and execution, and scenario outcomes are evaluated by discussion, consensus, or a vote, and are usually organized around a timeline, addressing each phase of an event or operation in sequence, with attention to what military planners would call "branches and sequels" generated by intermediate decisions and the effect of chance on specific actions and outcomes.
For an example on the smallest scale, the facilitator might open a threat scenario of home invasion by saying:
"It's midnight, midweek, midwinter and the household is asleep. You are awakened by the sound of a diesel engine outside, and you look out through the bedroom window to see a pickup truck, lights off, 50 yards down the drive, approaching the house. That RF alarm linked to your trail cam and motion detector at the gate? It did not sound. What do you do now?"
Discussion must then be limited to that decision point, and lead to a consensus on what the immediate decision or first actions are, before moving on. If there is substantial disagreement, or distinctly different options proposed, then a branch point can be created, and these branches can be pursued in a subsequent iteration of the scenario. The performance of mechanical or electronic components of the security system can be, as shown, determined by the facilitator, based not upon whim but upon prior evaluations of the component's reliability or vulnerability to adversary action.
Someone is recording the discussion and/or taking copious notes. When the current event or decision point is resolved, the facilitator notes the passage of scenario time – usually a few minutes at most - and moves on to the next point in the timeline, e.g.:
"Two minutes have passed. You woke your spouse and when she tried to dial 9-1-1 as instructed, the landline was dead. She retrieved her cell phone as she headed down the second-floor hall to gather the kids – they are just now moving behind you into the master bedroom. You armed yourself with the shotgun from your closet and moved to the landing at the top of the stairs with a view of the living room below. You heard the truck stop in front of the garage, three or four doors open and close, and the back yard gate creak and slam. The motion-activated security lights outside have activated, but you cannot see through the shades to identify any activity. What do you do now?"
And so on, rinse and repeat. When actual confrontations occur on the scenario timeline (as a result of the scenario script or the participants' decisions), the facilitator will note the participants' intent and decisions, and simply report an outcome. "The door flew open and the first person through disappeared out of your field of view to your left before you could act; you fired on the second and heard him cry out and stumble as he followed his partner." There is no room for debate; participants cannot argue about outcomes or fight the scenario. The facilitator will end the scenario at his discretion and conduct a detailed AAR (after-action review, a process we will address in a separate article) with participants, to capture all decision points and draw out and document the group's assessment of system performance and shortfalls.
As we'll discuss in greater detail in Part 3 of this series, it is important for VA team members to refrain from applying their "insider" knowledge of security elements and adversary strategy to the framing of either friendly or adversary decisions in the course of a scenario. In a scenario like the example above, actual participants would have no idea of the motivation or objectives of the adversary, at least in the early moments. Will they retreat if confronted? Is robbery their goal, or something worse? How are they armed and equipped? The adversary scenario will have answers to all these questions, but participants should not, ideally, know these answers until they are revealed during the scenario run – or if they do know, they should not base their actions or decisions on that knowledge. This potential pitfall could be avoided if there were trusted individuals not a part of the VA team, who could be brought in to play those roles in a scenario, but that may not be possible. If we are truly concerned about security, we have to consider OPSEC (Operational Security) and not needlessly expose our plans, procedures, abilities, and assumptions to people who do not have a need to know.
Tactical Decision Games (TDG)
The TDG, developed by the US Marine Corps in the late 1980s, adds a bit more structure to the BOGSAT method, but is still essentially a group exercise (in this case, the VA team) in a classroom or seminar setting. It is more commonly used for training and education, but has value as an analytical tool as well.
Some form of graphical representation – a slide, map, plan, sketch, or sand table – and a narrative description of the scenario are the only resources required. A TDG scenario represents a snapshot in time, not unlike the brief time-hack described in the BOGSAT method.
A decision maker is identified: "You are the squad leader of 2nd Squad," a Marine TDG might begin. The scenario at that point in time is described by the facilitator, who provides only the information that would be known to the decision maker at that moment, and no more. Participants are each asked to decide, in a limited period of time and in the role of the decision maker, an immediate course of action, and be ready to present and explain that decision to the group. One or more participants are chosen to do so, and the group is then invited to discuss and critique their decisions.
The TDG is not an interactive wargame; it presents a specific problem and asks for a solution. Although conducted in a group, it does not call for collective effort or collaborative solutions – each individual participant comes up with his own solution independently.
The adversary's actions and status at the chosen moment are predefined in the scenario description, and the TDG does not usually consider an extended period of time, or an "action, reaction, counteraction" sequence. The facilitator may have a prepared timeline, script or scripts that describe sequels or outcomes or "next decisions" for a number of possible participant choices. These can drive a sequential series of TDGs, or an experienced facilitator may improvise them on the spot, with the starting point for each derived from the decision made in the previous exercise. This would be very similar to the discussion-format BOGSAT method described earlier, differing mainly in its focus on individual decisions under time pressure, vs. collective decisions driven by group discussion. Because it addresses specific, critical decision points one at a time, the TDG is another technique very useful to participants inexperienced in crisis decision making or in more complex, interactive wargames.
As an example, if active shooters are a threat identified in our Threat Assessment (and how could they not be, in modern America?) a simple but useful TDG might present a sudden, multi-victim shooting at a public gathering. The Site Survey allows the scenario design to closely model the location and its physical characteristics, and to describe and profile the attendees. The TDG participant might be identified as a citizen carrying a discreetly concealed handgun, attending the event with family members; or as an organizer of the event; or even as a law enforcement officer or hired security officer at the event. The larger adversary scenario could easily be broken down into several discrete, short-duration TDGs focusing on key decision points and actions, which might include:
- Detection and reaction to pre-attack indicators.
- Immediate response to the outbreak of violence.
- Success or failure in various degrees, in stopping the violence.
- Managing the immediate aftermath, including securing the area, treating casualties, and interacting with arriving first responders.
The hard work of VA evaluation takes place in the discussion that follows execution of each TDG. Was the decision plausible? Would it be likely to succeed and why, and against which adversaries? What works against a disturbed teenage loner might be disastrous in a situation involving multiple shooters in a gang-related drive-by, or a targeted shooting by other bad actors of unknown origin who intend to escape the scene. How would the chances of success be affected by varying adversary strategies or reactions unknown and unpredictable to the decision maker in the moment? What improvements in our security posture might improve the odds of success against different levels of threats, or even deter the adversary from acting?
Terrain Walk
This is another variation on the seminar or discussion-based format. It is essentially BOGSAT without the table, and is well-suited to a small, defined venue, because participants walk the terrain where the scenario takes place, discussing and evaluating performance along the scenario timeline. For those with less tactical training and experience, this can be more productive than attempting to visualize events, even with good graphical displays, while "sitting around a table." Its other advantage in a civilian setting is that a small group of people in street clothes can walk through a building, campus, or neighborhood without drawing undue attention. Its limitations are similar to those of the conference-table BOGSAT: vulnerability to biases and preconceived notions, and the absence of a thinking adversary who gets a vote.
Conclusion
All of the BOGSAT, seminar-style techniques presented in this article are relatively simple and accessible, usable by non-professionals, and can be more effective than relying on individual opinions and assumptions in assessing vulnerabilities and devising improvements in our security posture.
The most effective – but also the most difficult and demanding – VA technique is wargaming, which we will discuss in detail in Part 3 of this series.