“We have a problem,” the delegation of long-faced people say. They mean an “unknown unknown” has hit because nobody predicted it. This article is about how to find unknown unknowns, and exploit them. It applies across projects, operations and defence.
- The Rumsfeld quote
- Known and unknown – the terminology
- Unknown unknowns can be opportunities
- Managing unknown unknowns
- Sizing an unknown unknown – for people who like numbers
- Finally: Don’t forget the known unknowns
The Rumsfeld quote
On February 12, 2002 the US Defense Secretary, Donald Rumsfeld, answered a reporter’s question with a statement that was widely reported as humorous, but then set people thinking about whether he had said something wise.
“Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.”
Donald Rumsfeld was answering in the context of a threat from terrorists. See the commentary and origin at Wikipedia, or watch the recording below.
Known and unknown – the terminology
Known knowns can be managed. It’s either happening now (as a problem) or it’s a risk that can be quantified with an impact and probability estimate, and can usually be managed in some way – even if it is to delay it.
Known unknowns are fuzzy risks. We can see there’s something odd and it may be bad. We don’t understand it enough to manage it, and we can’t estimate its impact and/or probability. Ultimately we’ll know more, and then it becomes a risk or problem. Or perhaps the unknown evolves into two or three risks or problems, each of which is managed differently.
Unknown unknowns surprise us. Perhaps we should have thought about them.
(Side comment. Different people define these slightly differently. And they also come up with additional variations, such as the “unknown knowns” in info.veritasts.com. The article below is about unknown unknowns..)
A pandemic sweeps across the planet. Pandemics are known, the timing and nature is unknown. But even the experts were surprised by the speed and scale of this one.
A computer website collapses when it gets beyond a particular size, not because of the number of users but because people’s behaviour changed in ways that were unprecedent (and hence unknown).
A scientific theory states that nothing can escape a black hole, but now it appears they can burp – see NASA pictures.
Unknown unknowns can be opportunities
Military example. Donald Rumsfeld was discussing a threat from Islamic State in Syria and Iraq (ISIL). Lives were at risk. He had a duty to protect lives. However some of the actions that ISIL took had military upsides for the West – they exposed themselves to intelligence workers, who could then pass on precise information to the military. A loss could be turned into a much larger gain. (There’s an example of this inverted thinking in my story The price of secrecy-first.)
Environmental example. Environmental changes bring unknown unknowns, including rapid changes in customer behaviour. Some companies see this as an opportunity to develop new technologies.
Technology example. In the example above of a technical project hit by unexpected behaviour by users, it may also have represented an opportunity. To replace the architecture would require new technologies, and if user behaviour had changed then there must be new commercial opportunities. For some, the crisis can be turned into a profit.
Managing unknown unknowns
Unknown unknowns need to be taken from the first hunch, through to the point where people can follow-up … or we know with reasonable certainty that the hunch can be ignored.
Here are four techniques, to use together.
Lesson #1: Think outside of the box
For most of us, we’re working within the boxes defined in job descriptions. We are judged by our successes and failures within these walls. Within closed walls there are few upsides, and unknowns and risks come as downsides.
In the technology example above, project managers see a technology failure as a crisis for the project and hence for the project manager. There will be an impact on the project’s deadlines, costs, delivered outputs and quality. However for people outside the project, there’s a bigger picture. Higher management may use it as an opportunity for change.
Military intelligence officers (in the ISIL example above) should be coordinating with other intelligence units. That includes ones that cross different threats, and also ones that focus on counter-intelligence activities.
The unknown unknowns around planetary climate and biodiversity create obvious fears. It’s also provided constructive opportunities for companies to appeal to their customers by demonstrating their “greenness”. It has even brought China and the USA together on this one issue – the first big agreement for years.
Lesson #2: Watch for worsening symptoms
With hindsight, we can often see symptoms that warn of an unknown unknown. Initially the symptoms are easily dismissed as the random behaviour of our world. People talk about them, or fret internally, and then the symptom disappears or is explained by something else … or it becomes more serious.
The initial symptoms are sometimes vague: “It’s a feeling. I’ve seen things like this before. Perhaps this is important, perhaps not … but it’s worth checking.”
If the unknown unknown is an opportunity, our ambition is to find it early so we can exploit it rather than our competitors or adversaries. And if the unknown unknown is a threat, we want to find it early when there is still a chance to defuse it.
Lesson #3: Appoint an investigator and sponsor
Unknown-unknowns needs investigation, and for that an “investigator” is needed, and someone to “sponsor” the investigator.
Investigators. The Investigator researches the symptoms, looking for the potential underlying influences. The Investigator may have people who help with the research. It’s not uncommon to see several potential influences, but most are not strong enough to drive a major threat or opportunity. If there are indicators of a strong influence, then the Investigator’s role is to explore it like a scientist (as in lesson #4, below). So first define the hypothesis explaining how to demonstrate that influence may be significant. And then define a test for the null hypothesis: that the influence is irrelevant.
Sponsors. The Sponsor covers the costs for the Investigator, and any researchers or other resources the Investigator uses. The advantage of making this explicit is that the Sponsor protects the Investigator and can accelerate the next steps if there is something to follow. Sometimes the Investigator has no Sponsor and is working in free time with volunteers, using tools and information that already exists. It’s politically risky for an investigator to self-sponsor. Even the tacit support from a Sponsor can help.
Angels. If there’s no Sponsor and no Investigator, just one or more people with a hunch, then you need an Angel – somebody who can work the internal politics enough to find a sponsor and investigator, but who doesn’t want to be involved in the investigation. Sometimes the Angel benefits later if it turns out to be a significant opportunity, and sometimes the Angel is just focussed on “doing the right thing”.
And finally, there is an unofficial additional role that people see: the Gainsayers. They predict something, they fail to elaborate enough for people to follow-up, then they ultimately claim to have predicted it.
Lesson #4: Elimination with small steps
Exploring unknown unknowns is an exercise that is prone to disappointment but can have amazing successes.
Identify potential causes of the symptoms. Jot down some ideas about the causes. (More may come later as you learn.)
Test for the null hypothesis – i.e. that’s it’s irrelevant.
Focus on elimination – looking at possible causes. (Are they significant? Why?)
Do the easiest ones first.
Inter-dependence. Watch for causes that could be interconnected. If it’s significant inter-dependence, you may need to investigate them together.
Break then down. Some causes can be subdivided into smaller areas that are easier to investigate. Try looking for causes of causes.
Watch and wait. If there’s still no clear rationale, but “gut feeling” suggests something is there, it may help to wait and watch for new symptoms. And if nothing now appears, perhaps it’s because there’s nothing to find.
Sizing an unknown unknown – for people who like numbers
It’s not possible to size an unknown unknown. Putting a probability and impact estimate on an unknown is statistical nonsense. If you can’t define it, you won’t know if it occurs. (And contrary, if you can estimate it, then it’s not an unknown.)
(For tips on project risk analysis, taken from intelligence estimation, see the blog Stop project risk analysis failure – 6 tips from (secret) intelligence estimation.
Finally: Don’t forget the known unknowns
The risk of a global pandemic was known, what was unknown was the timing and the detail. For some businesses it was possible to insure against it.
Curiously, the pandemic had two upsides. The experts did not predict the world could roll out multiple effective vaccines within a year, and we did not realise it was possible to move the vast majority of office work into people’s homes. Those were also unknown unknowns.