Fight the unknown in risk identification, with an intelligence cycle
Fight the unknown in risk identification, with an intelligence cycle
30 June 2022
Risk identification in ambitious projects is difficult because of the large number of unknowns. Add an intelligence cycle to agile project management to make it faster and more efficient.
The cycle is part of intelligence-led project management. It extends agile project management to support high risk projects that are threatened with failure because of unknowns. The new techniques are an adaption of practices used by the intelligence profession to fight major threats, such as terrorism.
Intelligence-led project management relies on breaking a big threat into small, manageable challenges. The big questions become smaller, and ultimately they become manageable.
The challenges include understanding “unknowns” to the point where they can be described as risks (i.e. risk identification). There are also questions of how to manage the risks.
In a multimillion dollar project, there may be hundreds of questions that will need to be resolved, although initially it seems much smaller.
The intelligence cycle treats each question in a sequence: define the next question to be answered, research it, analyse it, engage with people outside the project, then decide on the next action. Getting it to a point of a major decision may involve a number of intelligence cycles, and cycles by different teams.
The action-on all-risks intelligence cycle for risk identification and finding methods to manage those risks
Step 1. The brief
The brief is the question to be answered – the challenge. In engineering terms, it’s the requirements for the risk identification. It could be an assumption to be tested, an unknown unknown to be investigated, an explanation to be provided, or a risk management method to be found.
Big questions need to be broken up to give initial smaller questions that can be answered even if the bigger question can’t yet.
Technical unknowns. In an agile environment, the product manager has overall responsibility for intelligence on the big questions. This responsibility also includes fitting the work into the sprint. The original idea or request wasn’t necessarily from the product manager – it may have come from the project manager, the owner, stakeholders, suppliers, or variety of other sources. However it’s the product manager’s responsibility to present the brief in a form that the tech team can work on. The product manager may delegate to a tech lead.
Non-technical unknowns. There are some the project manager can manage, such as for project planning and communications. It includes staffing issues, contract, finance, business and marketing. The project manager oversees the different unknowns, works to get people to take responsibility for research, and provides a joined-up view across the different areas. Pass the work of researching and analysing these unknowns to people with the skills and experience.
Track every question being followed. Use red, amber and green to indicate urgency. It’s a prioritised form of risk identification. An example below, followed by the broader context.
Question
Action
Urgency
Do the existing customer profiles work for us? (A known unknown)
@Martin M, report by end of sprint
Green
Why have none of us used this method before? (A suspicion of unknown-unknown)
@Constantina, in 3 days for initial checks
Amber
The sequence of moving from unknowns to risks to problem …
Step 2. Research (risk identification)
The research activity provides useful facts about an unknown or a poorly defined risk. It helps find the softer information that leads to a better understanding of the context. This is objective information on which the subsequent analysis can be performed.
The researchers are people with research skills. Their job title may be something completely different. It’s not necessarily a fulltime role – this may be just a hat they wear for research a single unknown or risk.
Methods. The types of research that are used include information collection and ordering, summarisation, interview and questioning, data mining, web crawling, and more. It’s common for the research to translate the unknown into a risk. Often there’s more than one risk implied from the research. (As a method of risk identification, it’s efficient.)
Outputs. The outputs from a research cycle may be built iteratively, with each intelligence cycle adding more detail. Some examples of outputs: data tables, scripts from interviews or discussions, lists of documents and information sources, summaries of larger collections of information, maps, descriptions of people’s or company’s roles, assessments of as-is states, and reviews of technology alternatives.
Quality. Research must be objective, to discover “truth” rather than to prove a point. Sometimes it identifies that there is no risk. Sometimes it’s unreliable and has to be ignored. And sometimes, it’s inconclusive towards the bigger question.
Mindset. Good research requires a unique mindset – it’s a profession. It’s different to the distinctive mindset of the analyst, which is why the roles are separated.
Roles in intelligence-led project management.
Step 3. Analysis
Analysis uses research and reasoning processes to provide deeper insights. It can take an “unknown” or unmanaged risk to a manageable risk. It can also indicate that there is nothing to worry about.
Inputs. Analysis uses the research, but it is not totally dependent on it. Analysis uses a wide range of facts from different areas, and also personal experiences. Analysis also looks at other project risks found by risk identification. While performing analysis, it’s common for the analyst to identify new questions that will need additional research.
People. Analysis can be performed by people whose job titles do not include the word “analyst”. Analytic thinking is common among architects, senior developers, and many people in junior management roles. And in the context of an intelligence cycle for an unknown or risk, it’s not necessarily a full-time role.
Methods include data analysis, market analysis, behavioural analysis, systems analysis and security analysis. In support of this, analysts use tools for visualising results, as well as specialist tools for analysing data. Analysts regularly use research tools, to find information that is readily available – the kind of facts that do not need additional research.
Approach. There are variations in how analysts work. A common approach is to start with tentative ideas, which they may try on trusted colleagues, then the inner team, and finally work a subset of it ready for dissemination.
Collaboration. It helps enormously if the others in the intelligence cycle can see the early thoughts as draft diagrams, or hear them within meetings. Researchers can shift the emphasis of their work, stakeholder liaisons can look at how it relates to other areas, and managers can start planning for the consequences.
Outputs. The intelligence cycle for managing unknowns and risks tends to result in short reports, presentations and diagrams. There are also intangible outputs such as come from the dialogues and meetings: a greater understanding, improved clarity, increased consensus on some points, new suspicions of other unknowns, and team bonding.
Mindset. Analysis should be neutral, seeking better understanding without bias. Or putting forward a fair selection of options, without slanting details to one. (If it’s obvious that one is best, let other people make that conclusion. If they suspect you of distorting the analysis, your work may be for nothing.)
Independence. The results of the analysis may have bad consequences. Don’t hide it, don’t agonise over it. There are people in the management layer whose job includes handling problems, and making them less painful.
Step 4. Engagement (liaison)
Engagement is the work of engaging with key people and teams outside the project, who may be influenced by the results, or who may be able to help in some ways. This is the work of a liaison.
Timing. The preparation for engagement starts early in the intelligence cycle for an unknown or risk. There needs to be prep for the main burst of engagement that follows the analysis and leads to the decision point.
Learning. At the start of an intelligence cycle there is little indication of what it will find, if anything. There is often ambiguous terminology and ill-formed ideas. This is a time for a liaison to seek understanding of the subject.
Networking. Throughout the cycle, there’s also a need to network – find people who may be able to help later.
During research and analysis. Once the intelligence cycle has started there are working documents that are shared within the team, and with a few experts outside the team. The liaisons use it to work out who will need to know, if anyone. Within a high risk project, there are so many unknowns and risks active at a time that it’s important to focus on what matters to individuals.
Prep for the decision. If the output of the cycle looks like it will need escalation, there’s additional work to refine draft messages, check them, and practice. Don’t apologise for bad news, or for not recognising it earlier. This is a high risk project with multiple unknowns, and there was not enough time to investigate all of them before starting work. The senior stakeholders accepted that when they approved the project.
Step 5. Decision point
The final activity in the intelligence cycle is the decision of what to do next with the unknown, and the immediate follow-up. There is a scale of responses.
It’s not a risk – abandon it
The research and analysis indicates that the unknown is not a risk, or it’s trivial compared to everything else. The product and project managers make the decision that the risk identification is “negative”.
It’s a small risk – postpone it
It’s now clear. The impact of the unknown can be described, and its likelihood can be estimated – i.e. the risk identification was successful and it’s now added to the risk register as a green risk. The analysis indicates the risk is not urgent compared to the other high risks. Or the analysis is inconclusive, but there are other higher risks to focus on. That’s a decision for the product and project managers, after consulting with external stakeholders. (The stakeholders who are impacted, not all of them.)
It’s a small risk – absorb it immediately
With the benefit of research and analysis, there’s sometimes an obvious way to resolve a risk, and do it with minimal effort. For example, change the project plan, use different technology choices, or an alternative method for handling data. That’s a choice for the product or project manager.
It’s significant risk – but further intelligence is needed
The intelligence cycle has not found enough evidence to justify major decisions, but there are indicators that it’s high risk. Or perhaps it’s clearly a risk, but there’s no obvious or efficient way to manage it. More research and analysis is needed, by these or other people. And there may need to be more liaison with external stakeholders in the project.
It’s serious – escalate upwards
The seriousness of the risk needs to be escalated so people can prepare, and to justify the costs of managing it. There is prep work for reporting or presenting to the executive, and working with the stakeholder liaisons to refine their message to the broader range of stakeholders. Don’t be surprised if the decision-makers ask for more intelligence, perhaps urgently. It’s the start of more intelligence cycles.
Related reading on intel-led risk identification and management
For more about managing intelligence cycles, there are ideas within “Leading Intelligence Analysis: Lessons from the CIA’s Analytic Front Lines”, by Bruce E Pease, 2020. (See more at https://us.sagepub.com/en-us/nam/leading-intelligence-analysis/book258717 .) It’s only partly relevant, because there are a major differences between CIA activities and high-risk technical projects.
The description of the intelligence cycle, is derived from chapter 5 of “Securing the State”, by David Omand, 2011. (See more at https://www.hurstpublishers.com/book/securing-the-state/ .) Chapter 6 is also useful for understanding the weaknesses in intelligence to which we’re susceptible.
The non-necessary cookies are to retain statistics about visitors, using Google Analytics. This helps us improve existing content and identify what new content to create. (There is also a non-necessary cookie to aid with caching pages, to improve website performance.)
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Fight the unknown in risk identification, with an intelligence cycle
Risk identification in ambitious projects is difficult because of the large number of unknowns. Add an intelligence cycle to agile project management to make it faster and more efficient.
The cycle is part of intelligence-led project management. It extends agile project management to support high risk projects that are threatened with failure because of unknowns. The new techniques are an adaption of practices used by the intelligence profession to fight major threats, such as terrorism.
Intelligence-led project management relies on breaking a big threat into small, manageable challenges. The big questions become smaller, and ultimately they become manageable.
The challenges include understanding “unknowns” to the point where they can be described as risks (i.e. risk identification). There are also questions of how to manage the risks.
In a multimillion dollar project, there may be hundreds of questions that will need to be resolved, although initially it seems much smaller.
The intelligence cycle treats each question in a sequence: define the next question to be answered, research it, analyse it, engage with people outside the project, then decide on the next action. Getting it to a point of a major decision may involve a number of intelligence cycles, and cycles by different teams.
Step 1. The brief
The brief is the question to be answered – the challenge. In engineering terms, it’s the requirements for the risk identification. It could be an assumption to be tested, an unknown unknown to be investigated, an explanation to be provided, or a risk management method to be found.
Big questions need to be broken up to give initial smaller questions that can be answered even if the bigger question can’t yet.
Technical unknowns. In an agile environment, the product manager has overall responsibility for intelligence on the big questions. This responsibility also includes fitting the work into the sprint. The original idea or request wasn’t necessarily from the product manager – it may have come from the project manager, the owner, stakeholders, suppliers, or variety of other sources. However it’s the product manager’s responsibility to present the brief in a form that the tech team can work on. The product manager may delegate to a tech lead.
Non-technical unknowns. There are some the project manager can manage, such as for project planning and communications. It includes staffing issues, contract, finance, business and marketing. The project manager oversees the different unknowns, works to get people to take responsibility for research, and provides a joined-up view across the different areas. Pass the work of researching and analysing these unknowns to people with the skills and experience.
Track every question being followed. Use red, amber and green to indicate urgency. It’s a prioritised form of risk identification. An example below, followed by the broader context.
…
Step 2. Research (risk identification)
The research activity provides useful facts about an unknown or a poorly defined risk. It helps find the softer information that leads to a better understanding of the context. This is objective information on which the subsequent analysis can be performed.
The researchers are people with research skills. Their job title may be something completely different. It’s not necessarily a fulltime role – this may be just a hat they wear for research a single unknown or risk.
Methods. The types of research that are used include information collection and ordering, summarisation, interview and questioning, data mining, web crawling, and more. It’s common for the research to translate the unknown into a risk. Often there’s more than one risk implied from the research. (As a method of risk identification, it’s efficient.)
Outputs. The outputs from a research cycle may be built iteratively, with each intelligence cycle adding more detail. Some examples of outputs: data tables, scripts from interviews or discussions, lists of documents and information sources, summaries of larger collections of information, maps, descriptions of people’s or company’s roles, assessments of as-is states, and reviews of technology alternatives.
Quality. Research must be objective, to discover “truth” rather than to prove a point. Sometimes it identifies that there is no risk. Sometimes it’s unreliable and has to be ignored. And sometimes, it’s inconclusive towards the bigger question.
Mindset. Good research requires a unique mindset – it’s a profession. It’s different to the distinctive mindset of the analyst, which is why the roles are separated.
Step 3. Analysis
Analysis uses research and reasoning processes to provide deeper insights. It can take an “unknown” or unmanaged risk to a manageable risk. It can also indicate that there is nothing to worry about.
Inputs. Analysis uses the research, but it is not totally dependent on it. Analysis uses a wide range of facts from different areas, and also personal experiences. Analysis also looks at other project risks found by risk identification. While performing analysis, it’s common for the analyst to identify new questions that will need additional research.
People. Analysis can be performed by people whose job titles do not include the word “analyst”. Analytic thinking is common among architects, senior developers, and many people in junior management roles. And in the context of an intelligence cycle for an unknown or risk, it’s not necessarily a full-time role.
Methods include data analysis, market analysis, behavioural analysis, systems analysis and security analysis. In support of this, analysts use tools for visualising results, as well as specialist tools for analysing data. Analysts regularly use research tools, to find information that is readily available – the kind of facts that do not need additional research.
Approach. There are variations in how analysts work. A common approach is to start with tentative ideas, which they may try on trusted colleagues, then the inner team, and finally work a subset of it ready for dissemination.
Collaboration. It helps enormously if the others in the intelligence cycle can see the early thoughts as draft diagrams, or hear them within meetings. Researchers can shift the emphasis of their work, stakeholder liaisons can look at how it relates to other areas, and managers can start planning for the consequences.
Outputs. The intelligence cycle for managing unknowns and risks tends to result in short reports, presentations and diagrams. There are also intangible outputs such as come from the dialogues and meetings: a greater understanding, improved clarity, increased consensus on some points, new suspicions of other unknowns, and team bonding.
Mindset. Analysis should be neutral, seeking better understanding without bias. Or putting forward a fair selection of options, without slanting details to one. (If it’s obvious that one is best, let other people make that conclusion. If they suspect you of distorting the analysis, your work may be for nothing.)
Independence. The results of the analysis may have bad consequences. Don’t hide it, don’t agonise over it. There are people in the management layer whose job includes handling problems, and making them less painful.
Step 4. Engagement (liaison)
Engagement is the work of engaging with key people and teams outside the project, who may be influenced by the results, or who may be able to help in some ways. This is the work of a liaison.
Timing. The preparation for engagement starts early in the intelligence cycle for an unknown or risk. There needs to be prep for the main burst of engagement that follows the analysis and leads to the decision point.
Learning. At the start of an intelligence cycle there is little indication of what it will find, if anything. There is often ambiguous terminology and ill-formed ideas. This is a time for a liaison to seek understanding of the subject.
Networking. Throughout the cycle, there’s also a need to network – find people who may be able to help later.
During research and analysis. Once the intelligence cycle has started there are working documents that are shared within the team, and with a few experts outside the team. The liaisons use it to work out who will need to know, if anyone. Within a high risk project, there are so many unknowns and risks active at a time that it’s important to focus on what matters to individuals.
Prep for the decision. If the output of the cycle looks like it will need escalation, there’s additional work to refine draft messages, check them, and practice. Don’t apologise for bad news, or for not recognising it earlier. This is a high risk project with multiple unknowns, and there was not enough time to investigate all of them before starting work. The senior stakeholders accepted that when they approved the project.
Step 5. Decision point
The final activity in the intelligence cycle is the decision of what to do next with the unknown, and the immediate follow-up. There is a scale of responses.
It’s not a risk – abandon it
The research and analysis indicates that the unknown is not a risk, or it’s trivial compared to everything else. The product and project managers make the decision that the risk identification is “negative”.
It’s a small risk – postpone it
It’s now clear. The impact of the unknown can be described, and its likelihood can be estimated – i.e. the risk identification was successful and it’s now added to the risk register as a green risk. The analysis indicates the risk is not urgent compared to the other high risks. Or the analysis is inconclusive, but there are other higher risks to focus on. That’s a decision for the product and project managers, after consulting with external stakeholders. (The stakeholders who are impacted, not all of them.)
It’s a small risk – absorb it immediately
With the benefit of research and analysis, there’s sometimes an obvious way to resolve a risk, and do it with minimal effort. For example, change the project plan, use different technology choices, or an alternative method for handling data. That’s a choice for the product or project manager.
It’s significant risk – but further intelligence is needed
The intelligence cycle has not found enough evidence to justify major decisions, but there are indicators that it’s high risk. Or perhaps it’s clearly a risk, but there’s no obvious or efficient way to manage it. More research and analysis is needed, by these or other people. And there may need to be more liaison with external stakeholders in the project.
It’s serious – escalate upwards
The seriousness of the risk needs to be escalated so people can prepare, and to justify the costs of managing it. There is prep work for reporting or presenting to the executive, and working with the stakeholder liaisons to refine their message to the broader range of stakeholders. Don’t be surprised if the decision-makers ask for more intelligence, perhaps urgently. It’s the start of more intelligence cycles.
Related reading on intel-led risk identification and management
For more about risk management in agile projects, see https://www.isaca.org/resources/isaca-journal/issues/2016/volume-2/risk-management-in-agile-projects
For 7 ways to identify risks, see https://projectriskcoach.com/7-ways-to-identify-risks/
For more about managing intelligence cycles, there are ideas within “Leading Intelligence Analysis: Lessons from the CIA’s Analytic Front Lines”, by Bruce E Pease, 2020. (See more at https://us.sagepub.com/en-us/nam/leading-intelligence-analysis/book258717 .) It’s only partly relevant, because there are a major differences between CIA activities and high-risk technical projects.
The description of the intelligence cycle, is derived from chapter 5 of “Securing the State”, by David Omand, 2011. (See more at https://www.hurstpublishers.com/book/securing-the-state/ .) Chapter 6 is also useful for understanding the weaknesses in intelligence to which we’re susceptible.
Contents
Related Posts
The assassination – a 2-page story of an ethical dilemma
How to build a product website that sells
Person Lurking – story of AI in intelligence analysis