The secret devil’s advocate – a story of managing the unknown

The Secret Devil’s Advocate is a fictional account of managing unknown-unknowns, as Donald Rumsfold once described it (see https://en.wikipedia.org/wiki/There_are_known_knowns). This story is not about terrorism but an entirely different context involving an authoritarian regime. For the story, this is China.

The 12-page story illustrates how to manage unknown-unknowns and their opportunities. It shows the reality of managing unknown-unknowns is messy and does not neatly fit into a prescribed method. See commentary below for some brief notes.

The story also illustrates the importance of the Duty of Care to intelligence staff in the field.

The Secret Devil's Advocate is set in the world of espionage.

The Secret Devil’s Advocate – story, fiction

Graziano breathed deeply as he stared at the dry words on the orders: URGENT on Kelsie Kriksaw. Sever all communications. He could feel moisture in his right eye as he thought of Kelsie. She’d been arrested for espionage, the Chinese had her, they had everything from her apartment, and they had probably been watching everyone she met.

Kelsie had been one of the few in Graziano’s flock that he had actually met. The other intelligence officers were just code names, used so he would not know their identities. Occasionally he could guess, but he kept his guesses to himself. Kelsie had come to visit him and his little team in the windowless office in headquarters. Her small round eyes took in the work conditions and she had a twitching half-smile that Graziano found cheeky. Though not in a sexual way.

“I want to see the people who I’m risking my life on,” she said. She was one of those people who spoke from the back of her throat to produce a growl that was disproportionate to her skinny frame.

She’d studied him, too. His shape from good food cooked by a loving wife, his immaculate clothes to make her proud of him, and the almost boyish innocence of his face from playing with their young son.

Graziano introduced his two research assistants. “They’ll be the ones who go through your contact reports and retrieve the nuggets of gold that our little system needs.” “Little? Not so small when it feeds from so many different intel sources, as well as our own reports. And definitely not unimportant,” Kelsie responded, “when you’re tracking thousands of potential enemy spies across the world.” Graziano smiled back: “Our Artificial Intelligence finds many, but some spies are smart at hiding from AI. That’s where your eyes on the ground come in.” “Got it,” Kelsie said sharply, “I trust you.”

And now she was in prison. There would be no half-smiles. Instead there would be questioning, probably interrogation, and years of confinement. Or perhaps they would trust her like their own: kneel, a bullet in the back of her head, and then the bill for the bullet is sent to her next of kin.

Graziano thought about her interrogation. He wondered if she would describe him and his team, and their work. There was nothing much she could say – the Chinese already knew his name. And as for his work – they had a similar unit, which reputedly had a hundred people instead of three, and a bigger budget for computing, plus intel data stolen from Western companies and governments.

For lunch, Graziano sat alone facing the window. He wanted to be with his own thoughts. He definitely did not want to be approached by Two Eyes. Graziano never referred to Two Eyes with that name, but the man’s face was a distraction when talking. Half of it had been paralysed by a stroke and one of his eyes had become as static as a glass eye. The other compensated by switching erratically from side to side. Graziano had found the best tactic was to stare firmly at the good eye, or look away entirely.

“They got Kelsie,” Two Eyes said by way of introduction. He waited for Graziano’s response, but the canteen was the wrong place to discuss these things so Graziano remained silent. “Clearly you know. But did you hear how they identified her? It was via one of the suspects in that stupid AI system of yours.”

Graziano frowned. He did not understand why Two Eyes was being aggressive to the Suspect Agent Identifier & Tracker. It’s job – his whole team’s job – was to help the agency’s overseas operatives by finding “enemy” agents and their supports.

“You don’t get it, do you? Graziano, your SAIT system is being used against us. The Chinese plant people for you to find – agent provocateurs. You asked Kelsie to investigates your suspects. What happens? The Chinese knew she was working for us. She was blown, thanks to your team.”

Graziano did not know how to respond. He had worked to protect undercover operatives, not to expose them. His mouth opened and closed erratically, and by the time he had found words Two Eyes had left. Graziano pecked uncertainly at the salad in its bowl. His mouth had dried and there was a pulsing tension from his chest muscles.

When he got back to the office his assistants were working on refining the funnel. The funnel was a filtering mechanism that sifted through the vast lists of people who had been in the wrong place at the wrong time, and there was a one in a hundred thousand chance that they might be involved in espionage. But if the same thing had happened several times, then SAIT looked at their profiles. And if SAIT was suspicious of a person, the details were passed to the local intelligence officer for them to check. That’s how Kelsie had been caught: checking on suspected spies.

Graziano could not concentrate. “It must be something I ate,” he said to the others. By mid afternoon he could leave it no longer. He walked the echoing corridors to his research manager. Karina Kristofferson was not just his line manager, she’d worked in the foreign intelligence service for long enough to have seen successes and disasters, and to know the political tricks that people played on each other.

“Yes?” Karina said as he arrived in her office. Karina had shortcut hair, thin eyebrows and a taste for elegant jackets and blouses which she adorned with a single piece of frill or feminine colour. Their love for good clothes was the favourite topic of conversation in social contexts.

Graziano explained.

“You’re upset? You blame yourself?”

“Yes. Our job is to help protect the field workers. Now I realise that what we’re doing is killing them. So yes, I’m more than upset.”

“Try not to blame yourself.” Karina tilted her head to one side in the way she did when sensitive private matters were being discussed.

“What you’ve hit is called an unknown unknown. We don’t have anything except one clue, and that clue may be entirely misleading.”

“Unknowns? I don’t know. I have known spies, I have suspect spies, and there are the people we’ve not yet identified as spies. Is that what you mean by ‘unknown unknowns’?”

“I’d like you to be my Devil’s Advocate on this.”

Graziano thought about his wife. She was strongly Catholic. The idea of him being a Devil’s Anything could upset her.

“Good. Here’s what we’re going to do. You are going to check the null hypothesis, and I’m going to work on a contingency.”

“Check nulls?” Graziano asked. He had wanted something much stronger.

“Yes. That’s why it’s called a Devil’s Advocate. I want you to look to see if there are other cases when someone was exposed after their name was added to your lists and passed to the local officer. And I’ll get information about Kelsie’s case, to see how the Chinese identified her.”

“What do I tell my research assistants?” Graziano asked.

“Nothing. It would just stress them. And they might start worrying about their jobs if they thought you unit could be closed down. So think of yourself as a Secret Devil’s Advocate.”

Closed down? The idea had not occurred to Graziano. But now that he considered it, he wondered whether that was inevitable. Kelsie had been exposed.

The research Karina described took days. They could not drop their existing routine, and some of the things she needed took time to find. Graziano collected the information, and he kept positive with his team. But every time an email arrived with capital letters his heart jumped as he wondered whether it was another arrest, or something worse.

His blood pressure increased and he started getting palpitations on the muscles on his left side, and he feared he was going to have a stroke. But his heart was steady and he had none of the other symptoms of a stroke and end up looking like Two Eyes. At night his dreams were disturbed and he tossed so much that his wife put pillows in the centre of the bed to protect herself. She complained he was grinding his teeth.

The next meeting with Karina was formal. She had brought someone from Security, and there was Kelsie’s superior, an analyst, and on the video monitor their department head was watching. Graziano tried not to look at the monitor – the Head’s emotionless face always made him feel uncomfortable.

Kelsie’s activities had definitely been what alerted the Chinese. They had caught her twice in suspicious circumstances, increased their own surveillance, and then caught her again when she was trying to recruit a local “asset”. They might have left her in place, watching to see who she met and what she was doing; but one of their own spies had been arrested, so in revenge their politicians decided to arrest three Western spies. Kelsie had been unlucky to be one of the three.

Have agent provocateurs been used before? “Yes,” Graziano explained. “There are reports from friendly intelligence agencies of similar things.” He was trying to sound official, like a Devil’s Advocate should sound. But he had never met a Devil’s Advocate, and the one’s in films came from Hollywood.

Had other intelligence workers been identified this way? Graziano had checked his friends and contacts. “Yes, certainly,” he answered, summarising their observations.

“This is no longer an unknown unknown,” Karina stated by way of summing up. “It’s jumped up to a known risk.”

“Is my unit going to be shut down?” Graziano asked.

Karina stopped looking at him. She looked at her department head on the monitor. “Do I have permission to proceed with Plan Beetlejuice?” she asked. The Head gave a tiny nod of approval.

“Plan Beetlejuice is for your team to continue, for the time being,” Karina explained to Graziano. “We tell our field operatives to pause on their own cross-checking of suspects. And while that’s happening, we look at risk-reduction mechanisms, options, and what this is costing us in money and losses. It’s extra work, and I’ll boost your team size while that’s happening.”

“So I’m no longer a Secret Devil’s Advocate?” he asked. He noticed the department head’s raised eyebrows. Perhaps being a Secret Devil’s Advocate was itself a secret.

“Keep questioning,” Karina said. “Keep looking for options.”

“Thank you,” Graziano said. He heard his voice. It was croaking. Her plan sounded like a final burst of effort before the closure of his unit. He wondered what he would be doing after his team was shut down.

The days turned to weeks. Nobody else was arrested, and there were no new rumours. And the Plan Beetlejuice research accumulated. Whether the accumulation of evidence was good or bad, Graziano could not tell. On some days he was excited, but most days were bad and he felt it was all going backwards. He tried to comfort himself by saying he never wanted to see another case like Kelsie. Perhaps closure would be best.

He eyed the final decision date as it steadily crept nearer. Graziano went to inspect the room where the final meeting would take place. It was large, with chairs that could be set out in a horseshoe to face towards one person. At least he’d have a last chance to argue for his little unit. What he feared most was a pre-emptive decision for reasons they labelled as secret.

On the night before the final day, Graziano barely slept. “Don’t take it personally,” his wife told him, “and just move on to whatever they give you next.” But she didn’t know the details, or about Kelsie. And he never mentioned the words Secret Devil’s Advocate.

Thirty minutes before the meeting was due to start, he received a notification of a change of meeting venue to Karina’s office. Her office table only had enough space for three people. And the ventilation was bad.

There was just Karina in the room. “Hello, Graziano. Somebody else wanted the big room.” She caught his glance at the empty seat. “It’s just us. Close the door.”

The obvious hit him: decisions required several people, but bad news is delivered by one person. “Any news of Kelsie?” he asked. He tried to sound professional.

“She’s had a consular visit. There are clear signs of level 2 interrogation, her strength is holding up well, and indications are that she has retained absolute secrecy about us and our work. We’re negotiating for her return, but it will take time.”

“And my unit? What will become of us?”

“The crisis brought your unit to the attention of senior people. There has been a financial review of the benefits compared to the costs.”

“You’re cancelling us?”

“On the contrary, this was an accident waiting to happen. It can be avoided with new processes and more care. So hopefully there’ll be no more Kelsie’s.”

“My team doesn’t get disbanded?” He was having trouble believing it.

“No. Our assessment also taught us something else: we could use the technique for counter-intelligence. We’ll be doubling your budget.”

He felt empty, and powerless. “Thank you.”

“You don’t sound excited.”

“I didn’t expect this. I’d been loosing hope.”

“I didn’t want to give you false hope. Graziano, why are you looking at me like that?”

“Sorry, I did not mean to show my thoughts. I wish I’d known you had a plan. We need hope, even when we know the odds are bad.” He realised he’d just criticised his superior for being heartless.

Karina’s face went poker neutral. There was no apology and the discussion became matter-of-fact.

Later Graziano was at a window seat in the canteen, eating alone while contemplating his world. A movement nearby made him turn. It was Two Eyes. “I told you that system of yours needed an overhaul,” Two Eyes said with a blatant lack of memory. “That money you’re going to be spending should have been going to keep more people in the field.”

Graziano looked at Two Eyes. More people in the field meant more casualties. “I measure success in lives – the people we protect while doing their job.”

Graziano could not concentrate. “It must be something I ate,” he said to the others. By mid afternoon he could leave it no longer. He walked the echoing corridors to his research manager. Karina Kristofferson was not just his line manager, she’d worked in the foreign intelligence service for long enough to have seen successes and disasters, and to know the political tricks that people played on each other.

“Yes?” Karina said as he arrived in her office. Karina had shortcut hair, thin eyebrows and a taste for elegant jackets and blouses which she adorned with a single piece of frill or feminine colour. Their love for good clothes was the favourite topic of conversation in social contexts.

Graziano explained.

“You’re upset? You blame yourself?”

“Yes. Our job is to help protect the field workers. Now I realise that what we’re doing is killing them. So yes, I’m more than upset.”

“Try not to blame yourself.” Karina tilted her head to one side in the way she did when sensitive private matters were being discussed.

“What you’ve hit is called an unknown unknown. We don’t have anything except one clue, and that clue may be entirely misleading.”

“Unknowns? I don’t know. I have known spies, I have suspect spies, and there are the people we’ve not yet identified as spies. Is that what you mean by ‘unknown unknowns’?”

“I’d like you to be my Devil’s Advocate on this.”

Graziano thought about his wife. She was strongly Catholic. The idea of him being a Devil’s Anything could upset her.

“Good. Here’s what we’re going to do. You are going to check the null hypothesis, and I’m going to work on a contingency.”

“Check nulls?” Graziano asked. He had wanted something much stronger.

“Yes. That’s why it’s called a Devil’s Advocate. I want you to look to see if there are other cases when someone was exposed where they’ve been using your lists, and checking on suspects. And I’ll get information about Kelsie’s case, to see how the Chinese identified her. It’s the possibility that the Chinese identified her for completely different reasons, and this was merely a small bit of extra evidence.”

“What do I tell my research assistants?” Graziano asked.

“Nothing. It would stress them. And they might start worrying about their jobs if they thought you unit could be closed down. So think of yourself as a Secret Devil’s Advocate.”

Closed down? The idea had not occurred to Graziano. But now that he considered it, he wondered whether that was inevitable. Kelsie had been exposed.

The research Karina described took days. They could not drop their existing routine, and some of the things she needed took time to find. Graziano collected the information, and he kept positive with his team. But every time an email arrived with capital letters his heart jumped as he wondered whether it was another arrest, or worse.

His blood pressure increased and he started getting palpitations on the muscles on his left side, and he feared he was going to have a stroke. But his heart was steady and he had none of the other symptoms of a stroke. At night his dreams were disturbed and he tossed so much at night that his wife put pillows in the centre of the bed to protect herself. She complained he was grinding his teeth.

The next meeting with Karina was formal. She had brought someone from Security, and there was Kelsie’s superior, an analyst, and on the video monitor their department head was watching. Graziano tried not to look at the monitor – the Head’s emotionless face always made him feel uncomfortable.

Kelsie’s activities had definitely been what alerted the Chinese. They had caught her twice in suspicious circumstances, increased their own surveillance, and then caught her again when she was trying to recruit a local “asset”. They might have left her in place, watching to see who she knew and what she was doing, but there had been a political directive from the highest levels to arrest three spies and imprison and question them. No reason was given. She had been unlucky to be one of the three.

Had it been done before? “Yes,” Graziano explained. “There reports from friendly intelligence agencies of similar things.” He was trying to sound official, like a Devil’s Advocate should sound. But he had never met a Devil’s Advocate, and the one’s in films came from Hollywood.

Had other intelligence workers been identified this way? Was it just time until they were exposed? Graziano had checked his friends and contacts. “Yes, certainly,” he answered, repeating their opinions.

“This is no longer an unknown unknown,” Karina stated by way of summing up. “It’s jumped up to a known risk.”

“Is my unit going to be shut down?” Graziano asked.

Karina stopped looking at him. She looked at her department head on the monitor. “Do I have permission to proceed with Plan Beetlejuice?” she asked. The Head gave a tiny nod of approval.

“Beetlejuice is to continue for the time being,” Karina explained to Graziano. “We tell our field operatives to pause on their own cross-checking of suspects. And while that’s happening, we look at risk reduction mechanisms, options, and what this is costing us in financial and other terms. It’s extra work, and I’ll boost your team size while that’s happening.”

“So I’m no longer a Secret Devil’s Advocate?” he asked. He noticed raised eyebrows from the security chief. Perhaps being a Secret Devil’s Advocate was itself a secret.

“Keep questioning. Keep looking for options.”

“Thank you?” Graziano said. He heard his voice. It was croaking. Her plan sounded like a final burst of effort before the closure of his unit. He wondered what he would be doing after his team was shut down.

The days turned to weeks. Nobody else was arrested, and there were no new rumours. And the “Beetlejuice” research accumulated. Whether the accumulation of evidence was good or bad, Graziano could not tell. On some days he was excited, and then there was a bad day and it felt like it was all going backwards.

He eyed the final decision date as it steadily crept nearer. Graziano went to inspect the room where the final meeting would take place. It was large, with chairs that could be set out in a horseshoes to face towards one person. At least he’d have a last chance to argue for his little unit. What he feared most was a pre-emptive decision for reasons they labelled as secret.

On the night before the final day, Graziano barely slept. “Don’t take it personally,” his wife told him, “and just move on to whatever they give you next.” But she didn’t know the details, or about Kelsie. And he never mentioned the words Secret Devil’s Advocate.

Thirty minutes before the meeting was due to start, he received a notification of a change of meeting venue to her office. Her office table only had enough space for three people. And the ventilation was bad.

There was just Kristina in the room. “Hello Graziano. Somebody else wanted the big room.” She caught his glance at the empty seat. “It’s just us. Close the door.”

He tried to be professional. “Any news of Kelsie?” he asked.

“She’d had a consular visit. There are clear signs of level 2 interrogation, her strength is holding up well, and indications are that she has retained absolute secrecy about us and our work. We’re negotiating for her return, but it will take time.”

“And my unit? What will become of us?”

“The crisis brought you unit to the attention of senior people. There has been a financial review of the benefits compared to the costs.”

“You’re cancelling us?”

“On the contrary, we’ve decided to triple the size of your unit. This was an accident waiting to happen. It can be avoided with new processes and more care. So hopefully there’ll be no more Kelsie’s. Our assessment also taught us something else: it’s possible to adapt your techniques to identify the network of likely contacts around foreign enemy agents. If we can find just a few of them, it will be much cheaper than doing it with human intelligence.”

“I’m still here?”

“Am I still a Secret Devil’s Advocate?”

Karina smiled. It was brief, but natural. “No. Mission accomplished.”

Later Graziano was at a window seat in the canteen, eating along while contemplating his world. A movement nearby made him turn. It was Two Eyes. “I told you that system of yours was complete rubbish and needed an overhaul,” Two Eyes said with a blatant lack of memory. “That money you’re going to be spending should have been going to keep more people in the field.”

Graziano looked at Two Eyes. More people in the field meant more casualties. “I measure success in lives – how to do our job with the least losses.”

Commentary on the Secret Devil’s Advocate story

This story was written to illustrate unknown-unknowns within espionage, supporting the ideas set out in 4 ways to manage unknown unknowns and their opportunities”

Graziano is the investigator. Titles such as this vary with context. Karina refers to him as a Devil’s Advocate.

Two Eyes, at the start, is a gainsayer, complete with selective memory.

Karina is the angel. From her wording she went straight to her department head to explain the risks and extra effort needed. She would have realised that mistakes can be avoided with better processes. And she was clearly on the lookout for a way to justify expanding Graziano’s team. But her lack of information to Graziano hurt him.

Although not directly obvious, the sponsor is her department head. For someone like Karina, every cost has to be justified and every major risk explained upwards.

Graziano and Karina use some of the standard methods – see Lesson #4. At the start, Graziano tests for the null hypothesis and Karina looks to identify potential causes. That provided enough evidence for the case to jump quickly from an “unknown unknown” to a manageable “known unknown”, and that allowed Karina to justify the costs of research – elimination. Given the tight timescale, they are likely to have done the easiest tests first and they may also have needed to breakdown bigger questions into smaller ones (although neither are illustrated in the short story). From her actions, Karina was clearly watching for inter-dependencies with other types of intelligence activity, although she kept the detail secret until the end.

The idea behind the story

The idea for this story came from a bit of reverse logic. In January 2021, David Omand, a former head of GCHQ, published a book on How Spies Think (https://www.penguin.co.uk/books/312/312746/how-spies-think/9780241385180.html). It has an exceptionally detailed and up-to-date account of practices for research, analysis and how the executive use this. It has the detail I can use in my storytelling, but it’s focussed on known-knowns. That’s inevitable, given how effort is focussed in the intelligence profession. But I also have a fascination in the unknown, as in the Rumsfold quote (https://en.wikipedia.org/wiki/There_are_known_knowns). So it was irritation and fascination that drove me to create this story.

“The Secret Devil’s Advocate” involves fictional people. No secrets are used in this story.

“The Secret Devil’s Advocate” involves fictional people. No secrets are used in my stories.

For another story on unknown-unknowns, see An agency man – a story of an ex intelligence officer in industry.