Tony Thorne was one of the officers on the Apollo project, advising the team on the task of merging large volumes of data. Thorne, a former counterterrorism officer with the Wales Extremism and Counter Terrorism Unit, said he was shocked by what he saw in Scotland. “We left Scotland with a process that was not complete or adequate in any way,” he said.
The key issues highlighted during the test run reemerged right away, according to emails and internal memos from 2014 and 2015 reviewed by BuzzFeed News.
Officers described a system that was “crashing routinely” and “timing out after 10 minutes,” with glitches so severe that they dramatically increased “the time required to perform a simple task.”
Even basic searches caused trouble. One officer had described how he put in a search term and received a result that was too broad. He started sifting through the documents manually to figure out which ones he actually needed — but as he did so, the system crashed. When he logged back in, he typed in the same search terms and found that “the search result was not the same.”
Officers using the new systems also reported serious difficulties with the very problem the NCIA was aiming to solve: communicating with other forces and agencies. After a suspicious person entered the UK by plane, an officer reported that they received an important intelligence report from officers at the airport in an unreadable format. Another told a member of the Apollo team that the NCIA’s inability to share intelligence with the other regions still using the old system was a critical risk that “may lead to intelligence failure.”
The quality of intelligence that did make it onto the system was often poor. In some cases, the NCIA was deluged with irrelevant information; in others, vital intelligence did not show up on the NCIA at all. One officer complained that the system “auto ingests” documents that had nothing to do with terrorism. “This issue was something that was always talked about,” the officer wrote, “however now we are live there appears to have been nothing more done about it.”
The NCIA was built on the template of an already existing system called the Home Office Large Major Enquiry System, four sources told BuzzFeed News. The problem, one said, is that HOLMES is used to investigate incidents that have already occurred whereas the NCIA is intended to prevent attacks from happening. Another officer told BuzzFeed News that building the NCIA on top of the HOLMES system caused defects that left large amounts of intelligence difficult to find.
Officers echoed these concerns in their emails and official reports. One of the key features borrowed from the HOLMES system was a search tool, much like Google, that was supposed to enable officers to quickly retrieve documents containing a certain word, regardless of where on the record the particular word appeared. If it worked, this would make it far easier to find specific intelligence on potential terrorists from hundreds of thousands of files.
But the search tool wasn’t working. Officers found that if they put in the same search term on multiple occasions, they would often get a different result each time. The search tool also wasn’t able to scan for dates of birth, making it much harder to pinpoint the right document.
This shortcoming dovetailed with another major problem. Early on, it became clear that many duplicated records would make their way onto the NCIA — since it was compiling data from multiple forces that often possessed the same file on a given individual. One internal report seen by BuzzFeed News acknowledges that this would cause a “knock on” effect that hampered analysts. But higher-ups ultimately decided that “no de-duplication would occur” until the entire UK was using the NCIA.
One Manchester-based officer who later started using the NCIA told BuzzFeed News that duplications made finding what you were looking for like “trying to find a needle in a haystack” — such a struggle that “you could miss vital intelligence leads.”
Thorne, the counterterror detective who worked on the NCIA, was growing increasingly concerned. “Unfortunately,” he wrote to colleagues in a February 2014 email, “as we are all fully aware the NCIA has struggled to deliver what was promised and has not been fit for purpose.”
The rollout of the NCIA pressed ahead.
