Balancing Artificial Intelligence with Deep & Dark Web Expertise
Human-powered analysis serves as the foundation for Business Risk Intelligence (BRI). Data is integral to what we do here at Flashpoint, which is why we’ve been actively cultivating a “get it done right” culture in terms of data science and data engineering. Through analytics to delivery, our data is processed through systems written by engineers with backgrounds in building data processing infrastructure, natural language processing systems, and automated data collection capabilities. The infrastructure we’re building is robust, elastic, and capable of processing large volumes of streaming data.
The Data Collection Conundrum
Organizing data from the Deep & Dark Web is a unique challenge, because the data comes from many different sources that require different collection techniques. This data reflect a variety of activities and engagements carried out on the Deep & Dark Web. Collecting and organizing data from these complex, hard-to-reach sources is a problem in and of itself—one that demands a well thought-out solution. By having well-structured sources in place, data scientists are able to spend less time cleaning up the data and more time actually engaging with the information contained within it.
Human Intelligence: BRI’s Gold Standard
Given the complex, dynamic nature of the data Flashpoint processes, there is no lack of innovative new ideas for our engineering team to explore. Structured data streams make it easy to quickly answer analytical questions like “what sources contain activity about a keyword?” or “which malicious file is being shared by whom and when?” Machine learning can provide valuable insight into the inner workings of the Deep & Dark Web. Insights gleaned from Flashpoint’s machine learning models are deemed inconclusive until verified by our subject-matter experts, looping in our gold standard of human-powered intelligence.
Explainability is Key
Since our data science initiatives are oriented around an overarching goal of understanding the Deep & Dark Web landscape through a justifiable lens, it’s important for our team to have a granular understanding of how our models work, both mathematically and intuitively. Therefore, it’s crucial that our models provide sample data points or analysis to justify each choice they make. If we’re able to understand the algorithm’s rationale for making a decision, we can tweak the parameters moving forward if anything is amiss. By continually iterating on the development of our models, we’re able to make them more accurate and reliable over time.
Putting the Pieces Together
Data science allows our team make sense of vast droves of data gleaned from the Deep & Dark Web, but it’s just one piece of the puzzle. This is especially true when working with unsupervised machine learning algorithms, which produce output that requires further interpretation from a subject-matter expert. For example, if a clustering algorithm groups an individual with other actors on the Deep & Dark Web based on their similarities with regards to certain attributes, our analysts would need to verify the accuracy of this classification, assess its relevance, and provide further elaboration. Here at Flashpoint, we are fortunate enough to have an unparalleled level of in-house subject-matter expertise to back up or dispute any intelligence claims made by data science.
A Culture of Learning and Growth
Flashpoint is a company dedicated to the collection, processing, and understanding of data. This is made clear by the engineering team’s learning and growth-oriented culture. Beyond the day-to-day sprint work of our core responsibilities, our engineering team hosts lectures and research paper reading groups to discuss new ideas and brainstorm potential initiatives to support the ongoing implementation of data science across every facet of the company. As someone with a passion for data science, I am fortunate to be part of a team where I am able to learn from and contribute to Flashpoint’s dynamic and fast-paced growth.
Flashpoint Analyst Team
The Flashpoint analyst team is composed of subject-matter experts with tradecraft skills honed through years of operating in the most austere online environments, training in elite government and corporate environments, and building and leading intelligence programs across all sectors. Our team covers more than 20 languages including Arabic, Mandarin, Farsi, Turkish, Kazakh, Spanish, French, German, Russian, Ukrainian, Italian, and Portuguese.