IAEA Symposium on International Safeguards

OSIS 2.0: Optimizing Analyst-Driven Automation of Open Source Information Collection and Processing

6 Nov 2018, 11:36

1m

Boardroom A (Vienna International Centre)

Contributor (Panel Session) Leveraging technological advancements for safeguards applications (TEC) [TEC] Automating and Optimizing Data Collection and Processing at HQ

Speaker

Mr Thomas Skoeld (IAEA)

Description

Collecting and processing open source (OS) information is an important aspect of the IAEA’s mandate to implement safeguards based on all relevant information related to States’ nuclear activities. Since the mid-1990s, the Division of Information Management (SGIM) has been collecting OS information into an internal database, the Open Source Information System (OSIS).

In the early stages, the SGIM collection and review process was predominately manual with classic internet searching, PDF printing, and running scripts for uploading files to the OSIS database. Over time the process has undergone numerous improvements to include elements of automation in order to increase efficiency in an ever-growing stream of open source information. While automation has unarguably been welcome for many of the processing steps, it has been essential to keep the analyst involved at key decision-making points such as judging information for relevance, categorization, and further distribution.

With the technological advancement of computing and machine learning during the past five years however, more options for additional automation of data processing have become available. In 2016, SGIM embarked on a project to integrate and further automate the continuous monitoring, collection, and processing of OS information.

This paper describes the process that culminated in the launch in early 2018 of OSIS 2.0, an in-house developed tool that has provided numerous improvements, including: automation of manual steps of collecting and formatting files; creation of a centralized space for analysts to collaborate on information collection and processing; and, improvement of the categorization and distribution capability. Furthermore, automation has enabled analysts to focus efforts more on analysis than collection and processing. The paper will also discuss possible next steps in integrating additional information collection processes into OSIS 2.0 and how far automation can be taken before it starts to have a diminishing effect on reliable information collection and processing.