Fifth Technical Meeting on Fusion Data Processing, Validation and Analysis

IMAS simulation management and remote data access for ITER

14 Jun 2023, 16:00

30m

Ghent University, Ghent, Belgium

Oral Information retrieval, dimensionality reduction and visualisation in fusion databases DB/2 Information retrieval, dimensionality reduction and visualisation in fusion databases

Speaker

Jonathan Hollocombe (UKAEA)

Description

While the Integrated Modelling & Analysis Suite (IMAS [1]) is being developed further, the number of simulations available in IMAS is increasing both at ITER and within the Members. As a result, tools are needed to manage, curate and expose the large IMAS simulation databases to the potential users. Ad-hoc solutions have been developed in some cases, such as the ITER scenario database (2500+ simulations) and its associated Python scripts that register some meta-data (stored in a yaml file associated with each simulation) and then browse or query this recorded information to identify the simulations of interest. This solution, while simple to set up, is not generic (scripts are tightly bound to ITER while IMAS allows for simulation on any machine), is not meant to scale to a very large number of simulation counts (queries need to go through all yaml files) and lacks important features.

In order to address this need for a general purpose tool for managing simulations in IMAS, a simulation data management tool (SimDB), has been developed. Using SimDB simulations are ingested along with meta-data that records the input, output and information about the code that captures the simulation provenance and ensures reproducibility of the simulation. Each simulation is given a globally unique identifier (UUID), and can be pushed to one or more remote archives where the data can be validated and made available to other users. The ingested simulations to be queried via a command-line interface or via a web frontend using a flexible query syntax. A SimDB catalogue has been set-up at ITER with meta-data from the Dataset Description and Summary IDSs being made available and queryable, making the SimDB meta-data directly interoperable with other IMAS-based catalogues.

In addition to SimDB, the IMAS access-layer has been extended to allow for remote access of IMAS data, using the simulation URIs returned from the SimDB queries. Based on the UDA client-server solution, this will allow for secured authentication of users and for full or partial access to the IDS data as well as controllable batching of requests to improve the performance depending on the capabilities of the network. The public SimDB servers, the SimDB unique identifier and the remote access URI provides a method to unambiguously refer to IMAS data in publications, improving the FAIRness of IMAS.

This poster will present the new ITER simulation management and remote access solutions and detail how they will facilitate FAIR data access to the ITER simulation catalogue.

[1] F. Imbeaux, Nucl. Fusion 55 (2015) 123006