Since 18 of December 2019 conferences.iaea.org uses Nucleus credentials. Visit our help pages for information on how to Register and Sign-in using Nucleus.

29 November 2021 to 6 December 2021
Virtual event
Europe/Vienna timezone
30 Nov - 3 Dec, 2021 Abstract submission open NOW

Near real-time streaming analysis of big fusiondata

3 Dec 2021, 13:50
15m
Virtual event

Virtual event

Regular Oral Big data Friday 3 Dec

Speaker

Ralph Kube (Princeton Plasma Physics Laboratory)

Description

While experiments on fusion plasmas produce high-dimensional data time series with
ever increasing magnitude and velocity, turn-around times for analysis of this
data have not kept up. For example, many data analysis tasks are often performed
in a manual, ad-hoc manner some time after an experiment. In this article we
introduce the DELTA framework that facilitates near real-time streaming analysis
of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, DELTA allows computationally
expensive data analysis tasks to be performed in between plasma pulses. This article describe the modular and expandable software architecture
of DELTA and present performance benchmarks of individual components as well as of an example workflows. Focusing on a streaming analysis workflow where ECEi data measured at KSTAR on NERSC's supercomputer we routinely observe data transfer rates of about 500 Megabyte per second.
At NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes in under
5 minutes. We further discuss how DELTA uses modern database systems and container orchestration services to provide web-based real-time
data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models.
By providing session leaders and physics operators results of higher order data analysis using live visualizations may make more informed
decisions on how to configure the machine for the next shot.

Country or International Organisation United States of America
Affiliation Princeton Plasma Physics Laboratory

Primary authors

Ralph Kube (Princeton Plasma Physics Laboratory) Dr Michael Churchill (Princeton Plasma Physics Laboratory) Dr CS Chang (Princeton Plasma Physics Laboratory) Dr Jong Choi (Oak Ridge National Laboratory) Dr Jason Wang (Oak Ridge National Laboratory) Dr Scott Klasky (Oak Ridge National Laboratory) Dr Laurie Stephey (Lawrence Berkeley National Laboratory) Dr Eli Dart (Lawrence Berkeley National Laboratory) Dr Minjun Choi

Presentation materials