Speaker
Description
In the LHD experiment, physical data is managed by the Kaiseki Server, or the Analyzed Data Server. In the past, the registration of physical data was done by each researcher in charge of measurement, but since many analysis programs require other physical data as input data, calculations could not be performed until the necessary data were registered. Therefore, the registration of the physical data was sometimes delayed.
To facilitate the registration of physical data after the experiment, automatic Integrated Data Analysis (aIDA) was developed. This system enabled data analysis without delay by starting an analysis program that uses a certain physical data as input data when it is registered. As a result, aIDA now have registered about 8 million pieces of data, or 45% of the total 18 million pieces of data
On the other hand, with the increase in the number of analysis programs managed by aIDA, the following problems have arisen: 1) each program uses a different version of the library and data analysis tools, and it is difficult to provide the run-time environment for each program 2) temporary files created during program execution remain, and waste disk spaces.
In order to solve these problems, we decided to run each program in a container using Docker. Each container can have a different execution environment, such as libraries, and analysis tools. In addition, the files created during execution are deleted at the end of execution, so there is no increase in the number of files.
Furthermore, by running in a container, the execution environment can be easily moved to other PC, and when a large amount of computation is required for batch processing, the processing speed can be easily improved by increasing the number of PCs used to run the computation. Container technology is also useful for transplanting the technology developed on the LHD to other experimental devices, and we are currently working on containerization of the analysis server. In this presentation, the current status of LHD analysis tools using container technology will be presented.
| Speaker's Affiliation | National Institute for Fusion Science |
|---|---|
| Member State or IGO | Japan |
