Speaker
Description
Across research disciplines, the size, complexity, and heterogeneity of data sets is growing rapidly. Large projects are often long-running collaborations between evolving groups of researchers. Integrating and documenting these large heterogeneous data collections is a requirement for these data to be exploited, and for them to retain their value and meaning over time. The traditional approach to these data management problems has been to have hard- coded, application-specific solutions. The Navigational Data Management (NDM) project provides a general solution that can be applied regardless of the science domain. NDM represents data and annotations as generalized objects, with relationships stored as labeled property graphs. Universal Resource Identifiers (URIs) provide a mechanism to refer to external data objects like files, MDSplus records, or drawings. The system stores metadata and data schemas and application behaviors as first class objects, allowing them to be modified, added to, and even annotated as the needs of the research group evolve. This new toolset is being applied to initial projects at several institutions as its development continues. NDM is coded as a single page application (SPA) using modern javascript front-end tools and a graph database as the back-end.
NDM was developed using an informal agile, prototype and iterate methodology. We will lay out the overall project design and how it changed in response to both technical challenges and initial user input. As the set of use cases, objects and relationship types grows, we revisit and generalize the software to accommodate them. This process will continue in order to become a system that can be applied across science domains and applications.
