By Jared Sagoff
Aspiring bakers are frequently called upon to adapt award-winning recipes based on differing kitchen setups. Someone might use an eggbeater instead of a stand mixer to make prize-winning chocolate chip cookies, for instance.
Being able to reproduce a recipe in different situations and with varying setups is critical for both talented chefs and computational scientists, the latter of whom are faced with a similar problem of adapting and reproducing their own ”recipes” when trying to validate and work with new AI models. These models have applications in many different fields.
“When we talk about data, we have a practical understanding of the digital assets we deal with,” said Eliu Huerta, scientist and lead for Translational AI at the U.S. Department of Energy’s (DOE) Argonne National Laboratory. ”With an AI model, it’s a little less clear; are we talking about data structured in a smart way, or is it computing, or software, or a mix?”
In a new study, Huerta and his colleagues have articulated a new set of standards for managing AI models. Adapted from recent research on automated data management, these standards are called FAIR, which stands for findable, accessible, interoperable and reusable.
“By making AI models FAIR, we no longer have to build each system from the ground up each time,” said Argonne computational scientist Ben Blaiszik. ”It becomes easier to reuse concepts from different groups, helping to create cross-pollination across teams.”
According to Huerta, the fact that many AI models are currently not FAIR poses a challenge to scientific discovery. ”For many studies that have been done to date, it is difficult to gain access to and reproduce the AI models that are referenced in the literature,” he said. ”By creating and sharing FAIR AI models, we can reduce the amount of duplication of effort and share best practices for how to use these models to enable great science.”
To meet the needs of a diverse community of users, Huerta and his colleagues combined a unique suite of data management and high performance computing platforms to establish a FAIR protocol and quantify the ”FAIR-ness” of AI models. The researchers paired FAIR data published at an online repository called the Materials Data Facility, with FAIR AI models published at another online repository called the Data and Learning Hub for Science, as well as with AI and supercomputing resources at the Argonne Leadership Computing Facility (ALCF). In this way, the researchers were able to create a computational framework that could help bridge various hardware and software, creating AI models that could be run similarly across platforms and that would yield reproducible results. The ALCF is a DOE Office of Science user facility.
Two keys to creating this framework are platforms called funcX and Globus, which allow researchers to access high performance computing resources straight from their laptops. ”FuncX and Globus can help transcend the differences in hardware architectures,” said co-author Ian Foster, director of Argonne’s Data Science and Learning division. ”If someone is using one computing architecture and someone else is using another, we now have a way of speaking a common AI language. It’s a big part of making AI more interoperable.”
In the study, the researchers used an example dataset of an AI model that used diffraction data from Argonne’s Advanced Photon Source, also a DOE Office of Science user facility. To perform the computations, the team used the ALCF AI Testbed’s SambaNova system and the Theta supercomputer’s NVIDIA GPUs (graphics processing units).
A paper based on the study, ”FAIR principles for AI models, with a practical application for accelerated high energy diffraction microscopy,” appeared in Nature Scientific Data on Nov. 10, 2022.
In addition to Huerta, other authors of the study include Argonne’s Nikil Ravi, Pranshu Chaturvedi, Zhengchun Liu, Ryan Chard, Aristana Scourtas, K.J. Schmidt, Kyle Chard, Ben Blaiszik and Ian Foster.
The research was funded by DOE’s Office of Advanced Scientific Computing Research, the National Institutes of Standards and Technology, the National Science Foundation and Laboratory Directed Research and Development grants.