Project Data Express

Advanced materials characterization techniques such as electron microscopy and x-ray tomography generate ever-increasing quantities of experimental data as instruments modernize and improve. In lockstep, research scientists are developing increasingly sophisticated analysis algorithms to exploit the information contained within these different datasets.

The computational challenges represented by this frontier are myriad – efficient acquisition, storage, transfer and processing of data is typically beyond the skill-set of an experimental scientist. Moreover, facilitating efficient data transformation pipelines frees the researcher to focus on the science itself, reducing the overall time from experiment to insight.

Work Package 5 – Data Express – aims to minimize the time-to-result in nanoscale materials characterization, with a strong focus on Transmission Electron Microscopy (TEM) applications. One objective is real-time processing of data streams from fast electron cameras, fully utilizing all available computational resources on one or more systems, all while being practical to use for an experimental scientist with perhaps limited computer science knowledge.

The work package contributes to development on the open-source LiberTEM project (https://libertem.github.io/), originally developed at FZJ, which provides a standardized framework for parallelization of 4D-STEM processing algorithms. The extension to real-time processing takes place in the associated LiberTEM-live project (https://libertem.github.io/LiberTEM-live/).