Presentation
Advanced HPC Workflows for Urgent and Interactive Computing Using Julia
Presenter
DescriptionModern data-driven discovery algorithms and workflows require the tight interpretation of Simulation, Data Analysis, and AI. This means that all too often modern workflows fail to mesh well with HPC environments which are optimized for isolated applications over integrated workflows; and high utilization over fast feedback.
An example of this is real-time data analysis for experiment steering: time at large scientific instruments (such as particle accelerators, electron microscopes, or telescopes) is a scarce resource. Yet modern instruments often produce data at a rate that outpaces local computing resources. Therefore, scientists are turning to live data processing at HPC centers in order to gain the necessary insight to effectively steer their experiments.
In this talk, we demonstrate how Julia’s unique language features make it a natural language for developing tightly integrated Simulation + Analysis + AI workflows. We will also show an example of a workflow that flexibly grows its pool of compute nodes on an HPC system, thereby overcoming the constraints of the resource scheduler.
An example of this is real-time data analysis for experiment steering: time at large scientific instruments (such as particle accelerators, electron microscopes, or telescopes) is a scarce resource. Yet modern instruments often produce data at a rate that outpaces local computing resources. Therefore, scientists are turning to live data processing at HPC centers in order to gain the necessary insight to effectively steer their experiments.
In this talk, we demonstrate how Julia’s unique language features make it a natural language for developing tightly integrated Simulation + Analysis + AI workflows. We will also show an example of a workflow that flexibly grows its pool of compute nodes on an HPC system, thereby overcoming the constraints of the resource scheduler.
TimeWednesday, June 513:00 - 13:30 CEST
LocationHG E 3
Session Chairs
Event Type
Minisymposium
Climate, Weather, and Earth Sciences
Physics
Computational Methods and Applied Mathematics