Lotaru: local runtime prediction for scientific workflow tasks
Workflow
Overview
publications using this workflow item
purpose
- Estimates the runtime of scientific workflow tasks on the heterogeneous nodes of a cluster from measurements taken on a single local machine, so that a scheduler can reason about task placement without profiling every node. Microbenchmarks characterise each target machine, a Bayesian ridge regression relates local runtimes to input size, and the estimate is transferred to the target node. This run replays the evaluation of the Lotaru papers over the bundled execution traces of five nf-core workflows on six machines.
FONDA Responsible researcher
- Fabian Lehmann Alumni
- Jonathan Bader Graduate Student
- Odej Kao Faculty Member
- Ulf Leser Faculty Member
subproject
Run Metadata
workflow runs
Reproducibility
workflow code link
input datasets used
trace types
- Kubernetes Job and Pod status, container termination metadata, program log, provenance, and per-machine prediction CSVs
trace data format
- Kubernetes JSON, plain-text log, CSV, and conda package list
Execution Environment
workflow engine
- Java Workflow Engine
language
compute cluster
- FONDA Cluster Compute Cluster
backend
- Kubernetes Backend