Energy-aware resource allocation for data analysis workflows in heterogeneous cluster environments Journal Article uri icon

abstract

  • Scientific workflows automate complex data analysis processes and are increasingly employed to handle large-scale datasets. Scientific communities leverage heterogeneous computing clusters, consisting of interconnected resources with diverse computational capabilities, to execute their large-scale data analysis workflows due to privacy and economic factors. Consequently, they often impose high computational demands and lead to significant energy consumption. Designing an effective energy-aware scheduling approach is therefore crucial for heterogeneous computing systems, as it enables high processing efficiency while minimizing overall energy usage. However, achieving both generality in workflow structures and solvability of the corresponding optimization models remains a major challenge in workflow scheduling. In this paper, we introduce a novel mathematical approach that models the non-overlapping execution of tasks successfully by representing disjoint time intervals as linear constraints. By applying this technique, we formulate a mixed-integer linear programming model that can be solved within a reasonable time, providing exact optimal solutions for arbitrary DAG workflow structures whenever computationally tractable. We analyze the trade-off between energy consumption and makespan. In addition, post-optimality analyses, including sensitivity analysis and relaxation of optimal energy consumption for makespan improvement, are investigated. Experimental results demonstrate that the proposed MILP approach consistently outperforms competing methods, achieving up to approximately 6%–7% reduction in energy consumption compared with the best competing approach and up to about 10%–11% compared with the worst-performing method.

keywords

  • Data analysis workflow
  • Energy optimization
  • Heterogeneous cluster environments
  • Mixed integer linear programming
  • Post-optimality analysis
  • Scheduling

publication date

  • September 2026

volume

  • 51