Multi-objective hyperparameter optimisation for edge machine learning

Abstract

The high-luminosity upgrade of the Large Hadron Collider at Cern intensifies the demand for advanced trigger algorithms that provide rapid and correct decision-making for classifying particle jets. This challenge can be solved by applying high-performance edge computing. Field- Programmable Gate Array as a form of edge computing hardware can meet strict latency requirements, but they have limited resources for the implementations. Machine Learning models are typically large in size and suffer from high memory and computational requirements, making them difficult to exploit directly on Field-Programmable Gate Arrays. To obtain a high-performance model that meets the hardware-related restrictions, hyperparameter optimisation has an important role. To conduct multi-objective hyperparameter optimisation for edge machine learning based on the two aforementioned issues, this thesis compares the current popular hyperparameter optimisation methods and libraries to appropriately select one for multi-objective tasks. By conducting multiple hyperparameter optimisation experiments, two new loss functions were designed to satisfy different multi-objective tasks for achieving best model performance. The experiments with the Tree-structured Parzen Estimator and the Optuna and hls4ml libraries demonstrate that by defining the loss function appropriately, it is feasible to implement multi-objective hyperparameter optimisation.