ML-based Reinforcement Learning Approach for Power Management in SoCs

This paper presents a machine learning-based reinforcement learning approach, mapping Finite State Machines, traditionally used for power management control in SoCs, to Markov Decision Process (MDP)-based agents for controlling power management features of Integrated Circuits with application to complex multiprocessor-based SoCs such as CPUs, APUs and GPUs. We present the problem of decision-based control of a number of power management features in ICs consisting of numerous heterogeneous IPs. An infinite-horizon fully observable MDPs are utilized to obtain a policy of actions maximizing the expectation of the formulated Power Management utility function. The approach balances the demand for desired performance while providing an optimal power saving as opposed to commonly used FSM-based power management techniques. MDP framework was employed for power management decision-making under conditions of uncertainly for reinforcement learning. We describe in detail converting power management FSMs into infinite-horizon fully observable MDPs. The approach optimizes itself using reinforcement learning based on specified reward structure and previous performance, yielding an optimal and dynamically adjusted power management mechanism in respect to the formulated model.