Real-time peer-to-peer energy trading of multi-carrier energy buildings: A multi-agent deep reinforcement learning solution

Abstract

Multi-carrier energy buildings (MCEBs) integrated with renewable energy resources (RERs), energy storage systems (ESSs), and energy conversion technologies provide a flexible, economical, reliable, and environmentally friendly energy systems. However, independent operation of MCEBs may limit these benefits. Local peer-to-peer (LP2P) energy trading enhances these capabilities by enabling energy exchange among MCEBs. Implementing LP2P trading requires addressing key challenges such as privacy preservation, computational burden, and economic incentives. In this paper, a distributed market mechanism based on mid-market rate (MMR) pricing is adopted for LP2P energy trading among MCEBs within building communities (BCs). The complexity of such a mechanism results in a high computational burden and necessitates a robust method for handling uncertainties. Traditional model-based optimization struggles with uncertainty management and scalability issues. Therefore, a model-free artificial intelligence-based multi-agent deep reinforcement learning (MADRL) approach is employed. Specifically, a twin delayed deep deterministic policy gradient (TD3) algorithm is used to train the environment, enabling efficient handling of continuous state and action spaces. Numerical results demonstrate that the proposed LP2P energy trading framework is highly practical, ensuring low computational burden while preserving privacy. Moreover, the mechanism encourages peer participation, ultimately reducing MCEB operation costs.