Multi-advisor deep reinforcement learning for smart home energy control

Abstract

Effective automated smart home control is essential for smart-grid enabled approaches to demand response, named in the literature as automated demand response. At it’s heart, this is a multi-objective adaptive control problem because it requires balancing an appliance’s primary objective with demandresponse motivated objectives. This control problem is difficult due to the scale and heterogeneity of appliances as well as the time-varying nature of both dynamics and consumer preferences. Computational considerations further limit the types of acceptable algorithms to apply to the problem. We propose approaching the problem under the multi-objective reinforcement learning framework. We suggest a multi-agent multi-advisor reinforcement learning system to handle the consumer’s time-varying preferences across objectives. We design some simulations to produce preliminary results on the nature of user preferences and the feasibility of multi-advisor reinforcement learning. Further smarthome simulations are designed to demonstrate the linear scalability of the algorithm with respect to both number of agents and number of objectives. We demonstrate the algorithms performance in simulation against a comparable centrallized and decentrallized controller. Finally, we identify the need for stronger performance measures for a system of this type by considering the effect on agents of newly selected preferences.