基于深度Q神經網絡（DQN）的空調冷卻水系統無模型優化
Model-free optimization for air conditioning cooling water systems based on deep Q network (DQN)
摘要：
在建筑空調水系統的優化控制領域，基于模型的控制方法得到了廣泛的研究和驗證。但基于模型的控制很大程度上依賴于精確的系統性能模型和足夠的傳感器，而這對于某些建筑來說是很難獲得的。針對這一問題，本文提出了一種基于深度Q神經網絡（DQN）的空調冷卻水系統無模型優化方法，該方法以室外空氣濕球溫度、系統冷負荷及冷水機組開啟狀態為狀態，以冷卻塔風機和水泵的頻率為動作，以系統性能系數（COP）為獎勵。根據實際系統的實測數據進行建模，在模擬環境中使用基于粒子群優化算法的模型優化方法、基于Q值（Q learning）優化的強化學習方法和基于DQN的無模型優化方法進行實驗，結果表明基于DQN的無模型優化方法的優化效果最好，有7.68%的平均COP提升與7.15%的節能率，在復雜系統下擁有較好的節能效果。
Abstract：
Model-based control methods have been widely investigated and validated in the domain of optimal control for building air conditioning water systems. However, the performance of model-based control highly depends on accurate system models and enough sensors, which are difficult to obtain in some buildings. To overcome this problem, a model-free optimization method for air conditioning cooling water system based on deep Q network (DQN) is proposed. The wet bulb temperature of outdoor air, system cooling load and chiller on/off states are taken as the states, the frequencies of cooling tower fans and cooling water pumps are taken as the actions, and the reward is the system COP. In the simulation environment built by the measured data of an actual system, the model optimization method based on particle swarm optimization, the reinforcement learning method based on Q-value (Q learning) optimization and the model-free optimization method based on DQN are used to conduct experiment. The results show that the model-free optimization method based on DQN has the best optimization effect with 7.68% average COP improvement and 7.15% energy saving rate, which has a better energy saving effect in complex systems.
Keywords:model-freeoptimization;deepQnetwork(DQN);coolingwatersystem;optimalcontrol;energyconsumption