1、 International Journal of Heating, Ventilating, Air-conditioning and Refrigerating Research Editor Reinhard Radermacher, Ph.D., Professor and Director, Center for Environmental Energy Engineering, Department of Mechanical Engineering, University of Maryland, College Park, USA Associate Editors Micha
2、el J. Brandemuehl, Ph.D., P.E., Professor, James E. Braun, Ph.D., P.E., Professor, Ray W. Herrick Laboratories, Alberto Cavallini, Ph.D., Professor, Dipartmento di Fisicia Tecnica, University of Padova, Italy Arthur L. Dexter, D.Phil., C.Eng., Professor of Engineering Science, Department of Leon R.
3、Glicksman, Ph.D., Professor, Departments of Architecture and Richard R. Gonzalez, Ph.D., Director, Biophysics and Biomedical Modeling Division, Anthony M. Jacobi, Ph.D., Professor and Associate Director ACRC, Department of Keith E. Starner, P.E., Engineering Consultant, York, Pennsylvania, USA Jean-
4、Christophe Visier, Ph.D., Head, Centre Scientifique et Technique du Btiment, Energy Management Automatic Controller Division, Mame La Valle, France Joint Center for Energy Management, University of Colorado, Boulder, USA School of Mechanical Engineering, Purdue University, West Lafayette, Indiana, U
5、SA Engineering Science, University of Oxford, United Kingdom Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, USA U.S. Army Research Institute of Environmental Medicine, Natick, Massachusetts, USA Mechanical and Industrial Engineering, University of Illinois, Urbana-Champaig
6、n, USA Policy Committee Editorial Assistant Stephen W. Ivesdal, Chair, Member ASHRAE P. Ole Fanger, Fellow/Lfe Member ASHRAE Ken-Ich Kimura, Fellow ASHRAE Kristie Blase W. Stephen Comstock Publisher ASHRAE Staff John W. Mitchell, Fellow ASHRAE Frank M. Coda, Member ASHRAE W. Stephen Comstock, Associ
7、ate Member ASHRAE Bany Publishing Manager Mildred Geshwiler, Special Publications Editor Erin S. Howard, Assistant Editor Christina Helms, Assistant Editor Michshell Phillips, Secretary 02003 by the American Society of Heating, Refrigerating and Air- Conditionine Engineers. Inc 1791 Tullie Circle. A
8、tlanta. Georeia passages or reproduce illustrations in a review with appropriate credit; nor may any part of this book he reproduced, stored in a retrieval system, II 30329. All rights reserved. Periodicals postage paid at Atlanta, Georgia, and additional mailing offces. HVAC Ei (Engineering Informa
9、tion, inc.) Ei Compcndex and Engineering Index; IS1 (Institute for Scientific Information) Weh Science and Research Alert; and BSRIA (Building Services Research however, that assumption is gener- ally not true. Therefore, in this work, the equivalent overall heat transfer coefficient (U,) has been d
10、efined as follows: By substituting Equation 7 into Equation 8, the final form of U, becomes The radiant heat transfer coefficient (Equation 10) found in the literature (ASHRAE 2000) is h, = 5 x lO-s(AUST+273)2+(T,m+273)2.(AUST+273)+(Tpm+273). (10) The approximate expression for AUST (Kilkis et al. 1
11、994) was used in this study. ZE when 26C I Toa I36“C, (Toa - 45) where d = room position index with values as noted below: 0.5 for an interior space, 1 .O for a room with one outdoor exposed side with fenestration less than 5% of the total room SUT- 2.0 for a room with fenestration greater than 5%,
12、and 3.0 for a room with two or more outdoor exposed sides. face area or The equivalent overall heat transfer coefficient (U,) defined in Equation 9 can be used in place of the overall heat transfer coefficient (U,) in the panel model; however, U, cannot be VOLUME 9, NUMBER 3, JULY 2003 255 180 160 1
13、40 T L 120 0“ 100 8 3 80 3 60 u 40 20 O m E .- -e- Min et al. (1 956) +AA and Hatton (2000) 4- Proposed correlaon qo (V=Om/s or NC) .f . , . . . . . ,. . . . , . . . . . I . “i . .“.,. . . . . . 12 14 16 18 20 22 24 26 Tfl CI Figure 4. Total Cooling Capacity and Radiation Heat Flux determined explic
14、itly because the mean panel surface temperature (T,) is unknown. This unknown Tpm can be determined by solving the panel model equations (Equations 4 through 7) and Equation 9 for given boundary conditions iteratively. Once U, and TPm have converged, other quantities, such as the panel cooling capac
15、ities (qo, qc, and q,) and heat transfer coeffi- cients (h, and h,) can be determined. CONVECTIVE AND RADIATIVE HEAT FLUX The total cooling capacity of the CRCP, when placed in the space illustrated in Figure 1, is presented in Figure 4. The panel heat transfer is strongly a function of air velocity
16、 due to the increasing convective heat transfer. As may be noted, the radiative heat transfer is essentially insensitive to the air velocity since the panels operate with a very small water temperature rise, or a nearly constant surface temperature. The convective heat fluxes calculated with the sim
17、plified correlation and Awbi and Hattons correlation closely agree. The rate of total cooling enhancement by considering the mixed convection effect is pre- sented in Figure 5. It shows that the total cooling capacity of a radiant panel can be enhanced dramatically by air motion. CONCLUSIONS Panel c
18、ooling capacity is enhanced significantly when mixed convection is considered. How- ever, when the diffuser discharge air velocity is less than 2 mis, the impact of mixed convection on the panel cooling capacity is small. Therefore, the correlation for the natural convection heat transfer coefficien
19、t can be used to estimate panel cooling capacity instead of the mixed convec- tion correlation for low velocities. 256 HVAC Part 1: Measuring of the performance with free flow. Fisher, D.E., and C.O. Pedersen. 1997. Convective heat transfer in building energy and thermal load calcu- Hottel, H.C., an
20、d A. millier. 1958. Evaluation of flat-plate collector performance. Trans. ofthe Confer- Kilkis, B.I., S.S. Sager, and M. Uludag. 1994. A simplified model for radiant heating and cooling panels. Kochendrfer, C. 1996. Standard testing of cooling panels and their use in system planning. ASHRAE Min, T.
21、C., L.F. Schutrum, G.V. Parmelee, and J.D. Vouris. 1956. Natural convection and radiation in a 578-585. Deutsches Institut fur Normung. lations. ASHRAE Transactions 103(2): 137-148. ence on the Use of Solar Energy 2(1): 74. University of Arizona Press. Simulation Practice and Theory 2(2): 61-76. Tra
22、nsactions 102( i): 651-658. panel heated room. Heating Piping and Air Conditioning (HPAC) May: 153-160. 258 HVAC Drees and Braun 1996; Massie 1998). Gregor P. Henze is an assistant professor of architectural engineering and Jobst Sehoenmann is a graduate student at the University of Nebraska, Omaha,
23、 Neb. 259 260 HVAC for a description of the results of the anal- ysis, see Henze et al. (2002). During a subsequent investigation phase presented in the companion papers Henze et al. (1 997a) and Henze and Krarti (1 999), it was determined to what extent the performance merits of optimal control can
24、 be retained when the optimal controller is subject to uncertainty in the external variables influencing the physical process, such as future weather variables and cooling loads. This predictive optimal strategy is based on closed-loop optimization; Le., an optimal storage charging and discharging s
25、trategy is developed at every time step over a marching plan- ning horizon utilizing updated forecasts. The prediction of climate conditions, utility rates, and cooling loads is updated at the beginning of each time step over the optimization period, and a new optimal strategy is computed. Only the
26、control action of the first hour is executed at each time step. Short horizons (= 21 hours) were found to be only slightly inferior relative to a strat- egy that is optimal over the entire simulation horizon (e.g., one month). Thus, there is no point in predicting further into the future than 24 hou
27、rs. An alternative approach to dealing with the dependency of predictive optimal control on accurate plant models and forecasters is to extract rules by analyzing prototypical optimal refer- ence cases. Like RL control, the rule-based controller does not require detailed plant models. Drees and Braw
28、 (1996) describe a rule-based controller appropriate for monthly billing periods, while Ruchti et al. (1996) investigate the relationship between forecaster certainty and controller performance. Henze and Dodier (2002) investigated learning control of a grid-independent photovoltaic system consistin
29、g of a collector, storage, and a load. The approach is based on Q-learning, a model-free reinforcement learning algorithm that optimizes control performance through explo- ration. Q-learning was used to find a strategy that performs better than a conventional control strategy with respect to a cost
30、function that places more weight on meeting a critical base load than on those noncritical loads exceeding the base load. In the investigation reported in the present paper, Q-learning as an approach to reinforcement learning is applied to the control of TES systems. Kretchmar et al. (2001) employed
31、 reinforcement learning assisted by artificial neural net- works to learn to improve multiple-input multiple-output (MIMO) control performance of a heating system within a stable environment guaranteed by robust control. VOLUME 9, NUMBER 3, JULY 2003 26 1 The original contribution of this work is th
32、e application of reinforcement learning-based opti- mal control to thermal energy storage systems. The authors acknowledge and build on previous theoretical work on reinforcement learning, in particular the original Q-learning algorithm by Watkins and Dayan (1992), the use of real variables by Gulla
33、palli (1 990), as well as continuous function approximation and model-based schemes as surveyed by Sutton and Barto (1998). Problem Statement Our goal is to design a controller that learns to charge and discharge a commercially available thermal energy storage system in a commercial building to expl
34、oit load-shifting utility rate incentives and yield operating cost savings. The cost function may be arbitrarily complex and tailored to the demands of the application. For example, a weighted average of utility cost and energy consumption may be the quantity to be minimized. The optimization is acc
35、omplished without a system model or forecasting model; the Q-learning algorithm derives a near-optimal control strategy from a history of empirical data. DESCRIPTION OF THE ANALYSIS Base Case We will state cost savings relative to the base case, which is a chilled water system that expe- riences the
36、 same cooling load and weather profiles and uses the same chiller and air-handler sub- ject to the same utility rate structure as the corresponding cool storage system. The only differences are that (I) the chiller is sized to fully meet the peak cooling load, i.e., it is not down- sized as for the
37、case of chiller-priority, and (2) there is no thermal energy storage system avail- able. This base case represents the standard case of a cooling system without cool storage and reveals the benefit a thermal energy storage system can provide under real-time pricing. The performance metric for all ca
38、ses is the cost of operating the central chilled water plant using electrical utility rate structures with energy charges ($/kWh) only; demand charges ($/kW) are not considered due to a focus on real-time pricing rates. Similarly, changes in operation and maintenance costs are ignored. Thermal Stora
39、ge System and Chilled Water Plant Modeling The defining feature of any storage system is its ability to bridge a temporal gap between sup- ply and demand. In a thermal energy storage system, the temporal occurrence of electrical cool- ing-related loads can be separated from that of the thermal (cool
40、ing) loads. Consequently, there is flexibility in the choice of the cooling source: either direct cooling from the chiller, or dis- charging the storage tank, or a combination of those. At night, when cooling is not usually required, the tank is recharged. For a description of the system model for t
41、he TES system dis- cussed in this report, refer to Henze et al. (1997b). In summary, the state-of-charge of the cool storage serves as the state variable x and the rate of charging and discharging as the control vari- able u. The charge/discharge rate is the control variable from which temperature s
42、etpoints and other relevant operational parameters within the chilled water plant can be derived. State transitions of the thermal storage system are described in discrete time by subject to the state constraints 262 HVAC choosing chilled water as the storage medium does not change the general concl
43、usions of this paper. The chiller load Qch,k is the cooling load Qk plus the control variable ub which may be positive (charging storage) or negative (discharging storage): VOLUME 9, NUMBER 3, JULY 2003 263 The electric input ratio EZR represents the electrical input required to produce 1 ton (3.517
44、 kW) of cooling. It is determined from manufacturers data and adjusted to EZR for environmen- tal conditions based on the assumption that the actual chiller coefficient of performance varies proportionally to that of a Carnot refrigeration cycle. This assumption has an error of no more than *lo% (He
45、nze and Krarti 1998). Furthermore, a linear relationship of the chiller capacity with ambient conditions as well as a second-order polynomial relationship of the chiller effi- ciency with part-load ratio PLR are assumed. Thus, we have and v a + bPLR + cPLR2 EIRchw,adj = PLR In this study, the coolin
46、g plant arrangement depends on the cooling load profile. In the pres- ence of significant base cooling loads, a base chiller continuously operates in chilled-water mode, and a dedicated TES chiller, operating in either chilled-water or icemaking mode, is responsible for load-shifting (dual-chiller m
47、ode). If base cooling loads are nonexistent or mar- ginal, only a TES chiller is provided (single-chiller mode). The duai-chiller configuration ana- lyzed in this report is shown in Figure l. Pumps, fans, cooling tower, and all other equipment needed to facilitate cooling are provided as well. Not s
48、hown in the functional sketch is a plate-frame heat exchanger that isolates the TES chiller and the storage tanks from the building distribution loop as well as a number of control valves. Rajacied Heal 1 r Cooling Tower , / , . - L 8 , I BeseChlllor i-. . ., Ice Storage Swm Thaerit ; _., To buildin
49、g cooling load Figure 1. Central Chilled Water System Schematic with Thermal Energy Storage 264 HVAC Shavlik and Dietterich 1990). Learning can make control more effective by (1) making predictions about the physical plant under control, (2) making predictions about the environment by observation and exploration, and (3) making predictions about the controller to enhance performance. While adaptive control adapts to changes in the environment, learning control exhibits the memory (usually imple- mented as a statistical summary) t
