1、Designation: D5157 97 (Reapproved 2014)Standard Guide forStatistical Evaluation of Indoor Air Quality Models1This standard is issued under the fixed designation D5157; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of las
2、t revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon () indicates an editorial change since the last revision or reapproval.1. Scope1.1 This guide provides quantitative and qualitative tools forevaluation of indoor air quality (IAQ) models. These toolsinclu
3、de methods for assessing overall model performance aswell as identifying specific areas of deficiency. Guidance isalso provided in choosing data sets for model evaluation and inapplying and interpreting the evaluation tools. The focus of theguide is on end results (that is, the accuracy of indoorcon
4、centrations predicted by a model), rather than operationaldetails such as the ease of model implementation or the timerequired for model calculations to be performed.1.2 Although IAQ models have been used for some time,there is little guidance in the technical literature on theevaluation of such mod
5、els. Evaluation principles and tools inthis guide are drawn from past efforts related to outdoor airquality or meteorological models, which have objectives simi-lar to those for IAQ models and a history of evaluationliterature.(1)2Some limited experience exists in the use ofthese tools for evaluatio
6、n of IAQ models.2. Referenced Documents2.1 ASTM Standards:3D1356 Terminology Relating to Sampling and Analysis ofAtmospheres3. Terminology3.1 Definitions: For definitions of terms used in thisstandard, refer to Terminology D1356.3.2 Definitions of Terms Specific to This Standard:3.2.1 IAQ model, nan
7、 equation, algorithm, or series ofequations/algorithms used to calculate average or time-varyingpollutant concentrations in one or more indoor chambers for aspecific situation.3.2.2 model bias, na systematic difference between modelpredictions and measured indoor concentrations (for example,the mode
8、l prediction is generally higher than the measuredconcentration for a specific situation).3.2.3 model chamber, nan indoor airspace of definedvolume used in model calculations; IAQ models can bespecified for a single chamber or for multiple, interconnectedchambers.3.2.4 model evaluation, na series of
9、 steps through which amodel developer or user assesses a models performance forselected situations.3.2.5 model parameter, na mathematical term in an IAQmodel that must be estimated by the model developer or userbefore model calculations can be performed.3.2.6 model residual, nthe difference between
10、an indoorconcentration predicted by an IAQ model and a representativemeasurement of the true indoor concentration; the value shouldbe stated as positive or negative.3.2.7 model validation, na series of evaluations under-taken by an agency or organization to provide a basis forendorsing a specific mo
11、del (or models) for a specific applica-tion (or applications).3.2.8 pollutant concentration, nthe extent of the occur-rence of a pollutant or the parameters describing a pollutant ina defined airspace, expressed in units characteristic to thepollutant (for example, mg/m3, ppm, Bq/m3, area/m3, or col
12、onyforming units per cubic metre).4. Significance and Use4.1 Using the tools described in this guide, an individualseeking to apply an IAQ model should be able to (1) assess theperformance of the model for a specific situation or (2)recognize or assess its advantages and limitations.4.2 This guide c
13、an also be used for identifying specific areasof model deficiency that require further development or refine-ment.5. Components of Model Evaluation5.1 The components of model evaluation include the fol-lowing: (1) stating the purpose(s) or objective(s) of theevaluation, (2) acquiring a basic underst
14、anding of the specifi-cation and underlying principles or assumptions, (3) selecting1This guide is under the jurisdiction of ASTM Committee D22 on Air Qualityand is the direct responsibility of Subcommittee D22.05 on Indoor Air.Current edition approved Sept. 1, 2014. Published September 2014. Origin
15、allyapproved in 1991. Last previous edition approved in 2008 as D5157 97 (2008).DOI: 10.1520/D5157-97R14.2The boldface numbers in parentheses refer to the list of references at the end ofthis standard.3For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Servi
16、ce at serviceastm.org. For Annual Book of ASTMStandards volume information, refer to the standards Document Summary page onthe ASTM website.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States1data sets as inputs to the evaluation process,
17、 and (4) selectingand using appropriate tools for assessing model performance.Just as model evaluation has multiple components, modelvalidation consists of one or more evaluations. However,model validation is beyond the scope of this document.5.1.1 Establishing Evaluation Objectives:5.1.1.1 IAQ mode
18、ls are generally used for the following: (1)to help explain the temporal and spatial variations in theoccurrences of indoor pollutant concentrations, (2) to improvethe understanding of major influencing factors or underlyingphysical/chemical processes, and (3) to predict the temporal/spatial variati
19、ons in indoor concentrations that can be expectedto occur in specific types of situations. However, modelevaluation relates only to the third type of model useprediction of indoor concentrations.5.1.1.2 The most common evaluation objectives are (1)tocompare the performance of two or more models for
20、a specificsituation or set of situations and (2) to assess the performanceof a specific model for different situations. Secondary objec-tives include identifying specific areas of model deficiency.Determination of specific objectives will assist in choosingappropriate data sets and quantitative or q
21、ualitative tools formodel evaluation.5.1.2 Understanding the Model(s) to be Evaluated:5.1.2.1 Although a model user will not necessarily know orunderstand all details of a particular model, some fundamentalunderstanding of the underlying principles and concepts isimportant to the evaluation process.
22、 Thus, before evaluating amodel, the user should develop some understanding of thebasis for the model and its operation. IAQ models cangenerally be distinguished by their basis, by the range ofpollutants they can address, and by the extent of temporal orspatial detail they can accommodate in inputs,
23、 calculations, andoutputs.5.1.2.2 Theoretical models are generally based on physicalprinciples such as mass conservation. (2, 3) That is, a massbalance is maintained to keep track of material entering andleaving a particular airspace. Within this conceptualframework, pollutant concentrations are inc
24、reased by emis-sions within the defined volume and by transport from otherairspaces, including outdoors. Similarly, concentrations aredecreased by transport exiting the airspace, by removal tochemical/physical sinks within the airspace, or for reactivespecies, by conversion to other forms. Relations
25、hips are mostoften specified through a differential equation quantifyingfactors related to contaminant gain or loss.5.1.2.3 Empirical models (3) are generally based on ap-proaches such as least-squares regression analysis, using mea-surements under different conditions across a variety ofstructures,
26、 at different times within the same structure, or both.Theoretical models will generally be suitable for a wide rangeof applications, whereas empirical models will generally beapplicable only within the range of measurements from whichthey were developed.5.1.2.4 Some combination of theoretical and e
27、mpirical com-ponents is also possible. Specific parameters of a theoreticalmodel may have relationships with other factors that can bemore easily quantified than the parameters themselves. Forexample, the rate of air infiltration into a structure coulddepend on outdoor windspeed and the indoor-outdo
28、or tempera-ture difference, or the emission rate from a cigarette coulddepend on the combustion rate and the constituents of theparticular brand smoked. Given sufficient data, such relation-ships could be estimated through techniques such as regressionanalysis.5.1.2.5 IAQ models may be specified for
29、 a particular pol-lutant or in general terms; this distinction is important, forexample, because particle-phase pollutants behave differentlyfrom gas-phase pollutants. Particulate matter is subject tocoagulation, chemical reaction at surfaces, gravitationalsettling, diffusional deposition, resuspens
30、ion and interception,impaction, and diffusional removal by filtration devices;whereas some gaseous pollutants are subject to sorption and, insome cases, desorption processes.5.1.2.6 Dynamic IAQ models predict time-varying indoorconcentrations for time steps that are usually on the order ofseconds, m
31、inutes, or hours; whereas integrated models predicttime-averaged indoor concentrations using average values foreach input parameter or averaging these parameters during thecourse of exercising the model. Models can also differ in theextent of partitioning of the indoor airspace, with the simplestmod
32、els treating the entire indoor volume as a single chamber orzone assumed to have homogeneous concentrations through-out; more complex models can treat the indoor volume as aseries of interconnected chambers, with a mass balance con-ducted without each chamber and consideration given tocommunicating
33、airflows among chambers.5.1.2.7 Generally speaking, as the model complexity growsin terms of temporal detail, number of chambers, and types ofparameters that can be used for calculations, the users task ofsupplying appropriate inputs becomes increasingly demanding.Thus users must have a basic unders
34、tanding of the underlyingprinciples, nature and extent of inputs required, inherentlimitations, and types of outputs provided so that they canchoose a level of model complexity providing an appropriatebalance between input effort and output detail.5.1.2.8 A number of assumptions are usually made whe
35、nmodeling a complex environment such as the indoor airspace.These assumptions, and their potential influence on the mod-eling results, should be identified in the evaluation process.One method of gaining insights is by performing sensitivityanalysis.An example of this technique is to systematically
36、varythe values of one input parameter at a time to determine theeffect of each on the modeling results; each parameter shouldbe varied over a reasonable range of values likely to beencountered for the specific situation(s) of interest.5.1.3 Choosing Data Sets for Model Evaluation:5.1.3.1 A fundament
37、al requirement for model evaluation isthat the data used for the evaluation process should beindependent of the data used to develop the model. Thisconstraint forces a search for available data pertinent to theplanned application or, if no appropriate data sets can be found,collection of new data to
38、 support the evaluation process. Suchdata should be collected according to commonly recognizedand accepted methods, such as those given in the compendiumdeveloped by the U.S. Environmental Protection Agency (4).D5157 97 (2014)25.1.3.2 The following series of steps should be used inchoosing data sets
39、 for model evaluation: (1) select situationsfor applying and testing the model; (2) note the model inputparameters that require estimation for the situations selected;(3) determine the required levels of temporal detail (forexample, minute-by-minute or hour-by-hour) and spatial detail(that is, numbe
40、r of chambers) for model application as well asvariations of the contaminants within each chamber; and (4)find or collect appropriate data for estimation of the modelinputs and comparison with the model outputs.5.1.3.3 Thus, the information required for the evaluationprocess includes not only measur
41、ed indoor concentrations at anappropriate level of temporal detail, but also suitable estimatesfor required input parameters. Among the inputs typicallyrequired are outdoor concentrations, indoor emission and sinkrates, coagulation coefficients, deposition rates and diffusioncoefficients for particl
42、es, and rates of airflow between indoorand outdoor airspaces (as well as flows among multiple indoorairspaces, if a multichamber model is used). If suitable data tosupport the choice of inputs are not available, the alternativesare as follows: (1) to compress the level of temporal detail formodel ap
43、plication to that for which suitable data can beobtained; (2) to provide best estimates for model inputs,recognizing the limitations imposed by this particular ap-proach; or (3) to collect the additional data required to enableproper estimation of inputs.5.1.4 Tools for Assessing Model Performance:5
44、.1.4.1 The tools to be used in assessing the performance ofIAQ models all involve comparisons between indoor concen-trations predicted by the model, Cp, and observedconcentrations, Co, comprising the data set(s) used for evalu-ation. These tools can be quantitative, involving various typesof statist
45、ical indexes, or qualitative, involving plots of Cp, Co,or differences between the two (that is, model residuals). Thetools presented below are classified by use for (1) assessing thegeneral agreement between predicted and observed concentra-tions and (2) assessing bias in the mean or variance ofpre
46、dicted values relative to that for observed values.5.1.4.2 The following tools are to be used for assessing thegeneral agreement between Cpand Co:(1) Correlation coefficient, r, ranging from 1 to 1, with 1indicating a strong, direct relationship between Cpand Co,0indicating no relationship, and 1 in
47、dicating a strong butinverse relationship. The formula to be used for calculating thiscoefficient (5, 6) is as follows:r 5(i51nCoi2 Co!Cpi2 Cp!#/ (1)(i51nCoi2 Co!2#F(i51nCpi2 Cp!2Gwhere the summation extends across all Cpand Copairs andCoand Cpare averages (that is, Co5(i51nCoi/n, where n is thenumb
48、er of observed values).(2) Line of regression, the best-fit relationship between Cpand Co, ideally exhibiting a slope, b, of one and an intercept, a,of zero. Formulas to be used in calculating the slope andintercept are as follows:b 5(i51nCoi2 Co!Cpi2 Cp!#/(i51nCoi2 Co!2# (2)a 5 Cp2 b!Co!# (3)(3) No
49、rmalized mean square error (NMSE), a measure ofthe magnitude of prediction error relative to Cpand Co. Theformula to be used for calculating this measure2is as follows:NMSE 5 Cp2 Co!2/Co!Cp!# (4)where:Cp2 Co!25(i51nCpi2 Coi!2/n.The NMSE will have a value of 0 when there is perfectagreement for all pairs of Cpand Coand will tend towardhigher values as Cpand Codiffer by greater magnitudes. Forexample, if Cpand Codiffer consistently by 50 %, the NMSEvalue will be near 0.2; for differences of 100 %, the NMSEvalue will be near 0.5; for d
