1、 Rec. ITU-R SM.1809 1 RECOMMENDATION ITU-R SM.1809 Standard data exchange format for frequency band registrations and measurements at monitoring stations (2007) Scope To support frequency management and the work of the ITU-R in general, monitoring and measurement campaigns are conducted. These campa
2、igns produce large amounts of data that in many cases have to be compared or merged. This document describes a standard exchange format for frequency scan based monitoring data. The ITU Radiocommunication Assembly, considering a) that the success of monitoring campaigns lays in the total effort of t
3、he participating administrations and the combination of their monitoring and measurement data; b) that monitoring equipment, computer equipment and software is not standardized and the data is stored in many different, often proprietary, formats; c) that successful merging or combining monitoring da
4、ta not only depends on the data format in which the data is stored but also on the environmental and technical conditions under which the data is gathered, recommends 1 that data gathered during monitoring campaigns should be converted to the format as described in Annex 1 before it is exchanged; 2
5、that software for automated spectrum monitoring should contain the possibility to store its data or convert its data to the format as described in Annex 1; 3 that administrations should discuss the conditions under which a monitoring or measurement campaign is conducted and data is exchanged before
6、starting the campaign. Annex 1 Standard data exchange format for frequency band registrations and measurements at monitoring stations 1 General aspects The format described is a format derived from the radio monitoring data format (RMDF) used by the radio astronomy service to exchange spectrum data.
7、 The format is a line based ASCII file with linefeed carriage returns after each line. Although the data files can be compressed in an efficient way no specification for compression is included in this Recommendation nor in the RMDF specification. 2 Rec. ITU-R SM.1809 2 Standard data format The data
8、 file should consist of two sections: A “Header” section containing the static information relating to the monitoring task such as the location used for monitoring, time information and key monitoring parameters. A “Data” section containing all the measured results during the period of observation.
9、In the original RMDF specification two separate files were used for the header and data section. In this Recommendation only one file is used to guarantee the link between header and data. The name of the format is CEF: common data exchange format. 2.1 Header section The following fields and fieldna
10、mes should be used. All appropriate data fields should be included in the header area before the measured results are added. The header section can contain three types of information Essential, Optional and Additional Optional (marked E, O and AO in Table 1). Optional means that space is reserved in
11、 the header but the field containing the data is left blank. TABLE 1 Header fields Type Fieldname Data format Array(1)Description Example E FileType Text N Type and/or version of the datafile Common exchange format V2.0 E LocationName Text N Name of the location where the measurements are made NERA
12、E Latitude Text N DD.MM.SSx where “x” is “N” or “S” 52.10.04N E Longitude Text N DDD.MM.SSx where “x” is “E” or “W” 005.10.09W E FreqStart Numeric (real) Y Frequency (kHz) 1000.000 E FreqStop Numeric (real) Y Frequency (kHz) 2000.000 E AntennaType Text, Numeric (real), Numeric (real) Y Info, gain (d
13、Bi), Kfactor (dB/m) The gain and kfactor fields can be omitted if not used LPD, 7, 10 E FilterBandwidth Numeric (real) Y In kHz 0.2 E LevelUnits Text N dBuV, dBuV/m or dBm (note that “u” is used instead of “”) dBuV E Date Text N Date of measurements in the format YYYY-MM-DD (start date if measuremen
14、ts span midnight). Note that time is also stored in each line in the datasection 2006-06-25 E DataPoints Numeric (integer) Y Number of data elements in the data row (analyzer data points or receiver steps) 80000 Rec. ITU-R SM.1809 3 TABLE 1 (end) Type Fieldname Data format Array(1)Description Exampl
15、e E ScanTime Numeric (real) N The actual time taken (s) for the equipment to scan from FreqStart to FreqStop. For a digital system using fast Fourier transform (FFT) this time is the time needed to sample the data block 24.1 E Detector Text N RMS O Note Text N General comments O AntennaAzimuth Text
16、Y DDD.DD (0 = North) 181.12 O AntennaElevation Text Y DD.DD (0 = no elevation) 45.32 O Attenuation Numeric (integer) Y Equipment attenuator setting (dB) 3 O FilterType Text Y Filtertype bandwidth and shapefactor. For a digital system using FFT the window type used can be specified here Gaussian 3 dB
17、 shapefactor 3.2 O DisplayedNote Text N A small remark of less than 40 characters containing essential information which could be displayed next to the data on any final report O Multiscan Text N Y or N If this optional field is not present the value is automatically N AO Measurement Accuracy Numeri
18、c N Total accuracy of the system AO VideoFilterType Text Y Video Filtertype bandwidth and shapefactor (1)An explanation can be found in 2.4. Additional Optional fields are fields that may be added to the header in order to provide further information, however, these will not be automatically process
19、ed or recognised by the transfer software. The header and data sections should be separated by ONE blank line. 2.2 Data section The data area should consist of scans where for each scan a separate line is used. Each line should contain the start time of the scan in HH:MM:SS format converted to UTC (
20、or local time if requested by the coordinator) followed by a level reading for each analyzer frequency point or receiver frequency step, all separated by commas. For a monitoring campaign where no exact level measurement data is required the signal level values can be rounded to the nearest integer
21、value to decrease the size of the data file. For measurement campaigns an accuracy of one decimal should be used. A decimal point should always be used since the comma is used as separator. The first whitespace in each line separates the descriptor from the variable. 2.3 Example file FileType Standa
22、rd Data exchange Format 2.0 LocationName NERA 4 Rec. ITU-R SM.1809 Latitude 52.00.00N Longitude 005.08.00W FreqStart 7000 FreqStop 7200 AntennaType Inverted V FilterBandwidth 0.5 LevelUnits dBuV/m Date 2006-06-25 DataPoints 80000 ScanTime 7.5 Detector RMS Note This is a sample file to demonstrate th
23、e data format. 00:00:00,65,56,64,54,23,29,32,43,54,25,29,25,36etc,43,59 00:00:10,64,53,65,59,42,37,35,34,64,25,26,36,63etc,54,61 00:00:20,62,57,64,59,41,36,26,42,53,62,16,52,24etc,52,66 . . . etc . . . 23:59:30,53,33,61,44,25,44,36,26,46,24,26,24,63etc,29,56 23:59:40,54,32,62,48,24,42,35,26,24,64,24
24、,34,35etc,29,56 23:59:50,64,52,63,57,33,23,32,53,25,26,63,35,26etc,32,59 2.4 Multiscan For specific applications it can be necessary to scan multiple small frequency segments with large gaps in between. This optional field determines if the data file contains more than one of these segments. When th
25、is value is set to Y the fields indicated with Y in the column “array” change from one value to an array of values. The individual values in the array are separated by a semicolon. For example for part of the header of a multiscan file: FileType Common Exchange Format 2.0 this field will not change
26、FreqStart 3100;7000;5000.2 this field will change to an array of, in this case, 3 values FreqStop 3200;7200;5100.1 this field will change to an array of, in this case, 3 values The same is the case with the datasection. One line with 3 scans will look like this: 23:59:50,64,52,63,57,33,23,26,etc,38,
27、55; ,64,52,63,57,33,23,26,etc,32,46; ,64,52,63,57,33,23,26,etc,55,23 Note that only one timestamp is used for the complete array of scans and the scantime in the header is the total time to complete the array of scans. Another application of multiscan is channelscan. Start and stopfrequency are defi
28、ned equal so only one frequency is scanned. The line in the datasection now contains the scanned frequencies separated by semicolons. 2.5 Transfer software Various administrations use different data formats and should develop or obtain transfer software to translate their data format to and from the
29、 common exchange format. Depending on the structure of the internal data format, this transfer software may be a simple macro file or a complex program to convert the actual data to a derived data structure. Monitoring packages could include an integrated Rec. ITU-R SM.1809 5 transfer function but a
30、 freely distributable separate tool for each monitoring package could contribute to a smooth exchange of data between administrations. FIGURE 1 Schematic presentation of data transfer to and from the common exchange format 2.6 Verifying the integrity of the datafile Before importing and using a data
31、 file submitted by a third party it is recommended to check the integrity and compliance with the dataformat specification. The best way to do this is to write a small tool that checks the presence of all necessary fields. It should also check the date time integrity of the sequence of scans and the
32、 number of valid datapoints in each scan. The header can contain a number of optional and additional optional fields so a modified tool or script for particular measurement campaigns may be needed and distributed among participants. 2.7 Organizing and indexing a large number of measurement datafiles
33、 The header section of the datafile contains sufficient information to generate unique indicators for each measurement file. Recommended is to use Date LocationName and Note and if needed FreqStart and FreqStop so a list of indexed measurements could look like Fig. 2. 3 Considerations for starting a
34、 measurement/monitoring campaign In addition to the use of a standard data format it is recommended to agree on some basic issues before starting a monitoring or measurement campaign to make the exchange and effective use of each others monitoring data possible. The first set of items, given in Tabl
35、e 2, consists of technical and logistic issues to be agreed on. There is a strong relationship between some of the technical parameters and also the information content and measurement accuracy of the gathered data is influenced by equipment settings. The second set of items given in Table 3 and con
36、sists therefore of equipment issues directly influencing the generated data. 6 Rec. ITU-R SM.1809 FIGURE 2 Example of a list of indexed measurement files TABLE 2 General technical and logistic parameters Parameter Considerations Dates/times of measurements Data to be merged or compared should be gat
37、hered simultaneously or in a defined sequence Wanted geographic location To exclude or to make use of propagation effects Frequency range (FreqStart, FreqStop) As desired. Note that in many cases there is a direct relationship between the frequency span, the frequency resolution of measurements and
38、the measurement time Duration of monitoring This will vary depending on the task Revisit time This is the time between the measurements on each unique frequency as described in Recommendation ITU-R SM.1536. It should be short enough to detect the shortest duration of a transmission of interest. To p
39、revent unnecessary over sampling it is a good idea to agree on a defined revisit time Antenna (AntennaType) Directivity, gain, and antenna pattern should be chosen according to the measurement campaign and within certain boundaries the same for all participating stations Detector (Detector) The choi
40、ce of detector depends on the type of signal to be measured. Short pulse type signals may be best measured with a peak or sample detector but a noise measurement campaign for example should use an RMS detector. In analogue receivers/analyzers the detectors integration or sample time depends on the u
41、sed filter bandwidth and within the measurement time for a frequencystep these samples are converted in peak, r.m.s, average etc. values. In FFT based analyzers the detector function is based on the processing of successive frequencyscans but the result is the same Rec. ITU-R SM.1809 7 TABLE 3 Equip
42、ment parameters influencing the gathered data Parameter Considerations Number of frequency points in each scan (DataPoints) The chosen number of points should guarantee sufficient frequency resolution but also compatibility with other participants in the campaign. Interpolation or extrapolation to o
43、btain a uniform number of points per scan can be necessary Filter bandwidth (FilterBandwidth) To ensure that all frequencies are monitored with minimal overlap a bandwidth of about 120% of the stepsize is recommended for a scanning receiver. This of course depends fully on the shapefactor of the fil
44、ter. For digital equipment using FFT the used window and the number of data points within the window determines the frequency resolution. For (semi)analogue analyzers the 3 dB points of the Gaussian filter should overlap Scan Time (ScanTime) This is the time the equipment needs to scan from FreqStar
45、t to FreqStop. It is always shorter than the revisit time Attenuation (Attenuation) A high input attenuation increases the noise floor and should be avoided. On the other hand overloading of the receiver can cause blocking and intermodulation The attenuator setting should be as low as possible, depe
46、nding on local conditions. A setting of 0 dB is not recommended because the input impedance of the receiver is not defined resulting in a high measurement uncertainty RF dynamic range and reference level A sufficient dynamic range and reference level should be chosen to cope with both the strongest and weakest received signals. The chosen dynamic range and reference level puts the values to be presented in predetermined limits
