SMPTE RDD 31-2014 Deferred Demosaicing of an ARRIRAW Image File to a Wide-Gamut Logarithmic Encoding.pdf

1、 Copyright 2014 by THE SOCIETY OF MOTION PICTURE AND TELEVISION ENGINEERS 3 Barker Avenue, White Plains, NY 10601 (914) 761-1100 Approved August 16, 2014 re Deferred Demosaicing of an ARRIRAW Image File to a Wide-Gamut Logarithmic Encoding The attached document is a Registered Disclosure Document pr

2、epared by the proponent identified below. It has been examined by the appropriate SMPTE Technology Committee and is believed to contain adequate information to satisfy the objectives defined in the Scope, and to be technically consistent. This document is NOT a Standard, Recommended Practice or Engi

3、neering Guideline, and does NOT imply a finding or representation of the Society. Errors in this document should be reported to the proponent identified below, with a copy to engsmpte.org. This document is intended to support the development of applications that read and process ARRIRAW image files.

4、 It is not intended to support the development of hardware or software applications that create ARRIRAW image files, and creation of such files is reserved to individuals and organizations that have entered into agreements with the proponent identified below for such file creation. All inquiries in

5、respect of this document (other than reporting of errors in the document, which should be handled as described in the prior paragraph), including inquiries as to intellectual property requirements that may be attached to use of the disclosed technology, should be addressed to the proponent identifie

6、d below. Proponent contact information: Joseph Goldstone Arnold both of these, however, are given in this document. It is the intent of this RDD to document the structure and interpretation of ARRIRAW files generated by ARRI cameras, so that users of this document may develop applications correctly

7、identifying and interpreting such files. This document is specifically not intended to support development of hardware or of software applications creating ARRIRAW files. Permission to create such files, along with additional documentation to support that creation, is reserved by ARRI to members of

8、the ARRI Partner Program, the contact information for which is provided below. Assistance in correctly processing ARRIRAW files, including certification that the results of processing meet ARRI quality standards, is available to members of the ARRI Partner Program. Information on the ARRI Partner Pr

9、ogram is available at the address given below: Digital Workflow Solutions group ARRI Partner Program Arnold Application of per-channel white balance factors to the linear 16-bit unsigned integer photosite data, with the factors being dependent on correlated color temperature and a magenta/green bias

10、 factor; Demosaicing of the white-balanced photosite data; Conversion of the reconstructed sensor RGB by a 3x3 matrix, with the matrix coefficients being dependent on correlated color temperature and the presence or absence of an internal camera filter, to ARRI Wide Gamut RGB, a wide-gamut RGB encod

11、ing with defined primaries and white point; Exposure compensation, with the amount of compensation being dependent on exposure index; and finally Logarithmic encoding, with scaling and offset factors of the encoding algorithm determined by exposure index. Each of these steps is described below. 5.1

12、Unpacking and Linearization of Near-Logarithmic Unsigned Integer Data from ARRIRAW File Every sequential block of 12 8-bit bytes corresponds to 3 sequential 32-bit words containing 8 near-logarithmic 12-bit unsigned integer photosite values. These 8 near-logarithmic 12-bit unsigned integer photosite

13、 values are unpacked to form 8 linear 16-bit unsigned integer photosite values using an algorithm such as (or equivalent to) that given in Section 5.1.2.1.2. SMPTE RDD 31:2014 Page 6 of 17 pages 5.1.1 Reading of Image File Data as Packed Data The packed data are read as 8-bit bytes, every successive

14、 4 of which are to be assembled into a 32-bit word. Each byte contains photosite data in two half-bytes each comprised of 4 bits. Such a 4-bit half-byte represents either the most significant 4 bits of a 12-bit near-logarithmic value, the next-most-significant 4 bits of a 12-bit near-logarithmic val

15、ue, or the least significant 4 bits of a 12-bit near-logarithmic value. In the figure, these three interpretations of a 4-bit half-byte are represented by a large trapezoid, a small trapezoid, and a triangle, respectively. In the assembly of a 32-bit word from 4 8-bit bytes, the least significant 8

16、bits are read first, and the most significant 8 bits are read last. The resulting 3 32-bit words represent 8 successive packed photosite values, suitable for input to the algorithm given in pseudocode in Section 5.1.2. 5.1.2 Unpacking of 32-bit Words into 12-bit Encoded Values The 12-bit quantities

17、that have been packed into 32-bit words are unpacked prior to processing. The algorithm whose pseudocode is given in Section 5.1.2.1.2 performs this unpacking. It assumes that successive 32-bit words are stored adjacently in memory, and that the unpacking algorithm is executed on a machine with a li

18、ttle-endian architecture, taking as input blocks of 3 32-bit words and producing as output blocks of 4 32-bit words. In the figure below, the top element represents the unpacked data addressed as 32-bit words and the bottom element represents the unpacked data addressed as 16-bit half-words. Every r

19、esulting 32-bit word of unpacked data contains two 12-bit near-logarithmic photosite values. The lower-order 12 bits of the least significant 16 bits of the 32-bit word contain the 12-bit near-logarithmic value corresponding to the photosite encountered earlier in sensor traversal; the lower-order 1

20、2 bits of the most significant 16 bits of the 32-bit word contain the 12-bit near-logarithmic value encountered later in sensor traversal. When addressed as 16-bit half-words, those same unpacked data are in sensor traversal order and are interpreted as unsigned integers representing 12-bit near-log

21、arithmic encoded photosite values. 5.1.2.1.1 Variable definitions NumPixel number of pixels in the unpacked image InSize memory used to store the 12-bit packed image OutSize memory size required to store the unpacked image SMPTE RDD 31:2014 Page 7 of 17 pages lpIn pointer to the input packed image m

22、emory lpOut pointer to the output unpacked image memory MSB_Pixel pixel in the 16 most significant bits of the output 32-bit memory word LSB_Pixel pixel in the 16 least significant bits of the output 32-bit memory word InWord input 32-bit packed word 5.1.2.1.2 Pseudocode implementation void ARI_UnPa

23、ck12BitBayerPattern(uint32 *lpIn, uint32 *lpOut, .) OutSize = 16*InSize/12; NumPixel = OutSize * 2; While (NumPixel 0) InWord = lpIn0; MSB_Pixel = (InWord 20) LSB_Pixel = (InWord 8) lpOut0 = (MSB_Pixel 28) LSB_Pixel = (InWord 16) lpOut1 = (MSB_Pixel 4) LSB_Pixel = (InWord 24) lpOut2 = (MSB_Pixel 12)

24、 LSB_Pixel = (InWord lpOut3 = (MSB_Pixel 16) lpIn += 3; SMPTE RDD 31:2014 Page 8 of 17 pages lpOut += 4; NumPixel -= 8; 5.1.3 Linearization of 12-bit Near-Logarithmic Unpacked Sensor Data Photosite values are radiometrically linear representations of the energy they receive, represented as 16-bit un

25、signed integers, incorporating an offset such that a photosite receiving no energy would have a 16-bit unsigned integer value of 256. When stored as ARRIRAW file image data, the 16-bit photosite values are encoded into 12-bit values using a method that approximates a logarithmic encoding, but emphas

26、izes the preservation of deep shadow detail. The following equation reconstructs a 16-bit linear photosite value from an encoded 12-bit value . ) ) where is the integer part of the quotient of divided by 512 is modulo 512 5.2 Application of Per-Channel White Balance Factors After first subtracting t

27、he cameras average black level (which is fixed at value 256 for all files whose processing is described by this document), the linear 16-bit unsigned integer photosite data are multiplied by the White Balance Factors in the Image Content Information subheader of the ARRIRAW file that is appropriate

28、to their type, after which the average black level is added back to the calculated products. The mapping of photosite type to corresponding White Balance Factor is as follows: Photosite Type Corresponding Factor G0 White Balance Green Factor R White Balance Red Factor B White Balance Blue Factor G1

29、White Balance Green Factor The White Balance Green Factor is unity: When the 16-bit unsigned integer values representing photosite data are converted to equivalent floating-point values, and the White Balance Factors and are applied using floating-point arithmetic to these floating-point red and blu

30、e photosite data values, the calculations can each be performed with three operations: ) ) SMPTE RDD 31:2014 Page 9 of 17 pages The black-subtracted intermediate results in each case are converted to floating-point prior to multiplication by the floating-point white balance factors, with the resulti

31、ng product converted back to 16-bit integer form prior to the addition of the previously-subtracted black level. When the White Balance Factors and are applied to red and green photosite data values using unsigned integer arithmetic, careful ordering of operations should be used to ensure that the l

32、ower extent of black noise (i.e. photosite values below 256) are not clipped. Three example sets of White Balance Factor Values are given below, with the Green/Magenta Tint presumed to be zero. CCT Tint White Balance Red Factor White Balance Green Factor White Balance Blue Factor 3200 0 1.128195 1 2

33、.068762 5600 0 1.644962 1 1.366723 6500 0 1.745829 1 1.252820 At time of capture these factors are calculated from the chosen White Balance CCT and Green/Magenta Tint values. 5.3 Demosaicing of the Photosite Data A demosaicing algorithm is applied to the photosite data to produce spatially correspon

34、dent RGB data. As the photosites are arranged in a classic Bayer pattern, there are many algorithms in the literature. For an introduction, see “Color Processing in Digital Cameras”, by Adams, Parulski and Spaulding, in IEEE Micro, vol. 18 (6), pp. 20-30. For a more varied treatment see the compilat

35、ion of papers in “Single Sensor Imaging”, Rastislav Lukac, Ed., CRC Press 2009. 5.4 Conversion to Wide-Gamut Encoding The reconstructed sensor RGB data are transformed to the ARRI Wide Gamut color space, a wide-gamut color space the primaries and white point of which are defined by the CIE x,y chrom

36、aticity coordinates given below. x y Red 0.6840 0.3130 Green 0.2210 0.8480 Blue 0.0861 -0.1020 White 0.3127 0.3290 A 3x3 matrix transforms sensor RGB to ARRI Wide Gamut RGB. The application of this matrix is performed on the result of a subtraction of the cameras average black level from the sensor

37、RGB data, and the cameras average black level is added to the result of the matrix application. When used in floating-point arithmetic calculations, this matrix is applied to the left of a column vector of sensor RGB whose values have SMPTE RDD 31:2014 Page 10 of 17 pages been reduced by the cameras

38、 average black level. The results of the matrix application are then augmented by the cameras average black level. ) When such floating-point calculations are performed the value of each of the three components of is fixed at value 256 for all files whose processing is described by this document. Th

39、e black-subtracted intermediate results are converted to floating-point prior to multiplication by the floating-point matrix, with the resulting product converted back to 16-bit integer form prior to the addition of the previously-subtracted black level. When matrix operations are applied to sensor

40、RGB using unsigned integer arithmetic, careful ordering of operations should be used to ensure that the lower extent of black noise is not clipped. The contents of a 3x3 matrix appropriate for the correlated color temperature recorded in the White Balance CCT field of the Image Data Information subh

41、eader are carried in metadata as the first three columns of the Color Matrix field of that same subheader. If those Color Matrix metadata fields have not been filled in, the following two tables give matrix coefficients for appropriate to the correlated color temperature recorded in the White Balanc

42、e CCT field of that same Image Data Information subheader. The choice of which table is used depends on the content of the ND Filter field of the Lens Data Information Subheader of the ARRIRAW image metadata. White Balance CCT values within the range described below, but falling between the listed C

43、CTs, should use matrices that are linearly interpolated between the immediately surrounding rows. A single interpolation parameter controls this interpolation, with an interpolated matrix being computed as M = ) where the interpolation factor for CCT being computed as a = and are the matrix and CCT

44、from one adjacent row and are the matrix and CCT from the other adjacent row The CCT values and should be converted to floating-point numbers prior to their reciprocal values being computed, and all subsequent mathematical operations computing the interpolated matrix should be performed using floati

45、ng-point arithmetic. These matrices are derived from the camera systems measured spectral response, from a presumed illuminant, and from data sets of real-world spectral reflectances. The sensor response of real-world cameras being invariably different from those of the CIE 1931 Color Matching Funct

46、ions, such a derivation process for a 3x3 matrix always involves error minimization, and possibly may involve shifting residual error towards colors less frequently encountered in real-world scenes. Such error can be moved to different parts of the space of possible sensor RGB values but it cannot b

47、e completely eliminated. It is therefore possible for sensor RGB at the periphery of sensor RGB space to be transformed into invalid colors (e.g. with negative R, G and/or B components, given the negative off-axis matrix elements) in the ARRI Wide Gamut color space. Gamut mapping or other methods fo

48、r the handling of such invalid colors are outside the scope of this document. SMPTE RDD 31:2014 Page 11 of 17 pages 5.4.1 Conversion Matrix Contents when no ALEXA Studio ND Type 1 Filter was used When the Filter Type component of the ND Filter field of the Lens Data Information subheader indicates n

49、o ND filter was in place at time of acquisition, the following table contains the matrix coefficients that are used for several common CCTs. CCT M1,1 M1,2 M1,3 M2,1 M2,2 M2,3 M3,1 M3,2 M3,3 2000 1.210510 -0.262282 0.051773 -0.121371 1.051117 0.070254 0.001944 -0.300355 1.298410 2100 1.202154 -0.252435 0.050281 -0.110522 1.039042 0.071480 0.007013 -0.298648 1.291635 2200 1.195132 -0.243392 0.048261 -0.101274 1.030202 0.071073 0.010983 -0.295148 1.284164 2400 1.184217 -