IEEE CAMERA PHONE IMAGE QUALITY TEST PLAN-2018 en IEEE Camera Phone Image Quality Test Plan (IEEE Computer Society).pdf

1、IEEE Camera Phone Image Quality Test Plan IEEE | 3 Park Avenue | New York, NY 10016-5997 | USA IEEE Conformity Assessment Program IEEE Camera Phone Image Quality Test Plan IEEE Camera Phone Image Quality Test Plan Trademarks and Disclaimers IEEE believes the information in this publication is accura

2、te as of its publication date; such information is subject to change without notice. IEEE is not responsible for any inadvertent errors. The ideas and proposals in this specification are the respective authors views and do not represent the views of the affiliated organization. The Institute of Elec

3、trical and Electronics Engineers, Inc. 3 Park Avenue, New York, NY 10016-5997, USA Copyright 2018 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Published May 2018. Printed in the United States of America. IEEE is a registered trademark in the U. S. Patent fitnes

4、s for a particular purpose; non-infringement; quality, accuracy, effectiveness, currency, or completeness of the Work or content within the Work. In addition, IEEE and the ICAP members disclaim any and all conditions relating to: results; and workmanlike effort. This ICAP document is supplied “AS IS

5、” and “WITH ALL FAULTS.” Although the ICAP members who have created this Work believe that the information and guidance given in this Work serve as an enhancement to users, all persons must rely upon their own skill and judgment when making use of it. IN NO EVENT SHALL IEEE-SA OR ICAP MEMBERS BE LIA

6、BLE FOR ANY ERRORS OR OMISSIONS OR DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO: PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER I

7、N CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS WORK, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE AND REGARDLESS OF WHETHER SUCH DAMAGE WAS FORESEEABLE. Further, information contained in this Work may be protected by intellect

8、ual property rights held by third parties or organizations, and the use of this information may require the user to negotiate with any such rights holders in order to legally acquire the rights to do so, and such rights holders may refuse to grant such rights. Attention is also called to the possibi

9、lity that implementation of any or all of this Work may require use of subject matter covered by patent rights. By publication of this Work, no position is taken by the IEEE with respect to the existence or validity of any patent rights in connection therewith. The IEEE is not responsible for identi

10、fying patent rights for which a license may be required, or for conducting inquiries into the legal validity or scope of patents claims. Users are expressly advised that determination of the validity of any patent rights, and the risk of infringement of such rights, is entirely their own responsibil

11、ity. No commitment to grant licenses under patent rights on a reasonable or non-discriminatory basis has been sought or received from any rights holder. This Work is published with the understanding that IEEE and the ICAP members are supplying information through this Work, not attempting to render

12、engineering or other professional services. If such services are required, the assistance of an appropriate professional should be sought. IEEE is not responsible for the statements and opinions advanced in this Work. IEEE Camera Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved

13、. ivAuthors Henry Koren, Chair Benjamin Tseng, Vice Chair Margaret Belska Elaine Jin Zhiyong Liu Robert Sumner Shiwei Wang Yajun Wang Dietmar Wller Mengran Zhai Reviewers Avi Kalderon Young Hwan (Steven) Kim Xiao Ning Liu Neelam Sharma Ravi Subramaniam IEEE Camera Phone Image Quality Test Plan Copyr

14、ight 2018 IEEE. All rights reserved. v Contents 1. Lab environmental conditions 1 1.1 Environment 1 2. Spatial Frequency Response (SFR) 2 2.1 Test chart . 2 2.2 Capture procedure. 2 2.3 Post processing 3 3. Lateral chromatic displacement . 4 3.1 Test chart . 4 3.2 Capture procedure. 4 3.3 Post proce

15、ssing 4 4. Chroma level . 5 4.1 Test chart . 5 4.2 Capture procedure. 5 4.3 Post processing 6 5. Color uniformity . 6 5.1 Test chart . 6 5.2 Capture procedure. 6 5.3 Post processing 7 6. Local geometric distortion 7 6.1 Test chart . 7 6.2 Capture procedure. 7 6.3 Post processing 8 7. Visual noise .

16、9 7.1 Test chart . 9 7.2 Capture procedure. 9 7.3 Post processing 9 8. Texture blur 10 8.1 Test chart . 10 8.2 Capture procedure. 10 8.3 Post processing 11 9. Naming conventions . 11 9.1 Folder naming 11 9.2 Image naming 11 10. Analysis . 12 11. Standard JSON Output Format . 12 12. Citations 13 IEEE

17、 Camera Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved. 1 IEEE Camera Phone Image Quality Test Plan 1. Lab environmental conditions 1.1 Environment The ambient room temperature during the acquisition of the test data shall be 23 C 2 C and the relative humidity shall be 50% 30

18、%. 1.1.1 Containment To prevent any color cast in the test lab it shall be a painted black or 18% Neutral gray. 1.1.2 Illumination Illumination criteria appears in the following table: Illuminant CCT Lux Notes D55 based on ISO 7589 2 5500 K 700 K 4800 K 6200 K 1000 100 900 1100 This can be a tunable

19、 LED or filtered halogen. Must include NIR for color uniformity test. TL84 Flourescent 4100 K 300 K 100 10 90 110 Must be fluorescent. Tungsten based on ISO 7589 2 3050 K incandescent 300 K 25 2.5 22.5 27.5 This can be tungsten or tunable LED. Must include NIR between 700800 nm for color uniformity

20、test. 1.1.3 Light placement Lights shall be placed such that no specular reflection ever appears on the chart. Specular reflection can cause underexposure or otherwise disrupt the accuracy of the test. 1.1.4 Light uniformity A calibrated illuminance meter shall be used to measure the uniformity. Uni

21、formity shall be 80% or better across the chart area for all tests except for the color uniformity test where it shall be at least 92%. IEEE Camera Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved. 2 The target uniformity is calculated by obtaining illuminance and spectral meas

22、urements across the target in at least a grid of 3 horizontal 3 vertical regions. The maximum, minimum, and average values are obtained from these measurements, and the uniformity is computed via: uniformity = 1 (max-min)/average. Lighting that is pulsed shall not be modulated at any frequency less

23、than 500 Hz. 1.1.5 Light warmup and level verification Lights shall be given at least the following amount of warm-up time to stabilize before any measurements or exposures are taken. Illuminant Min. warm-up time LED 10 seconds Incandescent A 1 minute Flourescent 4 minutes 1.1.6 Device preparation D

24、evice preparation is as follows: 1. Reset the device back to factory defaults 2. Perform device OS update to latest version 3. Record OS version (e.g., iOS 10.3.2) 4. Open Camera app 5. The camera shall not be switched out of its default mode, such as HDR 6. Disable live photos 7. Disable flash 8. M

25、ake sure self-timer mode is not enabled 9. If available, enable high-quality and uncropped capture mode 1.1.7 Viewing condition For all measurements, the image viewing condition shall be a 100 ppi computer monitor viewed at 100% magnification and 60 cm distance. 2. Spatial Frequency Response (SFR) 2

26、.1 Test chart A low contrast edge SFR (E-SFR) test chart that is compliant with ISO 12233 3 shall be used. The detailed requirement of the chart follows subclause 5.3.4 of IEEE Std 1858-2016 1. The height of the image plane at the distance of the test chart shall be between 600800 mm. 2.2 Capture pr

27、ocedure The capture procedure is as follows: a) Clean lens with microfiber cloth. IEEE Camera Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved. 3 b) Mount device 1) On a tripod or device holder system. 2) The mounted device shall be placed on a dolly or rail system that allows

28、the operator to vary the distance between the camera and the chart. 3) Orient the device the same as the test chart, i.e., horizontal chart would mean placing the device in landscape mode. 4) At close distance, align the center of the phones lens with the center of the test chart so that the chart a

29、ppears centered on the phones display. c) Framing 1) Slide the device back so the height of the chart covers the height of the sensor as shown in the middle of the display. The vertical framing arrows are used to adjust the magnification, and to center the target horizontally. 2) Some devices may no

30、t show the entire vertical height of the sensor within the display; an initial image shall be inspected to ensure the framing is proper. 3) If the display vertically crops the sensor, the proper distance to fill the FoV with the chart will need to be determined through a process of trial and error.

31、4) If highlights or shadows in the captured image are clipped, try to cause a compensation in the exposure by presenting white or black areas outside the region of interest. d) Shooting 1) Put an obstacle between the camera and the chart and remove it before capture to trigger the refocusing process

32、. 2) The display must not be touched during image capture except a slight touch on for the shutter button in case there is no alternative to pressing the button on the display. If touched accidentally, the camera program shall be closed and relaunched to reset the capture mode. 3) Lightly touch the

33、shutter button to trigger the exposure. Repeat to collect 10 images of the SFR test chart at each light level specified in 1.1.3. e) Record the shooting distance. 2.3 Post processing In order to avoid any autofocus misses or potential camera shake, the images shall be culled down to one for each lig

34、ht condition using the following procedure: a) Transfer the images to the computer where they will be analyzed. b) If both an HDR and non-HDR image is captured, use the non-HDR image and delete the other image. c) Rename the images according to Section 9. d) Perform SFR acutance analysis on the set

35、of 10 images. 1) The average acutance of two vertical and two horizontal regions that are within 10% the center shall be computed. 2) Identify the image that has the best acutancethis is reported as the objective metric (OM). 3) Delete the remaining images. 4) The QL is calculated from the OM. IEEE

36、Camera Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved. 4 3. Lateral chromatic displacement 3.1 Test chart A dot pattern test chart shall be used. The detailed requirement of the chart follows subclause 4.2.1 of IEEE Std 1858-2016 1. The size and the spacing of the dots shall

37、be adapted to the camera pixel count and FoV so that at least 300 dots with a minimum of 10 pixel diameters are visible. 3.2 Capture procedure The capture procedure is as follows: a) Clean lens with microfiber cloth. b) Mount device 1) On a tripod or device holder system. 2) The mounted device shall

38、 be placed on a dolly or rail system that allows the operator to adjust the distance. 3) Orient the device the same as the test chart, i.e., horizontal chart would mean placing the device in landscape mode. 4) Align the camera axis with the normal of test target and the center of the phone display w

39、ith the center of the test chart. c) Framing 1) Shooting distance shall be adjusted so that the height of the image plane at the distance of the test chart shall be between 600800 mm. 2) No element external to the test target shall appear in the frame. 3) Avoid spots, rows, and columns from being cr

40、opped as much as possible. 4) Some devices may not show the entire vertical height of the sensor within the display; an initial image shall be inspected to ensure the framing is proper. 5) If the display vertically crops the sensor, the proper distance to fill the FoV with the chart will need to be

41、determined through a process of trial and error. d) Shooting 1) The display must not be touched during image capture except a slight touch on for the shutter button in case there is no alternative to pressing the button on the display. If touched accidentally, the camera program shall be closed and

42、relaunched to reset the capture mode. 2) Lightly touch the shutter button to trigger the exposure. Repeat to collect three images of the dot pattern test chart at each light level specified in 1.1.3. e) Record the shooting distance. 3.3 Post processing Post processing shall be as follows: IEEE Camer

43、a Phone Image Quality Test Plan Copyright 2018 IEEE. All rights reserved. 5 a) Transfer the images to the computer where they will be analyzed. b) If both an HDR and non-HDR image is captured, use the non-HDR image and delete the other image. c) Rename the images according to Section 9. d) Perform L

44、ateral Chromatic Displacement (LCD) analysis on the set of three images. e) The LCD objective metric is expressed as arcminutes. f) The best LCD of the three images is reported as the objective metric (OM). g) The QL is calculated from the OM (the best LCD of the three images). 4. Chroma level 4.1 T

45、est chart X-Rite Digital ColorChecker SG target shall be used. The detailed requirement of the chart follows subclause 7.2.3 of IEEE Std 1858-2016 1. The Colorchecker SG chart shall be mounted against an 18% gray background with minimal to no mounting structures visible. The black border of the Colo

46、rcheckerSG chart shall be minimized to be approximately 2 cm or less around the active region. 4.2 Capture procedure The capture procedure is as follows: a) Clean lens with microfiber cloth. b) Mount device 1) On a tripod or device holder system. 2) The mounted device shall be placed on a tripod tha

47、t allows adjustment of the vertical field of view (FoV) such that the vertical height of the chart encompasses 1/3 of the vertical FoV of the image frame. 3) Orient the device the same as the test chart, i.e., horizontal chart would mean placing the device in landscape mode. 4) Align the center of t

48、he phones lens with the center of the test chart so that the chart appears centered on the image preview display. c) Framing 1) Move the camera to a distance so that the target occupies roughly 1/3 of the vertical FoV. The remaining FoV shall be encompassed by a spectrally neutral, 18% gray wall. 2)

49、 Some devices may not show the entire vertical height of the sensor within the preview image; an initial image shall be inspected to ensure the framing is proper. 3) If the display vertically crops the sensor, the proper distance to fill the required vertical FoV with the chart will need to be determined through a process of trial and error. d) Shooting 1) The display must not be touched during image capture (except a slight touch on for the shutter button in case there is no alternative to pressing the button on the display) be