Online Supplement

3D-MAD: A Full Reference Stereoscopic Image Quality Estimator based on Binocular Lightness and Contrast Perception

Yi Zhang and Damon M. Chandler

This webpage serves as the online supplement of the paper "3D-MAD: A Full Reference Stereoscopic Image Quality Estimator based on Binocular Lightness and Contrast Perception", IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3810-3825, November 2015.


Performance of 3D-MAD on Individual Distortion Types
We report the performance of 3D-MAD and various 2D/3D IQA algorithms on subsets of the LIVE, MCL, and IRCCyN/IVC 3D image databases corresponding to the individual distortion type. Test results of five distortion types in LIVE, six in MCL, and three in IRCCyN/IVC are presented in Table I in terms of CC, SROCC, and RMSE values.
 
Table I: SROCC values of 3D-MAD and 2D/3D IQA algorithms on different types of distortion on the LIVE, MCL, and IRCCyN/IVC 3D databases. Italicized entries denote 2D IQA algorithms. Results of the best-performing 3D IQA algorithm are bolded.
    SSIM MS-SSIM VIF MAD You Benoit Wang Ko Chen Lin Shao 3D-MAD
  CC
LIVE (Phase I) JP2K 0.859 0.916 0.936 0.951 0.921 0.914 0.870 0.910 0.912 0.836 0.921 0.955
JPEG 0.480 0.622 0.633 0.762 0.602 0.607 0.482 0.529 0.590 0.211 0.629 0.732
WN 0.893 0.929 0.930 0.935 0.908 0.907 0.875 0.885 0.926 0.875 0.929 0.952
GBLUR 0.911 0.936 0.941 0.961 0.899 0.924 0.934 0.953 0.935 0.945 0.942 0.959
  FF 0.689 0.803 0.858 0.842 0.730 0.782 0.764 0.828 0.747 0.699 0.843 0.881
LIVE (Phase II) JP2K 0.697 0.833 0.808 0.873 0.776 0.853 0.731 0.904 0.839 0.745 0.793 0.886
JPEG 0.645 0.814 0.782 0.858 0.776 0.821 0.662 0.707 0.831 0.593 0.759 0.889
WN 0.925 0.941 0.821 0.887 0.877 0.933 0.938 0.899 0.957 0.894 0.811 0.957
GBLUR 0.828 0.688 0.983 0.939 0.576 0.747 0.878 0.714 0.962 0.604 0.957 0.984
FF 0.856 0.841 0.940 0.920 0.788 0.867 0.879 0.824 0.907 0.695 0.936 0.932
MCL JP2K 0.897 0.951 0.936 0.970 0.902 0.930 0.911 0.916 0.923 0.947 0.859 0.965
JPEG 0.909 0.956 0.941 0.945 0.859 0.887 0.915 0.944 0.907 0.925 0.923 0.952
WN 0.874 0.895 0.943 0.945 0.829 0.857 0.896 0.880 0.873 0.835 0.888 0.929
GBLUR 0.926 0.934 0.965 0.955 0.884 0.927 0.929 0.922 0.890 0.901 0.872 0.970
SBLUR 0.901 0.935 0.938 0.975 0.889 0.942 0.907 0.916 0.915 0.942 0.862 0.966
  TERROR 0.853 0.857 0.778 0.862 0.795 0.795 0.825 0.830 0.854 0.769 0.827 0.860
IRCCyN/IVC JP2K 0.860 0.944 0.922 0.968 0.836 0.951 0.887 0.853 0.820 0.881 0.802 0.974
JPEG 0.850 0.951 0.941 0.926 0.848 0.930 0.851 0.860 0.775 0.755 0.882 0.935
BLUR 0.517 0.487 0.498 0.485 0.555 0.535 0.494 0.467 0.487 0.459 0.424 0.734
    SROCC
LIVE (Phase I) JP2K 0.857 0.898 0.902 0.925 0.884 0.887 0.870 0.891 0.896 0.839 0.883 0.916
JPEG 0.436 0.599 0.582 0.736 0.547 0.565 0.445 0.527 0.558 0.207 0.599 0.700
WN 0.938 0.942 0.932 0.950 0.929 0.939 0.939 0.933 0.948 0.928 0.930 0.950
GBLUR 0.879 0.928 0.931 0.954 0.910 0.911 0.918 0.941 0.926 0.935 0.910 0.942
FF 0.586 0.735 0.804 0.772 0.629 0.683 0.654 0.756 0.688 0.658 0.793 0.833
LIVE (Phase II) JP2K 0.703 0.817 0.826 0.869 0.834 0.842 0.727 0.902 0.833 0.719 0.788 0.895
JPEG 0.679 0.827 0.778 0.839 0.755 0.839 0.694 0.728 0.840 0.613 0.745 0.866
WN 0.920 0.947 0.820 0.885 0.878 0.926 0.934 0.900 0.955 0.907 0.807 0.952
GBLUR 0.836 0.801 0.950 0.924 0.275 0.766 0.882 0.836 0.910 0.711 0.939 0.942
  FF 0.835 0.830 0.934 0.918 0.740 0.862 0.865 0.811 0.889 0.701 0.935 0.922
MCL JP2K 0.900 0.928 0.933 0.930 0.919 0.927 0.904 0.897 0.902 0.919 0.862 0.930
JPEG 0.885 0.895 0.895 0.885 0.803 0.823 0.888 0.898 0.857 0.882 0.888 0.899
WN 0.912 0.928 0.944 0.952 0.855 0.886 0.915 0.919 0.889 0.859 0.890 0.950
GBLUR 0.912 0.936 0.952 0.953 0.881 0.916 0.913 0.920 0.893 0.895 0.873 0.970
SBLUR 0.905 0.938 0.954 0.972 0.891 0.944 0.918 0.930 0.920 0.954 0.859 0.972
TERROR 0.891 0.908 0.789 0.892 0.871 0.842 0.874 0.890 0.903 0.815 0.889 0.903
IRCCyN/IVC JP2K 0.875 0.941 0.924 0.965 0.820 0.947 0.910 0.857 0.823 0.885 0.802 0.972
JPEG 0.832 0.968 0.956 0.915 0.842 0.916 0.855 0.863 0.775 0.782 0.896 0.918
  BLUR 0.408 0.431 0.321 0.535 0.556 0.496 0.470 0.417 0.459 0.467 0.321 0.789
RMSE
LIVE (Phase I) JP2K 6.674 5.306 5.319 4.033 5.522 5.293 6.434 5.570 5.358 7.512 5.071 4.255
JPEG 6.584 5.530 5.539 4.468 6.111 5.534 6.077 5.935 5.646 8.253 5.594 4.650
WN 8.510 6.406 7.655 6.293 7.166 7.321 9.024 8.585 6.495 8.535 6.153 5.148
GBLUR 6.440 5.395 5.564 5.123 9.432 6.552 5.450 5.297 5.677 5.340 5.696 4.725
  FF 10.524 7.671 6.478 6.878 8.643 8.020 9.093 7.450 8.652 9.825 6.922 6.138
LIVE (Phase II) JP2K 8.434 7.133 6.016 6.362 6.373 6.203 8.305 7.195 6.452 8.611 7.009 4.933
JPEG 5.644 4.317 4.809 3.793 4.739 4.324 5.541 5.234 4.086 6.233 4.781 3.492
WN 4.150 4.291 8.905 6.153 6.769 4.578 3.746 4.936 3.156 5.727 7.355 4.214
GBLUR 8.358 10.857 5.312 7.187 11.536 9.855 7.933 10.537 4.269 11.871 6.491 3.511
FF 6.180 6.779 4.604 5.278 7.518 6.429 5.893 7.410 5.208 8.611 4.874 4.736
MCL JP2K 1.335 1.038 1.589 0.631 1.498 1.330 1.172 1.168 1.256 0.840 1.661 1.032
JPEG 1.032 0.753 1.045 0.759 1.294 1.149 1.153 0.880 1.064 0.977 1.077 0.797
WN 1.322 1.155 0.911 0.919 1.402 1.386 1.153 1.225 1.277 1.472 1.147 0.982
GBLUR 1.059 1.069 0.759 1.047 1.413 1.133 1.029 1.122 1.312 1.322 1.422 0.733
SBLUR 1.391 1.108 1.036 0.777 1.548 1.193 1.335 1.300 1.263 1.237 1.535 0.786
  TERROR 1.139 1.133 2.180 1.389 1.378 1.441 1.246 1.287 1.189 1.673 1.345 1.170
IRCCyN/IVC JP2K 11.948 8.010 8.991 7.047 13.480 7.329 11.212 11.848 13.220 11.648 13.642 5.835
JPEG 13.913 10.169 10.710 10.763 13.275 10.249 14.388 13.476 16.176 16.350 12.873 10.359
  BLUR 18.247 18.785 18.305 18.850 17.089 17.902 18.962 18.287 18.202 18.752 18.985 15.148
 
Analysis of parameters γ and γ2
In Eqs. (4) and (5), the parameter γ is a factor that aims to emphasize higher contrast values in the 2D-MAD stage. In Eqs. (11) and (12), the parameter γ2 is the transducer nonlinearity for the power summation in binocular contrast combination. Here we analyze the performance of 3D-MAD with different selections of these two free parameters. The corresponding results are shown in Table II.
 
Table II: Performance of 3D-MAD with different γ and γ2 applied on the LIVE 3D database. Best performances are bolded, and our selections of these two parameters are highlighted.
  γ ↓ γ2  3.5 3 2.5 2 1.5 1 0.5 0.25
LIVE3D Phase I 1 SROCC 0.9459 0.9462 0.9460 0.9457 0.9457 0.9454 0.9452 0.9450
CC 0.9530 0.9530 0.9529 0.9529 0.9528 0.9526 0.9527 0.9526
RMSE 4.9691 4.9692 4.9712 4.9738 4.9799 4.9865 4.9855 4.9891
1.25 SROCC 0.9456 0.9458 0.9454 0.9454 0.9456 0.9451 0.9449 0.9448
CC 0.9526 0.9526 0.9526 0.9525 0.9524 0.9523 0.9523 0.9523
  RMSE 4.9897 4.9894 4.9908 4.9928 4.9982 5.0039 5.0016 5.0033
1.5 SROCC 0.9454 0.9455 0.9451 0.9448 0.9449 0.9449 0.9447 0.9445
CC 0.9522 0.9522 0.9522 0.9521 0.9520 0.9519 0.9520 0.9520
RMSE 5.0113 5.0105 5.0113 5.0127 5.0173 5.0222 5.0187 5.0187
1.75 SROCC 0.9449 0.9450 0.9447 0.9446 0.9446 0.9443 0.9442 0.9443
CC 0.9517 0.9518 0.9518 0.9517 0.9517 0.9516 0.9517 0.9517
  RMSE 5.0328 5.0314 5.0317 5.0325 5.0364 5.0405 5.0359 5.0344
2 SROCC 0.9446 0.9448 0.9445 0.9444 0.9444 0.9442 0.9440 0.9439
CC 0.9513 0.9514 0.9514 0.9514 0.9513 0.9512 0.9513 0.9514
RMSE 5.0535 5.0516 5.0514 5.0515 5.0547 5.0581 5.0525 5.0497
2.25 SROCC 0.9443 0.9445 0.9441 0.9441 0.9441 0.9440 0.9438 0.9436
CC 0.9508 0.9509 0.9509 0.9510 0.9509 0.9509 0.9510 0.9511
  RMSE 5.0785 5.0753 5.0735 5.0719 5.0731 5.0747 5.0683 5.0649
2.5 SROCC 0.9440 0.9442 0.9439 0.9438 0.9439 0.9437 0.9434 0.9433
CC 0.9505 0.9505 0.9506 0.9506 0.9506 0.9506 0.9507 0.9508
    RMSE 5.0969 5.0932 5.0909 5.0888 5.0894 5.0904 5.0832 5.0788
LIVE3D Phase II 1 SROCC 0.9103 0.9107 0.9117 0.9126 0.9144 0.9157 0.9159 0.9055
CC 0.9194 0.9201 0.9208 0.9215 0.9221 0.9229 0.9210 0.9122
RMSE 4.4387 4.4216 4.4019 4.3832 4.3665 4.3464 4.3984 4.6242
1.25 SROCC 0.9116 0.9123 0.9133 0.9143 0.9158 0.9183 0.9192 0.9111
CC 0.9210 0.9216 0.9223 0.9230 0.9237 0.9246 0.9234 0.9162
  RMSE 4.3980 4.3810 4.3616 4.3426 4.3246 4.3003 4.3318 4.5230
1.5 SROCC 0.9117 0.9125 0.9140 0.9150 0.9169 0.9195 0.9214 0.9143
CC 0.9219 0.9225 0.9233 0.9240 0.9247 0.9257 0.9252 0.9193
RMSE 4.3726 4.3556 4.3362 4.3166 4.2972 4.2689 4.2821 4.4422
1.75 SROCC 0.9125 0.9134 0.9142 0.9161 0.9180 0.9205 0.9228 0.9179
CC 0.9224 0.9231 0.9238 0.9245 0.9253 0.9264 0.9266 0.9217
  RMSE 4.3589 4.3418 4.3221 4.3019 4.2808 4.2488 4.2459 4.3787
2 SROCC 0.9134 0.9143 0.9154 0.9171 0.9188 0.9211 0.9243 0.9201
CC 0.9226 0.9233 0.9240 0.9248 0.9256 0.9269 0.9275 0.9235
RMSE 4.3538 4.3365 4.3165 4.2954 4.2728 4.2372 4.2201 4.3294
2.25 SROCC 0.9137 0.9149 0.9160 0.9171 0.9187 0.9211 0.9248 0.9216
CC 0.9226 0.9232 0.9240 0.9248 0.9257 0.9271 0.9281 0.9249
  RMSE 4.3550 4.3373 4.3169 4.2950 4.2708 4.2319 4.2024 4.2915
2.5 SROCC 0.9140 0.9146 0.9158 0.9172 0.9186 0.9211 0.9252 0.9224
CC 0.9224 0.9230 0.9238 0.9246 0.9256 0.9271 0.9285 0.9259
    RMSE 4.3605 4.3426 4.3217 4.2990 4.2732 4.2312 4.1907 4.2626
 
 
 
As shown in Table II, on the LIVE 3D Phase I database, the best performance is achieved when γ2 = 3.5 and γ = 1, while on the LIVE 3D Phase II database, γ2 = 0.5 and γ = 2.5. Note that when γ2 decreases from 3.5 to 0.5, the 3D-MAD performance on the LIVE 3D Phase I database does not change significantly, but on the LIVE 3D Phase II database, the performance can increase a lot. Thus, the selection of γ = 2 and γ2 = 0.5 seem to be a reasonable choice to obtain a performance balance between these two databases.