S4RR: Reduced-Reference IQA Based on Distortion Families of Local Perceived Sharpness
Submitted to SPIC

Discussion and demonstration for the motivation of using the four local statistics

In Section 3.3.1, four types of the local statistics and six distance measures are employed to quantify both the sharpness value changes and sharpness map structure changes between the reference and distorted sharpness maps. Here, we provide detailed discussion and demonstration for the proposed local statistics:

1. The local maximum statistic () mainly captures the individual sharpness value change caused by distortions. This statistic is more sensitive to the Gaussian blur distortion which decreases the sharpness values across all image regions, because the local maximum of a sharpness map (especially the edge area) is the first to be degraded by blurring. As demonstrated in Table I, the abridged S4RR model using the “ of LSD” features can achieve better performance on predicting qualities of Gaussian blur images in the LIVE and CSIQ databases as compared with other local statistics of LSD. On the TID2008 and IVC databases, the advantage of  is not quite obvious, but the performance is still competitive. Note that on the IVC database, using the “ of LSD” feature can achieve better results than using the “LSD” feature when testing on blurred images, which also demonstrates that “ of LSD” is more sensitive to the blurring artifact than “LSD” does.

2. The local standard deviation () and local sharpness distance () statistics mainly capture the sharpness map structure change caused by distortions. Both statistics are more sensitive to such distortions as JPEG/JPEG2000 compression and various noise corruptions, because all these distortions will add images additional artifacts (e.g., spot, aliasing, ringing, and blocking artifacts) that ultimately change the image's sharpness map structure. For example, on the edge area of an image, the JPEG2000 compression will probably keep the edges but add ringing artifact to the surrounding flat/texture area, in which case the local standard deviation of the sharpness map corresponding to that region will be changed. As demonstrated in Table I, the abridged S4RR models using these two statistics (denoted by “ of LSD” and “ of LSD”, respectively) can achieve impressive performance on predicting qualities of the JPEG and JPEG2000 compressed images on all five databases considered. Note that the “ of LSD” feature is also quite sensitive to the noise-corrupted images (as demonstrated by the TID2008 testing results of AGN, ACN, SCN, MN, etc.).

3. The top-variation statistic () is motivated by the work in Ref. [1] that the total variation of an image region in the spatial domain can be effective local sharpness measurement especially when the contrast is taken into account. Therefore, we employ the top-variation statistic built upon the LSD map to mainly capture the contrast change distortions. As demonstrated in Table I, the abridged S4RR model using the “ of LSD” feature can achieve very impressive performance on predicting qualities of the contrast change images in the CSIQ and TID2008 databases as compared with other local statistics of LSD.

Despite the different emphasis of different local statistics in representing distortions, we do find that for most distortion types, these four local statistics weigh similarly in predicting image quality (i.e., the performance difference among these local statistics is relatively trivial). For example, the “ of LSD” feature can also do a competitively good job in predicting qualities of the Gaussian-blurred images, and the JPEG/JPEG2000 compressed images. This fact demonstrates that the top-variation statistic () is also an efficient feature to capture both the sharpness value change and sharpness map structure change, which is quite reasonable. As we have found and also demonstrated in Table 8 in our paper, all four local statistics are required for better QA performance when all five testing databases are considered.

 

Table I: SROCC values tested on different distortion types in the LIVE, CSIQ, TID2008, IVC, and Toyama databases. Seven abridged versions of S4RR using only parts of the feature maps in the distortion-family-specific QA stage are tested. For reference, the SROCC values of the full S4RR algorithm using all 46 regression features are also included (denoted by “All”). 

FISH

of FISH

LSD

of LSD

of LSD

of LSD

of LSD

ALL

LIVE

JP2K

0.962

0.936

0.938

0.922

0.969

0.969

0.966

0.969

JPEG

0.913

0.958

0.975

0.967

0.970

0.968

0.976

0.975

AGN

0.973

0.954

0.980

0.980

0.854

0.940

0.881

0.965

GBLUR

0.952

0.946

0.936

0.923

0.921

0.918

0.911

0.944

 

FF

0.927

0.874

0.949

0.954

0.933

0.937

0.932

0.943

CSIQ

AGN

0.938

0.927

0.946

0.927

0.932

0.934

0.941

0.941

JPEG

0.934

0.936

0.954

0.943

0.955

0.956

0.960

0.959

JP2K

0.936

0.921

0.933

0.919

0.972

0.974

0.956

0.970

FN

0.879

0.886

0.929

0.925

0.937

0.936

0.937

0.935

GBLUR

0.936

0.921

0.972

0.976

0.971

0.971

0.960

0.958

Contrast

0.572

0.242

0.909

0.908

0.891

0.921

0.783

0.913

TID2008

AGN

0.822

0.816

0.906

0.913

0.918

0.909

0.941

0.939

ACN

0.790

0.760

0.854

0.833

0.852

0.845

0.883

0.909

SCN

0.797

0.782

0.921

0.913

0.924

0.921

0.943

0.955

MN

0.690

0.687

0.791

0.782

0.827

0.820

0.860

0.926

HFN

0.871

0.879

0.936

0.935

0.925

0.926

0.921

0.943

IN

0.511

0.176

0.680

0.686

0.697

0.703

0.713

0.834

QN

0.565

0.671

0.839

0.812

0.823

0.823

0.878

0.908

GBLUR

0.924

0.857

0.851

0.846

0.855

0.860

0.894

0.952

ID

0.940

0.916

0.952

0.951

0.952

0.948

0.969

0.973

JPEG

0.867

0.934

0.928

0.908

0.921

0.918

0.941

0.949

JP2K

0.947

0.933

0.955

0.949

0.958

0.958

0.978

0.981

JPEG_TE

0.795

0.804

0.837

0.826

0.837

0.840

0.872

0.889

JP2K_TE

0.578

0.583

0.869

0.824

0.911

0.913

0.904

0.948

NEPN

0.839

0.836

0.830

0.832

0.822

0.832

0.800

0.867

LBW

0.768

0.801

0.764

0.671

0.644

0.629

0.719

0.785

MS

0.380

0.433

0.439

0.443

0.500

0.477

0.481

0.518

 

Contrast

0.896

0.871

0.930

0.913

0.933

0.934

0.924

0.946

IVC

JP2K

0.827

0.734

0.878

0.885

0.953

0.955

0.945

0.922

JPEG

0.853

0.849

0.925

0.873

0.929

0.924

0.895

0.924

LAR coding

0.819

0.743

0.920

0.932

0.921

0.922

0.903

0.897

BLUR

0.828

0.932

0.804

0.863

0.951

0.940

0.952

0.945

Toyama

JP2K

0.882

0.900

0.833

0.710

0.926

0.950

0.897

0.951

 

JPEG

0.848

0.869

0.910

0.786

0.918

0.926

0.898

0.913

 

[1] C. Vu, T. Phan, D. Chandler, S3: A spectral and spatial measure of local perceived sharpness in natural images, IEEE Transactions on Image Processing, 21, (2012) pp. 934-945.