Current Location: vsd

Quick Nav:
Image Coding and Analysis Lab
Data compression, image analysis, human visual system models, and natural scenes
007 Engineering South
School of Electrical and Computer Engineering
Oklahoma State University
Stillwater, OK 74078 USA
Phone: (405) 744-2192

Home

Members

Publications

Projects

Photos

Contact

VQA Project Information

The Video Quality Assessment (VQA) Project

        Video Quality Assessement Project using ST-MAD is our Full-Reference objective video quality assessment method which uses the Most Apparent Distortion (MAD) to evaluate the quality of a distorted video in comparison to the original one (considered the best quality video). We use the LIVE Video Database (http://live.ece.utexas.edu/research/quality/live_video.html) to judge our algorithm and compare to some state of the arts quality assessment methods in term of correlation to subjective results.

Typical Frames from Tested Movie Clips:

 
     
     
     
   
 
 
 

Algorithm Description:

 1. OVERVIEW
      This research presents an algorithm for video quality assess- ment, spatiotemporal MAD (ST-MAD), which extends our previous image-based algorithm (MAD [1]) to take into account visual perception of motion artifacts. ST-MAD employs spatiotemporal images (STS images [2]) created by taking time-based slices of the original and distorted videos. Motion artifacts manifest in the STS images as spatial artifacts, which allows one to quantify motion-based distortion by using classical image-quality assessment techniques. STMAD estimates motion-based distortion by applying MADís appearance-based model to compare the distorted videoís STS images to the original videoís STS images. This comparison is further adjusted by using optical- ow-derived weights designed to give greater precedence to fast-moving regions located toward the center of the video. Testing on the LIVE video database demonstrates that ST-MAD performs well in predicting video quality.

2. IMPLEMENTATION
2.1. Spatial MAD
     The original (spatial-only) MAD algorithm consists of two stages: (1) a detection-based stage, which computes the perceived degredation due to visual detection of distortions d_detect; and (2) an appearance-based stage, which computes the perceived degredation due to visual appearance changes d_appear. The detection-based stage of MAD computes d_detect by using a masking-weighted block-based MSE computed in the lightness domain. The appearance-based stage of MAD computes d_appear by computing the average difference between the block-based log-Gabor statistics of the original image to those of the distorted image. Those two values are adaptive combined into a single scalar distortion value d_spatial for the entire video.
     The following figures show two typical frames from the original and distorted version of video 2. The Detection and Appearance Based Maps are normalized to the range [0,1] to demonstrate the ability of capturing the distoted regions which reduce the quality of the video.
 
Frame from Original Video Frame from Distorted Video
Detection-based Map Appearance-based Map

  2.2. Temporal MAD
     To estimate the perception of motion-based distortion, STMAD performs three steps: (1) it applies MADís appearancebased model to spatiotemporal images created from the orignal and distorted videos; (2) it weights these values by using optical- ow-derived weights designed to give greater precedence to fast-moving regions located toward the center of the video; and then (3) it combines the values from Step 1 with the weights from Step 2 using a combination rule that varies according to the prevailing amount of motion.
     Similar to the Spatial MAD, the Temporal MAD also capture the distorted regions of the STS Images. The following figures show two types of STS images (column-based and row-based) from the original and distorted version of video 2. The Appearance Based Maps are normalized to the range [0,1] and our algorithm shows the good mapping from the distorted regions to the high intensity regions of the map.
 
Original Video Distorted Video Appearance-based Map
  Column STS Image  
  Row STS Image  

  2.3. Overall ST-MAD
     The previous sections described how to compute the spatial-based distortion d_spatial and the motion-based distortion d_motion. As a final step, these values are combined together to obtain the final output d of the ST-MAD algorithm; it is a scalar value that denotes the overall quality of the distorted video relative to the original video.

3. RESULTS
    To assess the performance of our ST-MAD algorithm, we use the LIVE video database [8] which contains 10 high quality videos with a variety of content as reference (originals), and 150 distorted videos (15 distorted video for each original video). The distortion types are MPEG-2 compression (MPEG-2), H.264 compression (H.264), simulated transmission of H.264 compressed bitstreams through error-prone IP networks (IP), and through error-prone wireless networks (wireless). Our algorithm has the best correlation coefficients in every types of videos (excepts Wireless) in comparison to the other ones.

   3.1. Pearson Correlation:
     
Wireless IP H.264 MPEG-2 All Data
PSNR 0.4675 0.4108 0.4385 0.3856 0.4035
VSNR 0.6992 0.7341 0.6216 0.5980 0.6896
MS-SSIM 0.7170 0.7219 0.6919 0.6604 0.7441
VQM 0.7325 0.6480 0.6459 0.7860 0.7236
MOVIE 0.8386 0.7622 0.7902 0.7596 0.8116
S-MAD 0.7887 0.7616 0.7014 0.6536 0.7366
T-MAD 0.7798 0.7554 0.9069 0.8290 0.8184
ST-MAD 0.8123 0.7900 0.9097 0.8422 0.8299


   3.2. Spearman Rank Order Correlation:
     
Wireless IP H.264 MPEG-2 All Data
PSNR 0.4334 0.3206 0.4296 0.3588 0.3684
VSNR 0.7019 0.6894 0.6460 0.5919 0.6755
MS-SSIM 0.7285 0.6534 0.7051 0.6617 0.7361
VQM 0.7214 0.6383 0.6520 0.7810 0.7026
MOVIE 0.8109 0.7157 0.7664 0.7733 0.7890
S-MAD 0.7754 0.7628 0.6638 0.6793 0.7211
T-MAD 0.7812 0.7459 0.9071 0.8292 0.8149
ST-MAD 0.8060 0.7686 0.9043 0.8478 0.8242
 
 
 

Download:

  Reprint Paper: ICIP_STMAD.pdf (18th IEEE Intl. Conf. on on Image Processing (ICIP), Sep 2011.)

  Matlab Code: STMAD_2011_Matlab (This paper was accepted and presented at ICIP 2011.)

 
 
 
 
This project was supported by the National Science Foundation, "Content-Based Strategies of Image and Video Quality Assessment".

PI: Damon Chandler, Oklahoma State University; Award #0917014.