Multiply Distorted Image DatabaseMDID

MDID introduction

MDID is a database for evaluating the results of image quality assessment metrics on multiply distorted images. It consists of 20 reference images and 1600 distorted images.

The reference images are selected from several popular databases [22, 30-34], and are cropped into the size of 512 X 384 pixels without scaling or rotation.

The distorted images are obtained from degrading reference images with random types and levels of distortions. In this way, each distorted image contains multiple types of distortions simultaneously. Five distortions, namely Gaussian noise (GN), Gaussian blur (GB), contrast change (CC), JPEG and JPEG2000 are introduced:

   Gaussian Noise:  The MATLAB function&imnoise*is used to add Gaussian noise to reference images.
   Gaussian Blur:   The MATLAB functions&fspecial*and&imfilter*are used to blur reference images.
   Contrast Change: A segmented linear function is used to cause larger contrast in a certain range and smaller contrast in the other.
   JPEG:            The MATLAB function&imwrite*is used to compress reference images.
   JPEG2000:        Kakadu tools [36] is used to compress reference images.
According to the characteristics of image acquisition, transmission and display, we designed the order to introduce distortions as follows: GB or CC first, JPEG or JPEG2000 second and GN last. The process of obtaining distorted images of a reference image can be epitomized as follows:
   (1).Generate a random 1℅5 integer matrix M, its elements range from 0 to 4 and denote the distortion levels of corresponding five distortions.
   (2).Introduce distortions to the reference image with parameter M in the order mentioned above.
All of these Ms are recorded in the files: &1.txt*, &2.txt*, &3.txt* and so on. To specify, &1.txt* contains the parameters Ms utilized to obtain the 80 distorted images corresponding to 1st reference image.

To obtain the quality of each distorted image, pair comparison sorting method is utilized to perform subjective experiments. This scheme uses the quality comparison information from subjects to sort images. Finally, mean opinion scores and standard deviations are calculated and then presented respectively in the files &mos.txt* and &mos_std.txt*. Values in&mos.txt*denote the quality scores for 1600 distorted images (higher values for better-quality images).

Results of metrics:

Several well-known full-reference image quality assessment metrics are tested on MDID, including PSNR, SSIM [9], VIF [11], IW-SSIM [12], FSIMc [13], GMSD [14]. Results are as follows:

            SROCC    KROCC    PLCC    RMSE
   PSNR    0.5998    0.4303   0.6301  1.7109
   SSIM    0.8332    0.6450   0.8462  1.1742
    VIF    0.9306    0.7714   0.9367  0.7717
  IW-SSIM  0.8911    0.7092   0.8983  0.9682
   FSIMc   0.8904    0.7122   0.8998  0.9612
   GMSD    0.8613    0.6790   0.8776  1.0565 
[9] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13  (2004) 600每612.
[11] H. R. Sheikh, A. C. Bovik, Image information and visual quality, IEEE Trans. Image Process.15(2006) 430每444.
[12] Z. Wang, Q. Li, Information content weighting for perceptual image quality assessment, IEEE Trans. Image Process. 20(2011) 1185每1198.
[13] L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: a feature similarity index for image quality assessment, IEEE Trans. Image Process. 20(2011) 2378每2386.
[14] W. F. Xue, L. Zhang, X. Q. Mou, A. C. Bovik, Gradient magnitude similarity deviation: a highly efficient perceptual image quality index, IEEE Trans. Image Process. 23 (2014) 684每695.
[22] E. C. Larson and D. M. Chandler, Most apparent distortion: Full-reference image quality assessment and the role of strategy, J. Electron. Imaging 19 (2010).
[30] ImageNet: [Online]. Available:
[31] SIPI: The USC-SIPI Image Database, [Online]. Available:
[32] A. Gallagher, T. Chen, Understanding Groups of Images of People, IEEE Conference on CVPR, 2009, pp. 256-263.
[33] J. Philbin, O. Chum, M. Isard, J. Sivicand, and A. Zisserman, Object retrieval with large vocabularies and fast spatial matching, in: Proceedings of the IEEE Conference on CVPR, 2007, 
     pp. 1-8.
[34] R. C. Gonzalez, R. E. Woods, Didital Image Processing, Third edition, Beijing: Publishing House of Electronics Industry, 2011.
[36] D. S. Taubman and M. W. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice. Norwell, MA: Kluwer, 2001.



All rights of the MDID Database are reserved. The database is only available for academic research and noncommercial purposes. Any commercial uses of this database are strictly prohibited. Please cite the following paper if you use this database in your research:

   W. Sun, F. Zhou, Q. M. Liao. MDID: a multiply distorted image database for image quality assessment, Pattern Recognit. 61C (2017) pp. 153-168.

Contact information:

Fei Zhou, Postdoctoral Fellow

Visual Information Processing Lab. (VIP Lab.)

Department of Electronic Engineering/Graduate School at Shenzhen Tsinghua University

Tsinghua campus, the university town of Xili, Nanshan District, Shenzhen, China


Update date:

Recently, 117 new volunteers have participated in the process of subjective assessment, and corresponding scores of distorted images are updated with these new ratings as well as former ones.