Image and Video Quality Resources

On these pages I am compiling a list of online resources primarily aimed at researchers in the image/video quality area, which may also be useful to the larger image processing and computer vision communities. I have included publicly available image and video databases, both with (annotated) and without subjective ratings, eye tracking databases, and links to standards. Do let me know if you are aware of other interesting online resources that I could include here.

Please also refer to the following papers for in-depth analysis of these databases:

Tables summarizing the test material and subjective experiments for most of the image and video quality databases are available for download in comma-separated value (.csv) file format.

Annotated Image Quality Databases

A57 Database (grayscale) (2007)
Small database with various distortion types (compression, blur, noise).

IRCCyN/IVC Watermarking Databases (grayscale) (2007-2011)
Six separate databases deadling with watermarking and encryption distortions. They comprise Broken Arrow (BA), Fourier Subband (FSB), Enrico, Meerwald (MW), Selective Encryption (SE), and H.264 watermarking databases.

Wireless Imaging Quality (WIQ) Database (grayscale) (2009)
JPEG compressed images and distortions introduced by a simulated wireless link.

Categorical Image Quality (CSIQ) Database (2010)
Distortions include JPEG and JPEG2000 compression, global contrast decrements, additive pink Gaussian noise, and Gaussian blur.

Camera Image Database (CID2013) (2013) new
480 images captured by 79 imaging devices (mobile phones, DSC, DSLR) in six image sets, evaluated by 188 observers.

Colourlab Image Database: Image Quality (CID:IQ) (2014) new
Distortions include JPEG and JPEG2000 compression, Poisson noise, blurring, and two gamut mapping algorithms.

Digitally Retouched Image Quality (DRIQ) Database (2012)
Images enhanced via manual digital retouching.

ESPL-LIVE HDR Subjective Image Quality Database (2016) new
1800 images created by tone-mapping and exposure fusion algorithms, with and without post-processing. Subjective ratings were obtained via crowdsourcing from over 5000 observers.

IRCCyN/IVC Image Quality Database (2005)
Test conditions include JPEG, JPEG2000 and LAR compression as well as blur.

IRCCyN/IVC Art Image Quality Database (2009)
Half of this dataset are "art" images. Test conditions include JPEG, JPEG2000 and LAR compression.

JPEG XR Database (2009)
Comparing the quality of JPEG XR compression to JPEG and JPEG2000.

LIVE Image Quality Assessment Database (2006)
Release 2 of this popular database. Distortions include JPEG, JPEG2000, white noise, Gaussian blur, and simulated Rayleigh fading channel (JPEG2000 bitstream).

LIVE Multiply Distorted Image Quality Database (2012)
Two multiple distortion scenarios: 1) images are first blurred and then compressed by a JPEG encoder. 2) images are first blurred and then corrupted by white Gaussian noise.

LIVE In the Wild Image Quality Challenge Database (2015) new
1162 images captured using a variety of mobile devices with authentic distortions. Subjective ratings were obtained via crowdsourcing from over 8000 observers.

MCL-JCI Database (2016) new
50 source images compressed using JPEG at quality factors (QF) ranging from 1 to 100. Stair quality functions (SQF) and just-noticeable difference (JND) data obtained from more than 150 volunteers are available.

MICT (a.k.a. Toyama) Image Quality Evaluation Database (2008)
This database focuses on image compression using JPEG and JPEG2000.
IRCCyN/IVC collected ratings for the same images using an LCD screen in a separate experiment.

Multiply Distorted Image Database (MDID) (2016) new
1600 images with multiple random types and levels of distortions (noise, blur, contrast change (CC), JPEG and JPEG2000 compression).

Tampere Image Database (TID) (2008, 2013)
TID2013 is an extension of TID2008. Currently the largest image quality database available in the public domain, both in terms of test images and number of subjects. It contains a wide variety of distortions, including various types of noise, blur, JPEG and JPEG2000 compression, transmission errors, local image distortions, luminance and contrast changes.

VCL@FER Image Quality Database (2012)
Distortions include white Gaussian noise (AWGN), Gaussian blur, JPEG2000 and JPEG compression.

Annotated Video Quality Databases

Camera Video Database (CVD2014) (2014) new
234 videos captured by 78 different cameras (mobile phones, compact camera, video camera, SLR), evaluated by 210 observers.

EPFL-PoliMI Video Quality Assessment Database (2009)
Test conditions focus on H.264 compressed videos corrupted by simulated packet loss due to transmission over an error-prone network.

ETFOS CIF & VGA Video Quality Databases (ECVQ, EVVQ) (2012) new
90 test videos with distortions from H.264/AVC and MPEG-4 compression; DMOS based on 40 subjects.

IRCCyN/IVC Databases (2008-2013)
Over 10 different databases, mostly with H.264 and SVC. They include:
IVC 1080i, an HD database with both ACR and SAMVIQ MOS;
IVC Eyetracker SD, which includes eye tracking data;
IVC SD RoI, which includes the H.264 bitstreams;
IVC Influence Content, which comprises 60 different source videos;
IVC JEG264HMIX1, an HD database with PCAP files;
and several others.

IVP Subjective Video Quality Database (2011)
Progressive HD video compressed with MPEG-2, Dirac wavelet, and H.264 codecs as well as H.264 streams affected by simulated packet loss. DMOS are provided separately for expert and non-expert observers.

LIVE Video Quality Database (2010)
Test conditions include MPEG-2 compression, H.264 compression, simulated transmission of H.264 compressed bitstreams through error-prone IP wired and wireless networks.

LIVE Mobile Video Quality Database (2012)
Test conditions focus on H.264 compressed videos with distortions such as packet loss, frame freeze, and rate adaptation.

LIVE Mobile Stall Video Database (2016) new
176 videos generated from 24 reference videos with 26 unique stalling events, rated by 54 subjects.

MCL-V Database (2015) new
12 uncompressed HD source video clips with two distortion types (compression and image scaling) and four distortion levels.

MCL-JCV Database (2016) new
24 source clips compressed using H.264/AVC at quality factors (QF) ranging from 1 to 51. Just-noticeable difference (JND) data obtained from about 50 volunteers are available.

MMSP Scalable Video Database (SVD) (2010)
The test conditions include two scalable video codecs using multiple spatial and temporal resolutions. The database only includes the sources together with the software and process for creating the test conditions, rather than including the test videos as such. Subjects performed paired comparisons in side-by-side viewing sessions.

Poly@NYU Video Quality Databases (2008-2012)
4 separate but related tests using videos with different frame rates and quantization parameters.

Poly@NYU Packet Loss Database (2009)
Small test on the impact of packet loss in H.264 videos. Test clips are only 2~seconds long.

Roma Tre ReTRiEVED Video Quality Database (2014)
Test video sequences comprise practical transmission scenarios with different packet loss rates, jitter, delay, and throughput, using a network simulator.

TUM 1080p25 Dataset (2010)
1080p25 video sequences of the SVT test set, encoded with H.264/AVC and Dirac.

TUM 1080p50 Dataset (2012)
1080p50 video sequences of the SVT test set encoded with H.264/AVC. Subjective tests were conducted in 4 different scenarios (different displays, with and without sound).

VQEG FR-TV Phase I Database (2000)
The oldest public quality database (interestingly it came out several years before the first image quality database). Consequently, test conditions focus on MPEG-2 compression and transmission and even include some analog distortions.

VQEG HDTV Database (2010)
Test conditions include MPEG-2 and H.264 compression as well as different types of network impairments. 5 of the 6 sets in the HDTV test are being released via the Consumer Digital Video Library (CDVL); the sixth set is not public.

The videos in these databases are around 10 seconds long, with the exception of the NYU-PL database. Four databases in this list include the encoded bitstreams, the rest only provide the decoded video frames.

Annotated 3D Image/Video Quality Databases

IRCCyN/IVC 3D Image Quality Database (2008)
The first public-domain database on 3D image quality. Test conditions include JPEG and JPEG2000 compression as well as blur.

IRCCyN/IVC DIBR Image Database (2011)
Comparison of 7 depth image-based rendering algorithms on frames extracted from 3 video sequences (ACR and pair-comparison data is available).

IRCCyN/IVC NAMA3DS1-COSPAD1 3D Video Quality Database (2012)
Test conditions include H.264 and JPEG2000 compression as well as common image processing operations like downsampling or sharpening.

IVP Anaglyph Image Database (2013)
25 stereo image pairs and 100 anaglyph images.

IVY Lab Stereoscopic 3D Image Database (2013) new
120 stereo image pairs evaluated by 18 subjects.

LIVE 3D Image Quality Database (2012)
Phase I contains 20 pristine stereopairs with symmetrical distortions, while phase II contains 8 pristine stereopairs with both symmetrical and asymmetrical distortions. Distortions are similar to the LIVE database above.

MCL 3D Database (2014) new
9 image-plus-depth sources undergoing depth image-based rendering (DIBR) with different distortions.

MMSP 3D Image Quality Assessment Database (2010)
The test conditions represent different inter-camera distances. All images are JPEG-compressed.

MMSP 3D Video Quality Assessment Database (2010)
The first public-domain database on 3D video quality. The test conditions represent different camera distances. All videos are slightly cropped and compressed.

SCCH 3D Visual Discomfort Database (2015)
Video database with scenes of different content and characteristics, including subjective ratings of visual comfort, depth quantity and image quality.

VCL@FER 3D Video Quality Database (2016) new
8 scenes from the IRCCyN/IVC NAMA3DS1-COSPAD1 3D Video Quality Database with 22 degradation types.

Waterloo-IVC 3D Image Quality Database (2015) new
10 stereo scenes with three types of symmetrical and asymmetrical distortions.

Eye Tracking Databases

ADSC PASCAL animal classes Eye Tracking (PET) Database (2015) new
Eye movement data from 40 participants for six animal-centric classes from the PASCAL Visual Object Classes (VOC) Challenge 2012, for free-viewing and visual search tasks.

Actions in the Eye (2012)
Eye movement data from 16 participants for 1857 videos, with one group performing action recognition, the other free viewing.

Abnormal Surveillance Crowd Moving Noise (ASCMN) (2012)
Eye movement data from 13 participants for 24 surveillance-type videos.

DIEM Project (2011)
Eye movement data from up to 42 participants for 85 videos.

Fixations in Faces (FiFa) (2010)
Eye movement data from over 8 participants for 303 images containing faces.

GazeCom (2010)
Eye movement data from 54 subjects for 18 outdoor scenes (HD), 2 Hollywood trailers (SD), and static images taken from the outdoor scenes.

GIPSA Dynamic Natural Scenes (2014) new
Eye movement data from 72 subjects for 60 videos in 4 auditory conditions.

GIPSA Meeting Scenes (2015) new
Eye movement data from 40 subjects for 15 videos of 4 persons having a meeting (with and without sound).

IRCCyN/IVC Datasets:
Eyetracker 2006 05 (Image 1) Dataset (2006) contains eye tracking data of up to 40 observers for 27 images. The same database is also available here.
Berkeley (Image 2) Dataset (2006) contains eye tracking data from 25 observers for 84 images from the Berkeley Segmentation Dataset.
LIVE (Image 3) Dataset (2013) contains eye tracking data from 21 observers for 29 images from the LIVE Image Quality Database listed above.
3DGaze Dataset (2013) contains 18 stereoscopic images and the associated fixation density map, disparity map, depth map, and the raw eye tracking data.
SD 2008 11 (Video 1) Dataset (2009) contains eye tracking data from 37 observers for 51 H.264-compressed SD videos.
SD 2009 12 (Video 2) Dataset (2010) is the one listed above; it contains subjective ratings as well as eye tracking data from 30 observers for 100 SD videos in a quality rating task.

KTH Dataset (2011)
Fixation distance maps and eye tracking data of 31 observers for 100 images from the McGill calibrated colour image database.

LIVE DOVES (A Database Of Visual Eye movements) (2009)
Fixation points of 29 observers for 101 images from van Hateren's Natural Image Database, obtained using a dual-Purkinje eye tracker.

McGill ImgSal (2013)
Eye movement data of 21 observers for 235 images. Also includes labels of salient regions from 19 observers.

MIT CSAIL Saliency Dataset (2009)
Eye tracking data and fixation maps of 15 viewers for 1003 images.

MIT CVCL Search Model Dataset (2009)
Eye tracking data of 14 viewers for 912 images.

MIT Low-Resolution Saliency Dataset (2011)
Eye tracking data of 8 viewers for 1544 images at different resolutions.

NUS Eye Fixation (NUSEF) Dataset (2010)
Eye fixations of 75 viewers for 758 images.

NUS EyeCrowd Dataset (2014)
500 images of crowded scenes with eye tracking data from 16 viewers and annotation data on faces.

NUS FiWI (Fixations in Webpage Images) Dataset (2014)
149 webpages and eye tracking data from 11 subjects.

Object and Semantic Images and Eye-tracking (OSIE) Dataset (2014)
700 images with eye-tracking data of 15 viewers and annotation data of 5551 segmented objects with fine contours and 12 semantic attributes.

Pascal Objects Eye Tracking (POET) Dataset (2014)
Eye fixations of 5 viewers for 6270 images from the PASCAL Visual Object Classes (VOC) Challenge 2012.

SALICON (Saliency in Context) Dataset (2015)
10,000 training images and 5,000 validation images with saliency ground-truth.

Semiautomatic Visual-Attention Modeling (SAVAM) Database (2014)
Eye movement data of 50 viewers for 41 HD and UHDTV sequences.

SBU Gaze-Detection-Description Dataset (2013)
Eye fixations of 3 participants for 1000 images from the Pascal Visual Object Classes (VOC) dataset and of 8 participants for 104 images from the SUN09 dataset. Also includes descriptions, pre-trained object detectors, and associated bounding boxes.

SFU Video Dataset (2012)
Eye tracking data of 15 observers for 12 video sequences.

Stuttgart Video Eyetracking Benchmark Dataset (2014)
Eye tracking data of 25 observers for 11 video sequences.

Toronto Dataset (2006)
Eye movement data of up to 11 observers for 120 color images.

TU Delft Datasets
Image Quality Dataset 1 (2009) contains saliency maps for 29 reference images obtained using 20 observers.
Image Quality Dataset 2 (2011) contains eye tracking data from 75 observers for 160 JPEG-compressed images.
Image Quality Interactions (2011) contains eye tracking data from 14 observers for 54 images with varying quality for two different tasks (free viewing and quality scoring).
Video Quality Task Effect (2012) contains eye tracking data from 14 observers for 54 videos with varying quality for two different tasks (free viewing and quality scoring).

UCSB VIU Dataset (2014)
Eye movements and perceptual judgments from 161 observers viewing 800 natural images while completing 4 different tasks (free viewing, saliency judgment, and cued object search).

USC CRCNS Video Dataset (2004-2006)
Eye tracking data of 8 observers for 25 minutes of video (in original sequence as well as cut into short clips and scrambled).

USC Visual Attention Guided Bit Allocation (VAGBA) Dataset (2011)
Fixation points of 14 observers for 50 HD video clips in raw uncompressed format.

Visual stimulus, Intent, Person (VIP) Dataset (2013)
Eye tracking data from 75 participants for 150 images from the NUSEF dataset, with one group performing anomaly detection, the other free viewing. Includes demographic and personality data for trait-specific models.

Visual Attention for Image Quality (VAIQ) Dataset (2009)
Gaze patterns of 15 observers and saliency maps for 42 reference images from the IRCCyN/IVC, MICT, and LIVE databases listed above.

Also of interest: MIT Saliency Benchmark (2012).

Other Databases

Qualinet Databases
List of a large number of available multimedia databases collected as part of the QUALINET project.

Consumer Digital Video Library (CDVL)
A growing collection of high-quality uncompressed videos, some of which are annotated with subjective ratings.

ITU-T Test Signals
A collection of voice, audio, image, and video data referenced in various ITU-T recommendations.

ARRI Raw Color Sequence Dataset
High-resolution color image sequences captured with a professional digital cinema camera in raw format.

RMIT 3D Video Library
A collection of about 50 uncompressed stereoscopic 3D high definition video sequences.

Uncompressed Colour Image Database (UCID)
A dataset of images that were captured and are available in uncompressed form. The current version of the database (v2) has over 1300 images together with ground truth for image retrieval benchmarking.

The Official HD Video Clip List
Collection of links to various HD clips (both compressed and uncompressed).

Consolidated List of Test Video Clip Resources
Collection of links and references to test videos (both compressed and uncompressed).

Computer Vision Test Images
Collection of links to various databases for computer vision applications.

TU Delft Perceived Ringing Datasets
Maps of perceived ringing for 16 JPEG-compressed images from 12 observers, plus ringing annoyance ratings for 55 JPEG-compressed images from 20 observers.

UFRJ Blurred Image Database
585 images representing realistic blur (out-of-focus, motion, etc.) with ratings from users.

Local Masking in Natural Images Database
Psychophysical contrast threshold measurements for log-Gabor noise targets placed within local regions of 30 natural image masks.

Colourlab Image Database: Perceptual Projection Sharpness (CID:PPS) new
7 original images distorted by 6 levels of blur.

Spectral Image Database for Quality (SIDQ) new
9 original 160-band hyperspectral images of scenes representing pseudo-flat surfaces of different materials with 5 spectral distortions.

Standards

Video Quality Experts Group (VQEG)
A group of experts from various backgrounds and affiliations working in the field of video quality assessment. VQEG combines the expertise and resources found in several ITU Study Groups to work towards a common goal.

Video Services Forum (VSF), QoE Activity Group
This group is defining metrics for characterizing content quality of video streams and describing the relationship between transport losses and video payload as a function of content. This is joint work with the the ATIS IPTV interoperability Forum (IIF).