The following tools were used to automatically compress a set of images on default settings. The test images are PNG test corpus by Cédric Louvrier and Guetzli photo dataset (converted to JPEG at highest quality) by Jyrki Alakuijala. All images are publicly available for download.
The results were compared with the originals by using a flip test on the calibrated Macbook Pro Retina screen under normal viewing conditions. If an image is visually indistinguishable from its original, it is scored 1 (one), otherwise 0 (zero). The final score is the average of the relative values for each format.
It is highly recommended to view the images full size at 100% zoom. Some images especially on mobile are scaled down by the browser, making it harder to spot the differences.
|PNG||274 KB||296 KB||–||–||–||304 KB||–|
|JPEG||16.9 MB||16.9 MB||–||–||–||–||–|
|PNG||616 KB||578 KB||–||–||415 KB||–||–|
|JPEG||3.68 MB||2.08 MB||4.21 MB||3.74 MB||3.58 MB||3.63 MB||2.68 MB|
|PNG||1.2 ± 0.7||1.2 ± 1.9||–||–||1.6 ± 2.0||–||–|
|JPEG||2.3 ± 0.8||3.7 ± 1.9||1.3 ± 0.0||2.5 ± 0.7||2.5 ± 1.4||2.7 ± 1.9||3.3 ± 1.3|
*Score shows how many images are visually indistinguishable from originals under normal viewing conditions. Hover over results to see the actual number of images.
**Butteraugli score is in multiples of just noticeable difference (JND) with a confidence level of 95%.
- The human-rated scores correlate with the scores estimated by Butteraugli, which is considered a state-of-the-art image quality metric.
- The Guetzli score matches the results of the official study, where 75% of human ratings are in favor of Guetzli, which also implies Guetzli performed worse than the baseline JPEG encoder in 25% of cases.
- Guetzli achieved best quality/size ratio in some cases but failed to provide consistent quality especially at lower quality settings.
- Due to global error averaging and forced 4:2:0 chroma subsampling, the local error is virtually unlimited in tools like ImageOptim and TinyPNG, regardless of quality setting.
- ImageOptim forces 4:2:0 chroma subsampling even at higher quality 92-95, which leads to noticeable image quality degradation at comparable file size.
- TinyPNG and similar tools use stock PNGQuant at more aggressive settings, which greatly increases the likehood of a high local error in images mixing flat colored and photographic content, e.g. screenshots.
- TinyPNG images could be losslessly compressed by 6.3% more using Optimage.
- Guetzli is extremely slow requiring approximately 1 minute of CPU and 100 megabytes (MB) of memory per 1 megapixel (MPix).
- JPEGmini adds a hidden metadata containing version information and image hash to all recompressed JPEG files.
- ImageOptim will not strip metadata if recompression is suboptimal and increases the file size.