Changes in TIFF v4.0.10 — LibTIFF 4.7.2 documentation

# Changes in TIFF v4.0.10

References
Current Version | v4.0.10 (tag v4.0.10)
Previous Version | v4.0.9
Master Download Site | https://download.osgeo.org/libtiff/
Master HTTP Site #1 | http://www.simplesystems.org/libtiff/
Master HTTP Site #2 | http://libtiff.maptools.org/

This document describes the changes made to the software between the previous and current versions (see above). If you don't find something listed here, then it was not done in this timeframe, or it was not considered important enough to be mentioned. The following information is located here:

## Major changes

- The libtiff source repository is changed from CVS to Git and the master libtiff source repository is now at Gitlab. This is the first release to be made from the new Git repository.

## Software configuration changes

- Minimum CMake version is now v2.8.11 for the CMake-based build.

- Libwebp will be automatically detected and used by configure/cmake if present.

- Libzstd will be automatically detected and used by configure/cmake if present.

## Library changes

- Added ZSTD compression codec. Zstandard or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library.

  We require libzstd >= 1.0.0 so as to be able to use streaming compression and decompression methods.

  The default compression level we have selected is 9 (range goes from 1 to 22), which experimentally offers equivalent or better compression ratio than the default deflate/ZIP level of 6, and much faster compression.

  For example on a 6600×4400 16bit image, `tiffcp -c zip` runs in 10.7 seconds, while `tiffcp -c zstd` runs in 5.3 seconds. Decompression time for zip is 840 ms, and for zstd 650 ms. File size is 42735936 for zip, and 42586822 for zstd. Similar findings on other images.

  On a 25894×16701 16bit image:

  Compressor Relative Performance
  Compressor | Compression time | Decompression time | File size
  --- | --- | --- | ---
  ZSTD | 35 s | 3.2 s | 399,700,498
  ZIP/Deflate | 1m 20 s | 4.9 s | 419,622,336

  Please note that `COMPRESSION_ZSTD` is self-assigned the id 50000 by the libtiff project and is not officially registered with Adobe since Adobe's registration function is defunct.

- Added WebP compression codec.

  WebP is a high performance compressor intended for photos as commonly used on the Web. The WebP encoder is not designed for huge images, but serves very well for compressing strips and tiles in TIFF as long as the strips or tiles do not exceed the capability of the encoder.

  As a test of compression performance metrics, GraphicsMagick was used on an extremely high quality 8-bit TIFF image from a Hasselblad H4D-200MS camera with pixel dimensions of 16352×12264. The image was re-encoded with 1024×1024 tiles and various compression algorithms, using default settings for each algorithm. Based on this test, the compression and decompression performance (in iterations per second), the resulting file size, and the calculated total PSNR are provided here. It can be seen that WebP provided excellent encode and decode performance, and the compressed file size was very small:

  Compressor Relative Performance
  Compressor | Compression | Decompression | File size | PSNR
  --- | --- | --- | --- | ---
  None | 0.536 iter/s | 1.506 iter/s | 576.03MiB | ∞
  LZW | 0.105 iter/s | 0.266 iter/s | 270.68MiB | ∞
  ZStd | 0.020 iter/s | 0.518 iter/s | 238.42MiB | ∞
  LZMA | 0.009 iter/s | 0.056 iter/s | 247.61MiB | ∞
  ZIP | 0.009 iter/s | 0.301 iter/s | 247.88MiB | ∞
  JPEG | 0.446 iter/s | 0.760 iter/s | 18.59MiB | 39.00
  WebP | 0.019 iter/s | 0.330 iter/s | 9.38MiB | 37.78

  Please note that `COMPRESSION_WEBP` is self-assigned the id 50001 by the libtiff project and is not officially registered with Adobe since Adobe's registration function is defunct.

- `TIFFPrintDirectory()`: fix null pointer dereference on corrupted file. Fixes Bug 2770 - NULL Pointer Dereference in tiffinfo.c with crafted TIFF image.

- `_TIFFVGetField()`: fix heap out-of-bounds access when requesting `TIFFTAG_NUMBEROFINKS` on a EXIF directory. Fixes Bug 2765 - Heap Out-Of-Bounds Memory Access - 68122422. Reported by Google Autofuzz project.

- Fix a memory leak in `TIFFStreamOpen()`. `TIFFStreamOpen()` allocates a new `tiff{o,i}s_data`, but if `TIFFClientOpen()` fails then that struct is leaked.

- Fix for bug 2772. It is possible to craft a TIFF document where the IFD list is circular, leading to an infinite loop while traversing the chain. The libtiff directory reader has a failsafe that will break out of this loop after reading 65535 directory entries, but it will continue processing, consuming time and resources to process what is essentially a bogus TIFFdocument.

  This change fixes the above behavior by breaking out of processing when a TIFF document has >= 65535 directories and terminating with an error.

- `ChopUpSingleUncompressedStrip()`: avoid memory exhaustion (CVE-2017-11613). In `ChopUpSingleUncompressedStrip()`, if the computed number of strips is big enough and we are in read only mode, validate that the file size is consistent with that number of strips to avoid useless attempts at allocating a lot of memory for the `td_stripbytecount` and `td_stripoffset` arrays. Fixes Bug 2724 - memory exhaustion in ChopUpSingleUncompressedStrip.

- Port code: Add `strtol()`, `strtoll()` and `strtoull()`. Also update `strtoul()`. All use the same implementation from NetBSD libc.

- Fix for CVE-2018-7456 "NULL pointer dereference in `TIFFPrintDirectory()`".

- `TIFFWriteDirectorySec()`: avoid assertion. Fixes MapTools bugzilla #Bug 2795 - There is a reachable assertion abort in function TIFFWriteDirectorySec() of libtiff 4.0.9. A crafted input will lead to remote denial of attack. (:cve:`2018-10963) <2795>`.

- `LZWDecodeCompat()`: fix potential index-out-of-bounds write. Fixes MapTools bugzilla #Bug 2780 - A heap-buffer-overflow in function LZWDecodeCompat in libtiff4.0.9 (:cve:`2018-8905) <2780>`. The fix consists in using the similar code as `LZWDecode()` to validate we don't write outside of the output buffer.

- Remove builtin support for GUI warning and error message boxes. Now warnings always go to the console by default unless applications define their own warning and error handlers.

  GUI applications (and Windows CE) are required to define such handlers.

- Add tag and pseudo-tag definitions for ESRI LERC codec (out of tree codec whose source is at https://github.com/OSGeo/gdal/blob/master/gdal/frmts/gtiff/tif_lerc.c.

- Fix libtiff 4.0.8 regression when reading LZW-compressed strips with scanline API Fixes Bug 2800 - Regression: Opening a tiff file with v4.0.9 gives an error with LZWDecode.

- `TIFFSetupStrips()`: avoid potential `uint32` overflow on 32-bit systems with large number of strips. Probably relates to MapTools bugzilla #Bug 2788 - Heap Buffer Overflow in TIFFWriteScanline of tif_write.c (:cve:`2018-10779) <2788>`.

- Fix out-of-bound read on some tiled images.

- Avoid potential `int32` overflows in `multiply_ms()`.

- Only read/write `TIFFTAG_GROUP3OPTIONS` or `TIFFTAG_GROUP4OPTIONS` if compression is `COMPRESSION_CCITTFAX3` or `COMPRESSION_CCITTFAX4`.

- JBIG: fix potential out-of-bounds write in `JBIGDecode()`. Also fix a (harmless) potential use of uninitialized memory when `tif->tif_rawsize > tif->tif_rawcc`. In case libtiff is compiled with `CHUNKY_STRIP_READ_SUPPORT`, make sure that whole strip data is provided to `JBIGDecode()`.

- `LZMAPreEncode()`: emit verbose error if `lzma_stream_encoder()` fails (typically because not enough memory available)

## Tools changes

- tiff2pdf: Fix CVE-2017-9935, Bug 2704 - There is a heap based buffer overflow in the tiff2pdf tool of the libtiff library. A crafted TIFF document can lead to a heap based buffer overflow resulting in multiple damages..

- pal2rgb: Add workaround to pal2rgb buffer overflow.

- tiffset: Add support for `LONG8`, `SLONG8` and `IFD8` field types

- tiff2bw: avoid null pointer dereference in case of out of memory situation. Fixes MapTools bugzilla #Bug 2819 - There is a NULL pointer dereference at function LZWDecode in libtiff 4.0.9 (:cve:`2018-18661) <2819>`.

## Contributed software changes

None