Skip to content

Using xz-file-format for 1TB archive

How practical would it really be to use xz-file-format to create a 1TB archive with artifacts that have 3KB median size and 80KB average size ?

  • Each artifact would be a individually compressed block
  • An uncompressed index sorted index with fixed record sizes SHA256 => number of block is stored in the last block

The index does not fit in memory and needs sorting.

Migrated from T3046 (view on Phabricator)