Entropy Coders

View previous topic View next topic Go down

Entropy Coders

Post  Yann on Tue 17 Nov - 17:58

Entropy coders reduce a set of symbols according to their appearance probabilities.
They are typically used as stage 2 of many compression programs, including zip, rar or 7zip.

1) Range0 v0.7
New : small encoding & decoding speed improvements in extreme situations (badly or highly compressible data)
This is a semi-static Range Coder implementation. Data is divided into blocks of 128KB. Each compressed block starts with a frequency count header.
Compression speed : 145-188 MB/s
Decoding speed : 77-148 MB/s

2) Huff0 v0.9
New : Improved multi-threading code (benchmark mode only)
This is a semi-static huffman coder implementation. Data is divided into blocks of 16KB. Each compressed block starts with a huffman tree header.
Compression speed : 190-285 MB/s
Decoding speed : 184-280 MB/s

3) HuffX v0.1
New : Initial release, merge huff0 and range0, multi-threading available in benchmark mode
This is a mixed huffman and range coder, trying to provide the best of both worlds. Data is divided into blocks of 16KB.
Compression speed : 179-284 MB/s
Decoding speed : 141-271 MB/s


Detailed performance assessment :
Benchmark platform : Core 2 Duo E8400 (3GHz), Window Seven 32-bits



Comparing with other entropy coders :

Shindlet, from Piotr Tarsa, an optimized order-0 dynamic range coder written in assembler.
Compression speed : 18 MB/s
Decoding speed : 16.5 MB/s
enwik8 ratio : 1.590

o0c from Eugene Shelwien, a reference dynamic range coder
Compression speed : 5 MB/s
Decoding speed : 4.7 MB/s
enwik8 ratio : 1.585

o0rc from Eugene Shelwien; an adaptative dynamic range coder, with best-in-class compression ratio & excellent speed
Compression speed : 16 MB/s
Decoding speed : 15.2 MB/s
enwik8 ratio : 1.633 (best ratio)


Last edited by Yann on Sat 26 Mar - 13:56; edited 62 times in total

Yann
Admin

Number of posts : 174
Registration date : 2008-05-01

http://phantasie.tonempire.net

Back to top Go down

old versions

Post  Yann on Wed 25 Nov - 1:43

Range0

Range0 v0.7
small encoding & decoding speed improvements in extreme situations (badly or highly compressible data)

Range0 v0.6
Encoding & decoding speed improvements in extreme situations (badly or highly compressible data)

Range0 v0.5
Improved speed (compression 20%, decoding 10%) at the expense of compression rate (-0.1%)

Range0 v0.4
Slightly improved decoding speed

Range0 v0.3
Slightly improved compression rate; Corrected behavior with zero-filled sequences

Range0 v0.2
Slightly improved compression rate, compression animation report (cosmetic)

Range0 v0.1
Initial release


Huff0


Huff0 v0.9
Improved multi-threading code (benchmark mode only)

Huff0 v0.8
Multi-threading code (benchmark mode only)

Huff0 v0.7
Speed improvement in corner-case situations (badly or highly compressible data)

Huff0 v0.6
Significant speed improvement in extreme situations (badly or highly compressible data)

Huff0 v0.5
Small speed improvement

Huff0 v0.4
Improved compression rate

Huff0 v0.3
More flexible I/O, more precise benchmark, slightly improved compression rate

Huff0 v0.2
improved decoding speed

Huff0 v0.1
initial release


HuffX


HuffX v0.1
Initial release, merge huff0 and range0, multi-threading available in benchmark mode

Yann
Admin

Number of posts : 174
Registration date : 2008-05-01

http://phantasie.tonempire.net

Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum