me.lemire.integercompression

Class FastPFOR

java.lang.Object

me.lemire.integercompression.FastPFOR

All Implemented Interfaces:

IntegerCODEC

public final class FastPFOR
extends Object
implements IntegerCODEC

This is a patching scheme designed for speed. It encodes integers in blocks of 128 integers within pages of up to 65536 integers. Note that it is important, to get good compression and good performance, to use sizeable blocks (greater than 1024 integers). For arrays containing a number of integers that is not divisible by 128, you should use it in conjunction with another CODEC: IntegerCODEC ic = new Composition(new FastPFOR(), new VariableByte()).

For details, please see

Daniel Lemire and Leonid Boytsov, Decoding billions of integers per second through vectorization Software: Practice & Experience http://onlinelibrary.wiley.com/doi/10.1002/spe.2203/abstract http://arxiv.org/abs/1209.2137

For sufficiently compressible and long arrays, it is faster and better than other PFOR schemes.

Note that this does not use differential coding: if you are working on sorted lists, use IntegratedFastPFOR instead. For multi-threaded applications, each thread should use its own FastPFOR object.

Author:

Daniel Lemire

Constructor Summary

Constructors

Constructor and Description
FastPFOR() Construct the fastPFOR CODEC with default parameters.
FastPFOR(int pagesize) Construct the FastPFOR CODEC.

Method Summary

Modifier and Type	Method and Description
void	compress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos) Compress data in blocks of 128 integers (if fewer than 128 integers are provided, nothing is done).
String	toString()
void	uncompress(int[] in, IntWrapper inpos, int inlength, int[] out, IntWrapper outpos) Uncompress data in blocks of 128 integers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FastPFOR

public FastPFOR(int pagesize)

Construct the FastPFOR CODEC.

Parameters:

pagesize - the desired page size (for expert use)

FastPFOR

public FastPFOR()

Construct the fastPFOR CODEC with default parameters.

Method Detail

compress

public void compress(int[] in,
                     IntWrapper inpos,
                     int inlength,
                     int[] out,
                     IntWrapper outpos)

Compress data in blocks of 128 integers (if fewer than 128 integers are provided, nothing is done).

Specified by:

compress in interface IntegerCODEC

Parameters:

in - input array

inpos - location in the input array

inlength - how many integers to compress

out - output array

outpos - where to write in the output array

See Also:

IntegerCODEC.compress(int[], IntWrapper, int, int[], IntWrapper)

uncompress

public void uncompress(int[] in,
                       IntWrapper inpos,
                       int inlength,
                       int[] out,
                       IntWrapper outpos)

Uncompress data in blocks of 128 integers. In this particular case, the inlength parameter is ignored: it is deduced from the compressed data.

Specified by:

uncompress in interface IntegerCODEC

Parameters:

in - array containing data in compressed form

inpos - where to start reading in the array

inlength - length of the compressed data (ignored by some schemes)

out - array where to write the compressed output

outpos - where to write the compressed output in out

See Also:

IntegerCODEC.compress(int[], IntWrapper, int, int[], IntWrapper)

toString

public String toString()

Overrides:

toString in class Object