EncodedArray provides an API for arrays that store their elements in encoded/compressed form. This package is meant to be lightweight and only implements a simple codec
VarlenDiffArrayCodec. As codec implementations are often complex and have various dependencies, more advanced codecs should be implemented in separate packages.
Random access on an encoded array will typically be very inefficient, but linear access may be efficient (depending on the codec). Accessing the whole array contents at once, e.g. via
A[:], or copying/appending/conversion to a regular array, must be efficient.
An encoded array will typically have very inefficient random access, but may have efficient linear access and must be efficient when accessing the whole array contents at once via
getindex, copying/appending to a regular array, etc.
This package defines two central abstract types,
AbstractArrayCodec. It also defines a concrete type
EncodedArray that implements most of the API and only leaves
EncodedArrays.decode_data! for a new codec to implement.
Custom broadcasting optimizations are not implemented yet but will likely be added in the future.