abstract type AbstractArrayCodec <: Codecs.Codec end

Abstract type for arrays codecs.

Subtypes must implement the AbstractEncodedArray API. Most coded should use EncodedArray as the concrete subtype of AbstractArrayCodec. Codecs that use a custom subtype of AbstractEncodedArray must implement

EncodedArrays.encarraytype(::Type{<:AbstractArrayCodec},::Type{<:AbstractArray{T,N}})::Type{<:AbstractEncodedArray{T,N}}

AbstractEncodedArray{T,N} <: AbstractArray{T,N}

Abstract type for arrays that store their elements in encoded/compressed form.

In addition to the standard AbstractArray API, an AbstractEncodedArray must support the functions

• EncodedArrays.getcodec(A::EncodedArray): Returns the codec.
• Base.codeunits(A::EncodedArray): Returns the internal encoded data representation.

Encoded arrays will typically be created via

A_enc = (codec::AbstractArrayCodec)(A::AbstractArray)

or

A_enc = AbstractEncodedArray(undef, codec::AbstractArrayCodec)
append!(A_enc, B::AbstractArray)

Decoding happens via standard array conversion or assignment:

A_dec = Array(A)
A_dec = convert(Array,A)
A_dec = A[:]

A_dec = Array{T,N}(undef, size(A_enc)...)
A_dec[:] = A_enc

EncodedArray{T,N,C,DV} <: AbstractEncodedArray{T,N}

Concrete type for AbstractEncodedArrays.

Constructor:

EncodedArray{T}(
    codec::AbstractArrayCodec,
    size::NTuple{N,Integer},
    encoded::AbstractVector{UInt8}
)

Codecs using EncodedArray only need to implement EncodedArrays.encode_data! and EncodedArrays.decode_data!.

If length of the decoded data can be inferred from the encoded data, a constructor

EncodedArray{T,N}(codec::MyCodec,encoded::AbstractVector{UInt8})

should also be defined. By default, two EncodedArrays that have the same codec and size are assumed to be equal if and only if their code units are equal.

Generic methods for the rest of the AbstractEncodedArray API are already provided for EncodedArray.

VarlenDiffArrayCodec <: AbstractArrayCodec

VectorOfEncodedArrays{T,N,...}

A vector of encoded arrays.

The code units of all entries are stored in contiguous fashion using an ArraysOfArray.VectorOfArrays. All element arrays are encoded using the same codec.

VectorOfEncodedSimilarArrays{T,M,C,...}

A vector of encoded arrays that have the same original size.

The code units of all entries are stored in contiguous fashion using an ArraysOfArray.VectorOfArrays. All element arrays are encoded using the same codec.

¦>(A::AbstractArray{T}, codec::AbstractArrayCodec)::AbstractEncodedArray

Encode A using codec and return an AbstractEncodedArray. The default implementation returns an EncodedArray.

decode_data!(data::AbstractArray, codec::AbstractArrayCodec, encoded::AbstractVector{UInt8})

Depending on codec, may or may not resize decoded to fit the size of the decoded data. Codecs may require decoded to be of correct size (e.g. to improved performance or when the size/shape of the decoded data cannot be easily inferred from the encoded data.

Returns data.

encode_data!(encoded::AbstractVector{UInt8}, codec::AbstractArrayCodec, data::AbstractArray)

Will resize encoded as necessary to fit the encoded data.

Returns encoded.

EncodedArrays.getcodec(A::AbstractEncodedArray)::AbstractArrayCodec

Returns the codec used to encode/compress A.

read_autozz_varlen(io::IO, ::Type{<:Integer})

Read an integer of type T from io, using zig-zag decoding depending on whether T is signed or unsigned.

read_varlen(io::IO, T::Type{<:Unsigned})

Read an unsigned variable-length integer value of type T from io. If the next value encoded in x is too large to be represented by T, an exception is thrown.

See EncodedArrays.write_varlen.

write_varlen(io::IO, x::Unsigned)

Write unsigned integer value x to IO using variable-length coding. Data is written in LSB fashion in units of one byte. The highest bit of each byte indicates if more bytes will need to be read, the 7 lower bits contain the next 7 bits of x.