ValueShapes

Provides a Julia API to describe the shape of values, like scalars, arrays and structures.

AbstractScalarShape{T} <: AbstractValueShape

abstract type AbstractValueShape

An AbstractValueShape combines type and size information.

Subtypes are defined for shapes of scalars (see ScalarShape), arrays (see ArrayShape), constant values (see ConstValueShape) and NamedTuples (see NamedTupleShape).

Subtypes of AbstractValueShape must support eltype, size and totalndof.

Value shapes can be used as constructors to generate values of the given shape with undefined content. If the element type of the shape is an abstract or union type, a suitable concrete type will be chosen automatically, if possible (see ValueShapes.default_datatype):

shape = ArrayShape{Real}(2,3)
A = shape(undef)
typeof(A) == Array{Float64,2}
size(A) == (2, 3)
valshape(A) == ArrayShape{Float64}(2,3)

Use

(shape::AbstractValueShape)(data::AbstractVector{<:Real})::eltype(shape)

to view a flat vector of anonymous real values as a value of the given shape:

data = [1, 2, 3, 4, 5, 6]
shape(data) == [1 3 5; 2 4 6]

In return,

Base.Vector{<:Real}(undef, shape::AbstractValueShape)

will create a suitable uninitialized vector of the right length to hold such flat data for the given shape. If no type T is given, a suitable data type will be chosen automatically.

When dealing with multiple vectors of flattened data, use

shape.(data::ArraysOfArrays.AbstractVectorOfSimilarVectors)

ValueShapes supports this via specialized broadcasting.

In return,

ArraysOfArrays.VectorOfSimilarVectors{<:Real}(shape::AbstractValueShape)

will create a suitable vector (of length zero) of vectors that can hold flattened data for the given shape. The result will be a VectorOfSimilarVectors wrapped around a 2-dimensional ElasticArray. This way, all data is stored in a single contiguous chunk of memory.

AbstractValueShapes can be compared with <= and >=, with semantics that are similar to compare type with <: and >::

a::AbstractValueShape <= b::AbstractValueShape == true

implies that values of shape a are can be used in contexts that expect values of shape b. E.g.:

(ArrayShape{Float64}(4,5) <= ArrayShape{Real}(4,5)) == true
(ArrayShape{Float64}(4,5) <= ArrayShape{Integer}(4,5)) == false
(ArrayShape{Float64}(2,2) <= ArrayShape{Float64}(3,3)) == false
(ScalarShape{Real}() >= ScalarShape{Int}()) == true

ArrayShape{T,N} <: AbstractValueShape

Describes the shape of N-dimensional arrays of type T and a given size.

Constructor:

ArrayShape{T}(dims::NTuple{N,Integer}) where {T,N}
ArrayShape{T}(dims::Integer...) where {T}

e.g.

shape = ArrayShape{Real}(2, 3)

See also the documentation of AbstractValueShape.

ConstValueDist <: Distributions.Distribution

Represents a delta distribution for a constant value of arbritrary type.

Calling varshape on a ConstValueDist will yield a ConstValueShape.

ConstValueShape{T} <: AbstractValueShape

A ConstValueShape describes the shape of constant values of type T.

Constructor:

ConstValueShape(value)

value may be of arbitrary type, e.g. a constant scalar value or array:

ConstValueShape(4.2)
ConstValueShape([11 21; 12 22])

Shapes of constant values have zero degrees of freedom (see totalndof).

See also the documentation of AbstractValueShape.

NamedTupleDist <: MultivariateDistribution
NamedTupleDist <: MultivariateDistribution

A distribution with NamedTuple-typed variates.

NamedTupleDist provides an effective mechanism to specify the distribution of each variable/parameter in a set of named variables/parameters.

Calling varshape on a NamedTupleDist will yield a NamedTupleShape.

NamedTupleShape{names,...} <: AbstractValueShape

Defines the shape of a NamedTuple (resp. set of variables, parameters, etc.).

Constructors:

NamedTupleShape(name1 = shape1::AbstractValueShape, ...)
NamedTupleShape(named_shapes::NamedTuple)

Example:

shape = NamedTupleShape(
    a = ScalarShape{Real}(),
    b = ArrayShape{Real}(2, 3),
    c = ConstValueShape(42)
)

data = VectorOfSimilarVectors{Float64}(shape)
resize!(data, 10)
rand!(flatview(data))
table = shape.(data)
fill!(table.a, 4.2)
all(x -> x == 4.2, view(flatview(data), 1, :))

See also the documentation of AbstractValueShape.

ReshapedDist <: Distribution

An multivariate distribution reshaped using a given AbstractValueShape.

Constructors:

    ReshapedDist(dist::MultivariateDistribution, shape::AbstractValueShape)

In addition, MultivariateDistributions can be reshaped via

(shape::AbstractValueShape)(dist::MultivariateDistribution)

with the difference that

(shape::ArrayShape{T,1})(dist::MultivariateDistribution)

will return the original dist instead of a ReshapedDist.

ScalarShape{T} <: AbstractScalarShape{T}

An ScalarShape describes the shape of scalar values of a given type.

Constructor:

ScalarShape{T::Type}()

T may be an abstract type of Union, or a specific type, e.g.

ScalarShape{Real}()
ScalarShape{Integer}()
ScalarShape{Float32}()
ScalarShape{Complex}()

Scalar shapes may have a total number of degrees of freedom (see totalndof) greater than one, e.g. shapes of complex-valued scalars:

totalndof(ScalarShape{Real}()) == 1
totalndof(ScalarShape{Complex}()) == 2

See also the documentation of AbstractValueShape.

ShapedAsNT{names,...}

View of an AbstractVector{<:Real} as a mutable named tuple (though not) a NamedTuple, exactly), according to a specified NamedTupleShape.

Constructors:

ShapedAsNT(data::AbstractVector{<:Real}, shape::NamedTupleShape)

shape(data)

The resulting ShapedAsNT shares memory with data:

x = (a = 42, b = rand(1:9, 2, 3))
shape = NamedTupleShape(
    ShapedAsNT,
    a = ScalarShape{Real}(),
    b = ArrayShape{Real}(2, 3)
)
data = Vector{Int}(undef, shape)
y = shape(data)
@assert y isa ShapedAsNT
y[] = x
@assert y[] == x
y.a = 22
@assert shape(data) == y
@assert unshaped(y) === data

Use unshaped(x) to access data directly.

See also ShapedAsNTArray.

ShapedAsNTArray{T<:NamedTuple,...} <: AbstractArray{T,N}

View of an AbstractArray{<:AbstractVector{<:Real},N} as an array of NamedTuples, according to a specified NamedTupleShape.

ShapedAsNTArray implements the Tables API.

Constructors:

ShapedAsNTArray(
    data::AbstractArray{<:AbstractVector{<:Real},
    shape::NamedTupleShape
)

shape.(data)

The resulting ShapedAsNTArray shares memory with data:

using ValueShapes, ArraysOfArrays, Tables, TypedTables

X = [
    (a = 42, b = rand(1:9, 2, 3))
    (a = 11, b = rand(1:9, 2, 3))
]

shape = valshape(X[1])
data = nestedview(Array{Int}(undef, totalndof(shape), 2))
Y = shape.(data)
@assert Y isa ShapedAsNTArray
Y[:] = X
@assert Y[1] == X[1] == shape(data[1])
@assert Y.a == [42, 11]
Tables.columns(Y)
@assert unshaped.(Y) === data
@assert Table(Y) isa TypedTables.Table

Use unshaped.(Y) to access data directly.

Tables.columns(Y) will return a NamedTuple of columns. They will contain a copy the data, using a memory layout as contiguous as possible for each column.

ValueAccessor{S<:AbstractValueShape}

A value accessor provides a means to access a value with a given shape stored in a flat real-valued data vector with a given offset position.

Constructor:

ValueAccessor{S}(shape::S, offset::Int)

The offset is relative to the first index of a flat data array, so if the value is stored at the beginning of the array, the offset will be zero.

An ValueAccessor can be used to index into a given flat data array.

Example:

acc = ValueAccessor(ArrayShape{Real}(2,3), 2)
valshape(acc) == ArrayShape{Real,2}((2, 3))
data = [1, 2, 3, 4, 5, 6, 7, 8, 9]
data[acc] == acc(data) == [3 5 7; 4 6 8]

Note: Subtypes of AbstractValueShape should specialize ValueShapes.vs_getindex, ValueShapes.vs_unsafe_view and ValueShapes.vs_setindex! for their ValueAccessor{...}. Specializing Base.getindex, Base.view, Base.unsafe_view or Base.setindex! directly may result in method ambiguities with custom array tapes that specialize these functions in a very generic fashion.

const_zero(x::Any)

Get the equivalent of a constant zero for values the same type as .

const_zero_shape(shape::ConstValueShape)

Get the equivalent of a constant zero shape for shape shape that will only allow zero values to be set via an accessor.

ValueShapes.default_datatype(T::Type)

Return a default specific type U that is more specific than T, with U <: T.

e.g.

ValueShapes.default_datatype(Real) == Float64
ValueShapes.default_datatype(Complex) == Complex{Float64}

ValueShapes.default_unshaped_eltype(shape::AbstractValueShape)

Returns the default real array element type to use for unshaped representations of data with shape shape.

Subtypes of AbstractValueShape must implemenent ValueShapes.default_unshaped_eltype.

elshape(x)::AbstractValueShape

Get the shape of the elements of x

gradient_shape(argshape::AbstractValueShape)

Return the value shape of the gradient of functions that take values of shape argshape as an input.

nonstrict_const_zero_shape(shape::ConstValueShape)

Get the equivalent of a constant zero shape for shape shape that will ignore any attempt to set a value via an accessor.

Useful as a gradient/tangent varshape of constants, as they can ignore attempts to set non-zero values.

realnumtype(T::Type)

Return the underlying numerical type of T that's a subtype of Real.

Uses type promotion among underlying Real type in T.

e.g.

A = fill(fill(rand(Float32, 5), 10), 5)
realnumtype(typeof(A)) == Float32

replace_const_shapes(f::Function, shape::AbstractValueShape)

If shape is a, or contains, ConstValueShape shape(s), recursively replace it/them with the result of f(s::Shape).

resultshape(f, vs::AbstractValueShape)

Return the shape of values returned by f when applied to values of shape vs.

Returns missing if the shape of the function result cannot be determined.

ValueShapes.shaped_type(shape::AbstractValueShape, ::Type{T}) where {T<:Real}
ValueShapes.shaped_type(shape::AbstractValueShape)

Returns the type the will result from reshaping a real-valued vector (of element type T, if specified) with shape.

Subtypes of AbstractValueShape must implement

ValueShapes.shaped_type(shape::AbstractValueShape, ::Type{T}) where {T<:Real}

stripscalar(x)

Dereference value x.

If x is a scalar-like object, like a 0-dimensional array or a Ref, stripscalar returns it's inner value. Otherwise, x is returned unchanged.

Useful to strip shaped scalar-like views of their 0-dim array semantics (if present), but leave array-like views unchanged.

Example:

data = [1, 2, 3]
shape1 = NamedTupleShape(a = ScalarShape{Real}(), b = ArrayShape{Real}(2))
x1 = shape1(data)
@assert x1 isa NamedTuple

shape2 = ArrayShape{Real}(3)
x2 = shape2(data)
@assert x2 isa AbstractArray{Int,1}

totalndof(shape::AbstractValueShape)

Get the total number of degrees of freedom of values of the given shape.

Equivalent to the length of a vector that would result from flattening the data into a sequence of real numbers, excluding any constant values.

unshaped(x)::AbstractVector{<:Real}
unshaped(x, shape::AbstractValueShape)::AbstractVector{<:Real}

Retrieve the unshaped underlying data of x, assuming x is a structured view (based on some AbstractValueShape) of a flat/unstructured real-valued data vector.

If shape is given, ensures that the shape of x is compatible with it. Specifying a shape may be necessary if the correct shape of x cannot be inferred from x, e.g. because x is assumed to have fewer degrees of freedom (because of constant components) than would be inferred from the plain value of x.

Example:

shape = NamedTupleShape(
    a = ScalarShape{Real}(),
    b = ArrayShape{Real}(2, 3)
)
data = [1, 2, 3, 4, 5, 6, 7]
x = shape(data)
@assert unshaped(x, shape) == data
@assert unshaped(x.a) == view(data, 1:1)
@assert unshaped(x.b) == view(data, 2:7)

unshaped(d::Distributions.Distribution)

Turns d into a Distributions.Distribution{Multivariate} based on varshape(d).

valshape(x)::AbstractValueShape
valshape(acc::ValueAccessor)::AbstractValueShape

Get the value shape of an arbitrary value, resp. the shape a ValueAccessor is based on, or the shape of the variates for a Distribution.

variance_shape(variate_shape::AbstractValueShape)

Return the value shape of the variance of a distribution whose variates have the value shape variate_shape.

varshape(d::Distributions.Distribution)::AbstractValueShape

Get the value shape of the variates of distribution d.

ValueShapes.vs_getindex(data::AbstractArray{<:Real}, idxs::ValueAccessor...)

Specialize ValueShapes.vs_getindex instead of Base.getindex for ValueShapes.ValueAccessors, to avoid methods ambiguities with with certain custom array types.

ValueShapes.vs_setindex!(data::AbstractArray{<:Real}, v, idxs::ValueAccessor...)

Specialize ValueShapes.vs_setindex! instead of Base.setindex! or for ValueShapes.ValueAccessors, to avoid methods ambiguities with with certain custom array types.

ValueShapes.vs_unsafe_view(data::AbstractArray{<:Real}, idxs::ValueAccessor...)

Specialize ValueShapes.vs_unsafe_view instead of Base.view or Base.unsafe_view for ValueShapes.ValueAccessors, to avoid methods ambiguities with with certain custom array types.