LoweredDistributions.jl

Appearance

1

Benchmarks ​

LoweredDistributions benchmarks its lowering hot paths to track performance over time.

What the suite covers ​

  • Lowering: the adaptive lower dispatch on both sides of the c² split (an ErlangChain fit and a PhaseType hyperexponential fit), and the type-stable canonical lower(dist, PhaseType) form, including the AD-stable fixed-phase-count path.

  • Evaluation: the ctmc builder combined with transition_probability's matrix exponential, and the phase-type survival read directly off a canonical (α, S) — this package's closest analogue to a logpdf-style scoring hot path, since every representation canonicalises to a phase-type before it is scored.

  • Bridges: turning a lowering into a framework-native problem — Catalyst's reaction_system, JumpProcesses' jump_problem, and SciMLBase's ode_problem — timed as construction only, not simulation.

  • AD gradients: the same scenarios test/ad/runtests.jl exercises, timed per AD backend (ForwardDiff, ReverseDiff, Mooncake, Enzyme). A (scenario, backend) pair is only benchmarked once its gradient is confirmed finite, so a backend that cannot yet handle a scenario is silently omitted rather than failing the run.

Running the suite locally ​

bash
julia --project=benchmark benchmark/run.jl results.json

Or compare the working tree against another revision with AirspeedVelocity's benchpkg:

bash
benchpkg --rev=main,dirty --script=benchmark/benchmarks.jl

Framework-specialisation check ​

The lowering bridges above are framework-agnostic: reaction_system builds its Catalyst reactions off the same generic (α, S) phase-type walk regardless of which distribution produced it, rather than special-casing an Erlang chain. The block below checks that genericity is not a performance problem, by comparing an Erlang passage time built two ways: a hand-written Catalyst reaction chain (the framework-native form) against reaction_system applied to lower's ErlangChain fit of the equivalent Gamma. It runs live at docs-build time (a small Chairmarks budget per point), so the numbers below always reflect the current lowering code, not a pasted snapshot.

julia
using Catalyst, LoweredDistributions, Distributions, OrdinaryDiffEqTsit5
using Chairmarks

const RATE = 1.0
t = Catalyst.default_t()
Catalyst.@species Src(t) Snk(t)

function native_chain(k)
    species = [(Catalyst.@species $(Symbol(:S, i))(t))[1] for i in 1:k]
    rxs = Reaction[Reaction(RATE, [Src], [species[1]])]
    for i in 1:(k - 1)
        push!(rxs, Reaction(RATE, [species[i]], [species[i + 1]]))
    end
    push!(rxs, Reaction(RATE, [species[k]], [Snk]))
    return ReactionSystem(rxs, t; name = :native)
end

lowered_chain(k) = reaction_system(Gamma(k, 1 / RATE), Src, Snk)

function to_odeprob(rn)
    rn = complete(rn)
    u0 = [s => (isequal(s, Src) ? 1.0 : 0.0) for s in species(rn)]
    return ODEProblem(rn, u0, (0.0, 10.0))
end

for k in (3, 10, 25)
    b_construct_native = @be native_chain($k) seconds=0.5
    b_construct_lowered = @be lowered_chain($k) seconds=0.5
    prob_native = to_odeprob(native_chain(k))
    prob_lowered = to_odeprob(lowered_chain(k))
    b_solve_native = @be solve($prob_native, Tsit5()) seconds=0.5
    b_solve_lowered = @be solve($prob_lowered, Tsit5()) seconds=0.5
    construct_ratio = minimum(b_construct_lowered).time /
                       minimum(b_construct_native).time
    solve_ratio = minimum(b_solve_lowered).time / minimum(b_solve_native).time
    println("k = ", k, ": construction ratio (lowered/native) = ",
        round(construct_ratio; digits = 3), ", solve ratio = ",
        round(solve_ratio; digits = 3))
end
k = 3: construction ratio (lowered/native) = 1.015, solve ratio = 1.023
k = 10: construction ratio (lowered/native) = 0.987, solve ratio = 1.007
k = 25: construction ratio (lowered/native) = 1.014, solve ratio = 1.009

A ratio near 1 means the generic path costs about the same as writing the chain by hand. Construction runs a flat few percent slower — the extra allocations from the generic phase-species-building loop — and does not grow with the phase count; the ODE solve is at parity, since by the time the system reaches the solver both paths have produced the same list of reactions.

Reading the history ​

The plot and table below are generated by the benchmark-history CI job on every push to main and on tagged releases, and published to the repo's benchmarks branch. Each point is a full run of the suite above at that revision; a rising line in a timing panel is a regression, a falling one an improvement.

Performance history ​

The summary tracks each benchmark suite's headline timing across recent revisions.

Benchmark summary (overall) ​

Not enough comparable revisions to compute ratios yet — the summary populates once a second revision is benchmarked.

<details> <summary>Per-suite detail</summary>

Ratio summary ​

Most recent 1 revision, columns labelled by commit date.

AD gradients ​

Time ​

Benchmark9afc6521b0c198...
ctmc(specs...) builder + transition_probability gradient/ForwardDiff22.2 ± 7.7 μs
ctmc(specs...) builder + transition_probability gradient/Mooncake forward0.25 ± 0.02 ms
ctmc(specs...) builder + transition_probability gradient/Mooncake reverse1.26 ± 0.049 ms
ctmc(specs...) builder + transition_probability gradient/ReverseDiff (tape)0.804 ± 0.1 ms
lower(composer) joint-CTMC transition gradient/Enzyme forward0.0967 ± 0.0059 ms
lower(composer) joint-CTMC transition gradient/Enzyme reverse0.415 ± 0.039 ms
lower(composer) joint-CTMC transition gradient/ForwardDiff0.0432 ± 0.0038 ms
lower(composer) joint-CTMC transition gradient/Mooncake forward0.23 ± 0.017 ms
lower(composer) joint-CTMC transition gradient/Mooncake reverse1.91 ± 0.2 ms
lower(composer) joint-CTMC transition gradient/ReverseDiff (tape)2.73 ± 0.35 ms
lower(composer) scalar-composer survival gradient/Enzyme forward0.0475 ± 0.0056 ms
lower(composer) scalar-composer survival gradient/Enzyme reverse0.553 ± 0.04 ms
lower(composer) scalar-composer survival gradient/ForwardDiff20.5 ± 4.7 μs
lower(composer) scalar-composer survival gradient/Mooncake forward0.127 ± 0.02 ms
lower(composer) scalar-composer survival gradient/Mooncake reverse1.58 ± 0.079 ms
lower(composer) scalar-composer survival gradient/ReverseDiff (tape)0.82 ± 0.099 ms
lower(dist) adaptive Erlang survival gradient (integer shape)/Enzyme forward10.5 ± 3.8 μs
lower(dist) adaptive Erlang survival gradient (integer shape)/Enzyme reverse0.254 ± 0.023 ms
lower(dist) adaptive Erlang survival gradient (integer shape)/ForwardDiff7.75 ± 0.78 μs
lower(dist) adaptive Erlang survival gradient (integer shape)/Mooncake forward0.0381 ± 0.011 ms
lower(dist) adaptive Erlang survival gradient (integer shape)/Mooncake reverse0.47 ± 0.12 ms
lower(dist) adaptive Erlang survival gradient (integer shape)/ReverseDiff (tape)0.789 ± 0.096 ms
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Enzyme forward9.51 ± 0.96 μs
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Enzyme reverse0.241 ± 0.026 ms
lower(dist) adaptive Erlang survival gradient (non-integer shape)/ForwardDiff4.69 ± 0.84 μs
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Mooncake forward25.8 ± 2.2 μs
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Mooncake reverse0.338 ± 0.036 ms
lower(dist) adaptive Erlang survival gradient (non-integer shape)/ReverseDiff (tape)0.299 ± 0.041 ms
lower(dist) adaptive-dispatch survival gradient/Enzyme forward9.67 ± 1.2 μs
lower(dist) adaptive-dispatch survival gradient/Enzyme reverse0.236 ± 0.021 ms
lower(dist) adaptive-dispatch survival gradient/ForwardDiff5.06 ± 1.1 μs
lower(dist) adaptive-dispatch survival gradient/Mooncake forward26.5 ± 3.8 μs
lower(dist) adaptive-dispatch survival gradient/Mooncake reverse0.303 ± 0.028 ms
lower(dist) adaptive-dispatch survival gradient/ReverseDiff (tape)0.335 ± 0.051 ms
lower(dist, PhaseType) survival gradient (c² > 1)/Enzyme forward9.71 ± 1.8 μs
lower(dist, PhaseType) survival gradient (c² > 1)/Enzyme reverse0.241 ± 0.028 ms
lower(dist, PhaseType) survival gradient (c² > 1)/ForwardDiff5.16 ± 0.97 μs
lower(dist, PhaseType) survival gradient (c² > 1)/Mooncake forward27.1 ± 3.5 μs
lower(dist, PhaseType) survival gradient (c² > 1)/Mooncake reverse0.314 ± 0.029 ms
lower(dist, PhaseType) survival gradient (c² > 1)/ReverseDiff (tape)0.341 ± 0.051 ms
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Enzyme forward10.2 ± 3.6 μs
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Enzyme reverse0.242 ± 0.028 ms
lower(dist, PhaseType) survival gradient (c² ≤ 1)/ForwardDiff7.43 ± 0.74 μs
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Mooncake forward0.0379 ± 0.012 ms
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Mooncake reverse0.475 ± 0.05 ms
lower(dist, PhaseType) survival gradient (c² ≤ 1)/ReverseDiff (tape)0.794 ± 0.096 ms
lower(dist, PhaseType; phases) fixed-count survival gradient/Enzyme forward27 ± 2.8 μs
lower(dist, PhaseType; phases) fixed-count survival gradient/Enzyme reverse0.121 ± 0.012 ms
lower(dist, PhaseType; phases) fixed-count survival gradient/ForwardDiff16.1 ± 0.79 μs
lower(dist, PhaseType; phases) fixed-count survival gradient/Mooncake forward0.0649 ± 0.0046 ms
lower(dist, PhaseType; phases) fixed-count survival gradient/Mooncake reverse0.351 ± 0.042 ms
lower(dist, PhaseType; phases) fixed-count survival gradient/ReverseDiff (tape)2.9 ± 0.35 ms
matrix_exp/transition_probability direct gradient/Enzyme forward24.5 ± 10 μs
matrix_exp/transition_probability direct gradient/Enzyme reverse0.25 ± 0.031 ms
matrix_exp/transition_probability direct gradient/ForwardDiff13.1 ± 0.54 μs
matrix_exp/transition_probability direct gradient/Mooncake forward0.0698 ± 0.0085 ms
matrix_exp/transition_probability direct gradient/Mooncake reverse0.428 ± 0.12 ms
matrix_exp/transition_probability direct gradient/ReverseDiff (tape)0.78 ± 0.098 ms
ode_problem solve survival gradient (PhaseType)/Enzyme forward16.9 ± 0.68 ms
ode_problem solve survival gradient (PhaseType)/ForwardDiff0.29 ± 0.013 ms
ode_problem solve survival gradient (PhaseType, direct)/Enzyme forward17 ± 0.58 ms
ode_problem solve survival gradient (PhaseType, direct)/ForwardDiff0.296 ± 0.014 ms
phase_type hyperexponential (α, S) gradient/Enzyme forward6.19 ± 0.072 μs
phase_type hyperexponential (α, S) gradient/Enzyme reverse1.32 ± 0.035 μs
phase_type hyperexponential (α, S) gradient/ForwardDiff0.491 ± 0.051 μs
phase_type hyperexponential (α, S) gradient/Mooncake forward11.4 ± 0.56 μs
phase_type hyperexponential (α, S) gradient/Mooncake reverse14.7 ± 0.73 μs
phase_type hyperexponential (α, S) gradient/ReverseDiff (tape)8.06 ± 0.21 μs

Memory ​

Benchmark9afc6521b0c198...
ctmc(specs...) builder + transition_probability gradient/ForwardDiff0.22 k allocs: 26.3 kB
ctmc(specs...) builder + transition_probability gradient/Mooncake forward2.8 k allocs: 0.123 MB
ctmc(specs...) builder + transition_probability gradient/Mooncake reverse8.45 k allocs: 0.79 MB
ctmc(specs...) builder + transition_probability gradient/ReverseDiff (tape)8.96 k allocs: 0.371 MB
lower(composer) joint-CTMC transition gradient/Enzyme forward0.8 k allocs: 0.0514 MB
lower(composer) joint-CTMC transition gradient/Enzyme reverse2 k allocs: 0.196 MB
lower(composer) joint-CTMC transition gradient/ForwardDiff0.408 k allocs: 0.0391 MB
lower(composer) joint-CTMC transition gradient/Mooncake forward1.9 k allocs: 0.119 MB
lower(composer) joint-CTMC transition gradient/Mooncake reverse11.8 k allocs: 1.07 MB
lower(composer) joint-CTMC transition gradient/ReverseDiff (tape)0.0318 M allocs: 1.36 MB
lower(composer) scalar-composer survival gradient/Enzyme forward0.595 k allocs: 28.8 kB
lower(composer) scalar-composer survival gradient/Enzyme reverse1.47 k allocs: 0.111 MB
lower(composer) scalar-composer survival gradient/ForwardDiff0.305 k allocs: 20.8 kB
lower(composer) scalar-composer survival gradient/Mooncake forward1.44 k allocs: 0.0697 MB
lower(composer) scalar-composer survival gradient/Mooncake reverse10.9 k allocs: 0.984 MB
lower(composer) scalar-composer survival gradient/ReverseDiff (tape)9.19 k allocs: 0.379 MB
lower(dist) adaptive Erlang survival gradient (integer shape)/Enzyme forward0.259 k allocs: 16.7 kB
lower(dist) adaptive Erlang survival gradient (integer shape)/Enzyme reverse0.768 k allocs: 0.0801 MB
lower(dist) adaptive Erlang survival gradient (integer shape)/ForwardDiff0.124 k allocs: 12.5 kB
lower(dist) adaptive Erlang survival gradient (integer shape)/Mooncake forward0.586 k allocs: 0.0374 MB
lower(dist) adaptive Erlang survival gradient (integer shape)/Mooncake reverse3.29 k allocs: 0.58 MB
lower(dist) adaptive Erlang survival gradient (integer shape)/ReverseDiff (tape)8.89 k allocs: 0.368 MB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Enzyme forward0.247 k allocs: 12.7 kB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Enzyme reverse0.708 k allocs: 0.0726 MB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/ForwardDiff0.118 k allocs: 7.86 kB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Mooncake forward0.562 k allocs: 30.4 kB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/Mooncake reverse2.73 k allocs: 0.279 MB
lower(dist) adaptive Erlang survival gradient (non-integer shape)/ReverseDiff (tape)3.25 k allocs: 0.137 MB
lower(dist) adaptive-dispatch survival gradient/Enzyme forward0.261 k allocs: 13.7 kB
lower(dist) adaptive-dispatch survival gradient/Enzyme reverse0.647 k allocs: 0.0714 MB
lower(dist) adaptive-dispatch survival gradient/ForwardDiff0.125 k allocs: 8.48 kB
lower(dist) adaptive-dispatch survival gradient/Mooncake forward0.588 k allocs: 0.0315 MB
lower(dist) adaptive-dispatch survival gradient/Mooncake reverse2.51 k allocs: 0.273 MB
lower(dist) adaptive-dispatch survival gradient/ReverseDiff (tape)3.75 k allocs: 0.154 MB
lower(dist, PhaseType) survival gradient (c² > 1)/Enzyme forward0.261 k allocs: 13.7 kB
lower(dist, PhaseType) survival gradient (c² > 1)/Enzyme reverse0.651 k allocs: 0.0715 MB
lower(dist, PhaseType) survival gradient (c² > 1)/ForwardDiff0.125 k allocs: 8.48 kB
lower(dist, PhaseType) survival gradient (c² > 1)/Mooncake forward0.588 k allocs: 0.0315 MB
lower(dist, PhaseType) survival gradient (c² > 1)/Mooncake reverse2.56 k allocs: 0.275 MB
lower(dist, PhaseType) survival gradient (c² > 1)/ReverseDiff (tape)3.78 k allocs: 0.156 MB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Enzyme forward0.245 k allocs: 16.1 kB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Enzyme reverse0.631 k allocs: 0.0727 MB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/ForwardDiff0.117 k allocs: 12.1 kB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Mooncake forward0.556 k allocs: 0.0363 MB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/Mooncake reverse3.12 k allocs: 0.571 MB
lower(dist, PhaseType) survival gradient (c² ≤ 1)/ReverseDiff (tape)8.87 k allocs: 0.367 MB
lower(dist, PhaseType; phases) fixed-count survival gradient/Enzyme forward0.241 k allocs: 28.9 kB
lower(dist, PhaseType; phases) fixed-count survival gradient/Enzyme reverse0.621 k allocs: 0.0874 MB
lower(dist, PhaseType; phases) fixed-count survival gradient/ForwardDiff0.115 k allocs: 25.8 kB
lower(dist, PhaseType; phases) fixed-count survival gradient/Mooncake forward0.548 k allocs: 0.0613 MB
lower(dist, PhaseType; phases) fixed-count survival gradient/Mooncake reverse1.87 k allocs: 0.358 MB
lower(dist, PhaseType; phases) fixed-count survival gradient/ReverseDiff (tape)0.033 M allocs: 1.41 MB
matrix_exp/transition_probability direct gradient/Enzyme forward0.496 k allocs: 0.0319 MB
matrix_exp/transition_probability direct gradient/Enzyme reverse0.65 k allocs: 0.0721 MB
matrix_exp/transition_probability direct gradient/ForwardDiff0.123 k allocs: 21.7 kB
matrix_exp/transition_probability direct gradient/Mooncake forward1.05 k allocs: 0.0694 MB
matrix_exp/transition_probability direct gradient/Mooncake reverse2.99 k allocs: 0.563 MB
matrix_exp/transition_probability direct gradient/ReverseDiff (tape)8.86 k allocs: 0.367 MB
ode_problem solve survival gradient (PhaseType)/Enzyme forward0.0549 M allocs: 2.35 MB
ode_problem solve survival gradient (PhaseType)/ForwardDiff0.666 k allocs: 0.0489 MB
ode_problem solve survival gradient (PhaseType, direct)/Enzyme forward0.0549 M allocs: 2.35 MB
ode_problem solve survival gradient (PhaseType, direct)/ForwardDiff0.674 k allocs: 0.0493 MB
phase_type hyperexponential (α, S) gradient/Enzyme forward29 allocs: 1.06 kB
phase_type hyperexponential (α, S) gradient/Enzyme reverse18 allocs: 0.656 kB
phase_type hyperexponential (α, S) gradient/ForwardDiff9 allocs: 0.406 kB
phase_type hyperexponential (α, S) gradient/Mooncake forward0.124 k allocs: 7.08 kB
phase_type hyperexponential (α, S) gradient/Mooncake reverse0.204 k allocs: 10.3 kB
phase_type hyperexponential (α, S) gradient/ReverseDiff (tape)0.123 k allocs: 4.84 kB

Bridges ​

Time ​

Benchmark9afc6521b0c198...
jump_problem (JumpProcesses)1.22 ± 0.079 μs
ode_problem (SciMLBase)0.166 ± 0.0052 ms
reaction_system (Catalyst)0.0742 ± 0.0032 ms

Memory ​

Benchmark9afc6521b0c198...
jump_problem (JumpProcesses)0.052 k allocs: 2.38 kB
ode_problem (SciMLBase)0.596 k allocs: 0.0446 MB
reaction_system (Catalyst)0.68 k allocs: 25.7 kB

Evaluation ​

Time ​

Benchmark9afc6521b0c198...
ctmc builder + transition_probability4.77 ± 0.97 μs
phase-type matrix_exp3.8 ± 2.5 μs

Memory ​

Benchmark9afc6521b0c198...
ctmc builder + transition_probability0.11 k allocs: 7.69 kB
phase-type matrix_exp0.114 k allocs: 6.22 kB

Lowering ​

Time ​

Benchmark9afc6521b0c198...
canonical(Erlang branch)0.148 ± 0.038 μs
canonical(PhaseType branch)0.126 ± 0.059 μs
canonical(fixed phases)0.195 ± 0.028 μs
lower(Exponential)1.98 ± 0.064 μs
lower(Gamma, Erlang branch)0.0469 ± 0.025 μs
lower(Gamma, PhaseType branch)0.126 ± 0.061 μs

Memory ​

Benchmark9afc6521b0c198...
canonical(Erlang branch)4 allocs: 0.219 kB
canonical(PhaseType branch)4 allocs: 0.188 kB
canonical(fixed phases)4 allocs: 0.359 kB
lower(Exponential)15 allocs: 0.797 kB
lower(Gamma, Erlang branch)3 allocs: 0.0938 kB
lower(Gamma, PhaseType branch)5 allocs: 0.219 kB

time_to_load ​

Time ​

Benchmark9afc6521b0c198...
time_to_load0.509 ± 0.0023 s

Memory ​

Benchmark9afc6521b0c198...
time_to_load0.149 k allocs: 11.2 kB

Per-benchmark timelines ​

<details> <summary>Show 4 plots</summary>

plot_LoweredDistributions_1.pngplot_LoweredDistributions_2.pngplot_LoweredDistributions_3.pngplot_LoweredDistributions_4.png

</details>

</details>