Skip to content

Divergence API

Top-level divergence helpers are documented under Top-level API. This page documents estimator configuration classes from gmm_divergence.divergence.

gmm_divergence.divergence

Public divergence API.

estimate_divergence 

estimate_divergence(
    p: GaussianLike, q: GaussianLike, /, *, divergence: DivergenceSpec = "kl"
) -> DivergenceResult

Estimate a supported divergence between two Gaussian-family distributions.

This is the general divergence dispatch API. It currently routes to the already-supported KL, symmetric KL, and Jensen-Shannon helpers; it does not introduce new divergence estimators.

Parameters:

  • p (Gaussian or GaussianMixture) –

    The two Gaussian-family distributions to compare.

  • q (Gaussian or GaussianMixture) –

    The two Gaussian-family distributions to compare.

  • divergence (str or divergence configuration, default: "kl" ) –

    Divergence family to estimate. Supported string values are "kl", "symmetric_kl", and "jensen_shannon". Configuration objects such as divergence.KLDivergence(...) can carry the KL estimation method.

Returns:

  • DivergenceResult –

    Result object containing the estimated divergence and metadata about the computation.

Source code in src/gmm_divergence/divergence/_api.py 
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
def estimate_divergence(
    p: GaussianLike, q: GaussianLike, /, *, divergence: DivergenceSpec = "kl"
) -> DivergenceResult:
    """Estimate a supported divergence between two Gaussian-family distributions.

    This is the general divergence dispatch API. It currently routes to the
    already-supported KL, symmetric KL, and Jensen-Shannon helpers; it does not
    introduce new divergence estimators.

    Parameters
    ----------
    p, q : Gaussian or GaussianMixture
        The two Gaussian-family distributions to compare.
    divergence : str or divergence configuration, default="kl"
        Divergence family to estimate. Supported string values are ``"kl"``,
        ``"symmetric_kl"``, and ``"jensen_shannon"``. Configuration objects such
        as ``divergence.KLDivergence(...)`` can carry the KL estimation method.

    Returns
    -------
    DivergenceResult
        Result object containing the estimated divergence and metadata about
        the computation.

    """
    spec, options = _DIVERGENCE_REGISTRY.resolve(divergence)
    match spec.name:
        case "kl" if isinstance(options, KLDivergence):
            return kl_divergence(
                p, q, method=options.method, prefer_closed_form=options.prefer_closed_form
            )
        case "symmetric_kl" if isinstance(options, SymmetricKLDivergence):
            return symmetric_kl_divergence(
                p, q, method=options.method, prefer_closed_form=options.prefer_closed_form
            )
        case "jensen_shannon" if isinstance(options, JensenShannonDivergence):
            return jensen_shannon_divergence(
                p, q, method=options.method, prefer_closed_form=options.prefer_closed_form
            )
        case _:
            msg = "Unhandled divergence registry entry."
            raise AssertionError(msg)

KLDivergence dataclass 

KLDivergence(method: KLMethod = 'monte_carlo', prefer_closed_form: bool = True)

Configuration for the directed Kullback-Leibler divergence.

method class-attribute instance-attribute 

method: KLMethod = 'monte_carlo'

Method used to estimate KL(p || q).

prefer_closed_form class-attribute instance-attribute 

prefer_closed_form: bool = True

Use closed-form Gaussian KL when both inputs are single Gaussian.

SymmetricKLDivergence dataclass 

SymmetricKLDivergence(
    method: KLMethod = "monte_carlo", prefer_closed_form: bool = True
)

Configuration for the symmetric Kullback-Leibler divergence.

method class-attribute instance-attribute 

method: KLMethod = 'monte_carlo'

Method used for each directed KL estimate.

prefer_closed_form class-attribute instance-attribute 

prefer_closed_form: bool = True

Use closed-form Gaussian KL in each direction when possible.

JensenShannonDivergence dataclass 

JensenShannonDivergence(
    method: KLMethod = "monte_carlo", prefer_closed_form: bool = True
)

Configuration for the Jensen-Shannon divergence.

method class-attribute instance-attribute 

method: KLMethod = 'monte_carlo'

Method used for the KL estimates against the midpoint mixture.

prefer_closed_form class-attribute instance-attribute 

prefer_closed_form: bool = True

Pass through closed-form preference to the underlying KL estimates.

MonteCarlo dataclass 

MonteCarlo(sampling: int | SampleSpec | None = None)

Monte Carlo KL estimator configuration.

Estimates the KL divergence with samples from the reference distribution \(p\):

\[ D_{\mathrm{KL}}(p \| q) = \mathbb{E}_{X \sim p} \left[ \log p(X) - \log q(X) \right]. \]

This is the most general estimator: it can be used whenever p can be sampled and both p and q can evaluate log densities.

Sampling is configured explicitly with sampling.Draw(...), sampling.Samples(...), or sampling.Stratified(...). Adaptive standard-error control requires sampling.Draw(...) because it must draw additional batches.

Source code in src/gmm_divergence/divergence/_options.py 
39
40
41
42
def __init__(self, sampling: int | SampleSpec | None = None) -> None:
    if isinstance(sampling, int):
        sampling = Draw(n_samples=sampling)
    object.__setattr__(self, "sampling", sampling or Draw())

sampling class-attribute instance-attribute 

sampling: SampleSpec = field(default_factory=Draw)

Sampling specification used to estimate the expectation under p.

Unscented dataclass 

Unscented()

Unscented-transform KL estimator configuration.

Estimates the KL divergence by replacing random samples from \(p\) with deterministic sigma points generated from the Gaussian components of \(p\). For each component, the sigma points are chosen to capture its mean and covariance structure, then the estimator averages

\[ \log p(x) - \log q(x) \]

over those points. This can be useful when deterministic, low-sample-count estimates are preferred over Monte Carlo sampling.

MomentMatchedGaussian dataclass 

MomentMatchedGaussian(approximation: Approximation = 'moment_matching')

Moment-matched Gaussian KL estimator configuration.

Approximates one or both inputs with Gaussian summaries and then computes a Gaussian KL surrogate for

\[ D_{\mathrm{KL}}(p \| q). \]

This is a fast heuristic rather than an exact Gaussian-mixture KL computation. It is mainly useful as a rough baseline or when a cheap approximation is preferable to a sampled estimate.

The "moment_matching" strategy replaces each Gaussian mixture by a single Gaussian with the same mean and covariance, then computes the closed-form Gaussian KL divergence between those summaries. This preserves global first and second moments but discards multimodality.

The "nearest" strategy computes pairwise closed-form KL divergences between Gaussian components and returns the smallest component-level value. This captures the closest local match between the mixtures but ignores mixture weights and the full mixture shape.

approximation class-attribute instance-attribute 

approximation: Approximation = 'moment_matching'

Gaussian approximation strategy to use.

ClosedForm dataclass 

ClosedForm()

Closed-form Gaussian KL configuration.

Computes the exact KL divergence between two Gaussian distributions:

\[ D_{\mathrm{KL}}(p \| q) = \frac{1}{2} \left[ \mathrm{tr}(\Sigma_q^{-1}\Sigma_p) + (\mu_q-\mu_p)^\top\Sigma_q^{-1}(\mu_q-\mu_p) - d + \log\frac{\det\Sigma_q}{\det\Sigma_p} \right]. \]

This method is only valid when both inputs are single Gaussian distributions.

Variational dataclass 

Variational()

Variational KL estimator configuration.