nntoolbox.components.mixture module

Implement mixture of probability distribution layers

class nntoolbox.components.mixture.MixtureOfExpert(experts: List[torch.nn.modules.module.Module], gate: torch.nn.modules.module.Module, return_mixture: bool = True)

Bases: torch.nn.modules.module.Module

forward(input: torch.Tensor) → Union[Tuple[torch.Tensor, torch.Tensor], torch.Tensor]

Parameters

input –

Returns

if return_mixture, return the mixture of expert output; else return both expert score and expert output

(with the n_expert channel coming last)

training: bool

class nntoolbox.components.mixture.MixtureOfGaussian(in_features: int, out_features: int, n_dist: int, bias: bool = True)

Bases: torch.nn.modules.linear.Linear

A layer that generates means, stds and mixing coefficients of a mixture of gaussian distributions.

Used as the final layer of a mixture of (Gaussian) density network.

Only support isotropic covariances for the components.

References:

Christopher Bishop. “Pattern Recognition and Machine Learning”

forward(input: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

Parameters

input –

Returns

means, stds and mixing coefficients

in_features: int

out_features: int

weight: torch.Tensor