nntoolbox.vision.components.nap module

class nntoolbox.vision.components.nap.NeuralAbstractionPyramid(lateral_connections: List[torch.nn.modules.module.Module], forward_connections: List[torch.nn.modules.module.Module], backward_connections: List[torch.nn.modules.module.Module], activation_function: torch.nn.modules.module.Module, normalization: torch.nn.modules.module.Module, duration: int)[source]

Bases: torch.nn.modules.module.Module

Neural Abstraction Pyramid Module. Sharing weights both spatially and temporally:

a^t_l = norm(activation(f_l(a^{t - 1}_l) + g_l(a^{t - 1}_{l - 1}) + h_l(a^{t - l}_{l + 1})))

If f_l, g_l and h_l are repeated for all layers, then we can also share weights across depth dimension.

(UNTESTED)

References:

Sven Behnke and Ralil Rojas. “Neural Abstraction Pyramid: A hierarchical image understanding architecture.” http://page.mi.fu-berlin.de/rojas/1998/pyramid.pdf

Sven Behnke. “Hierarchical Neural Networks for Image Interpretation.” https://www.ais.uni-bonn.de/books/LNCS2766.pdf

Sven Behnke. “Face Localization and Tracking in the Neural Abstraction Pyramid.” https://www.ais.uni-bonn.de/behnke/papers/nca04.pdf

forward(input: torch.Tensor, return_all_states: bool = False, duration: Optional[int] = None) → Union[List[torch.Tensor], Tuple[List[torch.Tensor], List[List[torch.Tensor]]]][source]
Parameters

  • input –

  • return_all_states – whether to return output of all timesteps

  • duration – number of timesteps to process data

Returns

the output of last time steps and outputs of all time steps

get_initial_states(input: torch.Tensor) → List[torch.Tensor][source]
training: bool