gsnn.models.GroupEMANorm

Group-wise Exponential Moving Average Normalization.

EMANorm maintains running statistics using exponential moving averages but doesn’t depend on current batch statistics during normalization, making it very robust for small and variable batch sizes.

Classes

GroupEMANorm(*args, **kwargs)

Applies normalization within each channel group using exponential moving averages.
class gsnn.models.GroupEMANorm.GroupEMANorm(*args: Any, **kwargs: Any)[source]

Bases: Module

Applies normalization within each channel group using exponential moving averages.

This normalization maintains running statistics but doesn’t use current batch statistics for normalization, making it very stable for small batch sizes.

Parameters:

  • channel_groups (list or tensor) – Specifies which group each channel belongs to.

  • eps (float) – Small value to avoid division by zero. Default: 1e-5

  • momentum (float) – Momentum for updating running statistics. Default: 0.1

  • affine (bool) – If True, applies learnable scale and bias. Default: True

  • track_running_stats (bool) – If True, maintains running statistics. Default: True

forward(x)[source]