gsnn.gsnn.models.ChannelEMANorm

Channel-wise Exponential Moving Average Normalization.

ChannelEMANorm maintains running statistics using exponential moving averages for each individual channel independently, making it very robust for small and variable batch sizes.

Classes

ChannelEMANorm(*args, **kwargs)

Applies normalization per individual channel using exponential moving averages.
class gsnn.gsnn.models.ChannelEMANorm.ChannelEMANorm(*args: Any, **kwargs: Any)[source]

Bases: Module

Applies normalization per individual channel using exponential moving averages.

This normalization maintains running statistics for each channel independently but doesn’t use current batch statistics for normalization, making it very stable for small batch sizes.

Parameters:

  • num_channels (int) – Number of channels to normalize.

  • eps (float) – Small value to avoid division by zero. Default: 1e-5

  • momentum (float) – Momentum for updating running statistics. Default: 0.1

  • affine (bool) – If True, applies learnable scale and bias per channel. Default: True

  • track_running_stats (bool) – If True, maintains running statistics. Default: True

forward(x)[source]