AI

Batch Norm vs Layer Norm

Summary: Batch Norm -> normalizing each feature across batch. Layer Norm -> normalizing the values of each individual example independently. Deep Dive: Suppose we have 4 students (examples / rows)