Three ReLU networks with different widths learning noisy step(x)

Three deep ReLU networks learn sin(10x) + 0.5sin(3x)

ReLU Isn’t Always the Best Choice

The Hidden Problem with ReLU (And How Leaky ReLU Fixes It)

Three ReLU networks with different widths learning noisy sin

ReLU vs Leaky ReLU vs ELU Explained Simply

Three ReLU networks with different widths learning noisy parabola

ReLU Fixed One Problem… Created Another 😬