Stack neurons, build curves.
A neuron with no activation is linear regression. Each ReLU adds one kink. Stack enough kinks and you can fit anything.
Each dot is one hidden unit · position = its kink · color = sign of w1 · size = |v| · edge thickness in the diagram = |weight|
Width = 0 — a straight line. This is linear regression.