We use linear and nonlinear functions in the Neural network that will lead us to a beautiful structure that we call Universal approximation.
But what if, We want to tune a NN to work as $sin(x)$, we need more perceptions compared to replacing one of the perceptions (linear sigmoid) from some sin(x) like perceptron.
The network must propagate the signal from this signal to a single sin(x) perception to get an approximation of sin(x).
Reference:
- #citation needed