We use linear and nonlinear functions in the Neural network that will lead us to a beautiful structure that we call Universal approximation.

But what if, We want to tune a NN to work as $sin(x)$, we need more perceptions compared to replacing one of the perceptions (linear sigmoid) from some sin(x) like perceptron.

The network must propagate the signal from this signal to a single sin(x) perception to get an approximation of sin(x).

Reference:

  • #citation needed