But seriously, my actual question is whether an ideal resistor network can compute nonlinear functions since the individual resistors are linear in their input, ignoring possible nonlinear effects like temperature dependence of the conductivity of the resistors
There is a separate interesting question of whether you could exploit the nonlinearity of a network of non-ideal resistors in practice. Given that tom7 was able to use floating point inaccuracy as a source of nonlinearity for ML, I’m guessing the answer is probably “yes, but now your system is incredibly sensitive to stuff like ambient temperature”.
Just spitballing: but I'm thinking about how slide rules are linear but can do calculations that, I believe?, are not limited to being linear. Due I imagine to logarithmic rulings and that logarithms can be added/subtracted in a linear fashion.
That's more or less what we have now. Slide rules and computers are linear systems which can compute nonlinear functions. The problem is that computing anything takes time and must be a linear set of instructions executed in series (multiplied by many parallel cores).
Using an analog approach could be vastly more efficient as operations are inherently parallel. You can fire off every neuron in a layer simultaneously and produce a result within nanoseconds, for basically any number of neurons. You could probably do the entire network as a single atomic operation, but that's a bit beyond my knowledge of neural networks
However, in a real resistor, resistivity varies with temperature, and current through the resistor produces heat. Therefore, real resistors are in fact nonlinear circuit devices. Technically speaking. In reality, whenever electrical engineers are confronted with this nonlinearity, it's a bug that must be smashed and not a feature to exploit.
I love it when things are linear to first order.
>.>