The trick is to remember that a neural network is a function approximator. A good deal of AI research is in the business of casting "intelligence" as a "function" so that you can pose a problem and figure out how to feed it data, ie., well-defined input and output. That's why different function approximators can be used for AI, such as decision trees, etc., not just neural networks. What to model, and how to model it, are orthogonal problems, both interesting in their own right. It just happens that NN are particularly good at handling high-dimensional inputs, which are necessary for perception tasks, as well as for handling large vocabulary language modeling, so they are doing rather well lately.
On the other hand, there are lots of places where "functions" are useful, that have nothing to do with "intelligence", but where actually writing the function with full fidelity can be difficult or intractable. These are also opportunities where learnable function approximation can provide some great benefit, provided you can figure out how to pose the problem such that data is available for the learning part.
A good example is in physically-based rendering, you have lots of complicated aspects of light transport for which we can model the physics quite well, but when you get into complicated reflections and scattering, etc., this can all be modeled by a complicated function called the BRDF [0]. Hand-writing a good BRDF is possible, and quite typical of high-end renderers in fact, but it's no surprise that there's been research in replacing it with a "neural BRDF" [e.g. 1].
That's just to give an example of a place where a single, very targeted and small (but complicated) part of a larger framework can stand to benefit from data-driven modeling, and a neural network can be one good way to do that. Another example is similar usage in computation fluid dynamics [2], where we can hand write a pretty good model, but to capture what is missing, it can be useful to have an approximator. The problem there is that having approximated the function well, it doesn't necessarily lead to better human understanding of the phenomenon. Discovering the true, sparse latent variables in a way that is interpretable, instead of a black box, is a useful step towards that. (Which I guess the current article is aiming at, but I haven't read it in full yet.) But sometimes all you want is results, as in the case of synthesizing a good controller. For example in CFD, if you can use the blackbox model to generate a good stabilizer for an ocean platform, you don't really care about the physics, as long as it's accurate enough to be trustworthy. So the utility of these methods, like most things, is relative to what your goals are.
On the other hand, there are lots of places where "functions" are useful, that have nothing to do with "intelligence", but where actually writing the function with full fidelity can be difficult or intractable. These are also opportunities where learnable function approximation can provide some great benefit, provided you can figure out how to pose the problem such that data is available for the learning part.
A good example is in physically-based rendering, you have lots of complicated aspects of light transport for which we can model the physics quite well, but when you get into complicated reflections and scattering, etc., this can all be modeled by a complicated function called the BRDF [0]. Hand-writing a good BRDF is possible, and quite typical of high-end renderers in fact, but it's no surprise that there's been research in replacing it with a "neural BRDF" [e.g. 1].
That's just to give an example of a place where a single, very targeted and small (but complicated) part of a larger framework can stand to benefit from data-driven modeling, and a neural network can be one good way to do that. Another example is similar usage in computation fluid dynamics [2], where we can hand write a pretty good model, but to capture what is missing, it can be useful to have an approximator. The problem there is that having approximated the function well, it doesn't necessarily lead to better human understanding of the phenomenon. Discovering the true, sparse latent variables in a way that is interpretable, instead of a black box, is a useful step towards that. (Which I guess the current article is aiming at, but I haven't read it in full yet.) But sometimes all you want is results, as in the case of synthesizing a good controller. For example in CFD, if you can use the blackbox model to generate a good stabilizer for an ocean platform, you don't really care about the physics, as long as it's accurate enough to be trustworthy. So the utility of these methods, like most things, is relative to what your goals are.
[0]: https://pbr-book.org/3ed-2018/Color_and_Radiometry/Surface_R...
[1]: https://arxiv.org/abs/2111.03797
[2]: https://github.com/loliverhennigh/Computational-Fluid-Dynami...