GP is indeed wrong. We do use fundamental principles and philosophical positions to figure out what's wrong with our math when the results are wrong or nonsensical. These are in fact part of even an undergraduate physicist's training to be able to tell if some model is violating energy conservation or the second law of thermodynamics or the speed of light etc.

However, if you count the number of times a physicist uses math to resolve conceptual confusion and count when they use physical principles to fix the math, you will find the former many times larger than the latter.