A partial answer is that big fully connected neural networks _are_ pretty much untrainable. Neural networks only became successful once programmers started constraining the space they were optimizing over pretty radically (like requiring convolutional layers if you know you are trying to detect something local).