If cache size was the answer then Intel would have already stuck more on their CPUs, and AMD's 32MB on Zen3 would be carrying far more weight against the M1 than it is.
Bigger caches mean higher latency caches, it's not strictly a matter of bigger is better.
It’s an L2, amd zen has 2x16mb L3s, with a wacky point of unification. L3 is muuuuch slower than L2.
Apple’s L2 is huge and is the PoU for all the cores. I wouldn’t be surprised if it has direct routing to all the L1s since they are so small. 5nm doesn’t hurt.
Bigger caches mean higher latency caches, it's not strictly a matter of bigger is better.