It's interesting that Zen5's FPUs running in full 512bit wide mode doesn't actually seem to cause any trouble, but that lighting up the load store units does. I don't know enough about hardware-level design to know if this would be "expected".
The fully investigation in this article is really interesting, but the TL;DR is: Light up enough of the core, and frequencies will have to drop to maintain power envelope. The transition period is done very smartly, but it still exists - but as opposed to the old intel avx512 cores that got endless (deserved?) bad press for their transition behavior, this is more or less seamless.
On the L/S unit impact: data movement is expensive, computation is cheap (relatively).
In "Computer Architecture, A Quantitative Approach" there are numbers for the now old TSMC 45nm process: A 32 bits FP multiplication takes 3.7 pJ, and a 32 bits SRAM read from an 8 kB SRAM takes 5 pJ. This is a basic SRAM, not a cache with its tag comparison and LRU logic (more expansive).
Then I have some 2015 numbers for Intel 22nm process, old too. A 64 bits FP multiplication takes 6.4 pJ, a 64 bits read/write from a small 8 kB SRAM 4.2 pJ, and from a larger 256 kB SRAM 16.7 pJ. Basic SRAM here too, not a more expansive cache.
The cost of a multiplication is quadratic, and it should be more linear for access, so the computation cost in the second example is much heavier (compare the mantissa sizes, that's what is multiplied).
The trend gets even worse with more advanced processes. Data movement is usually what matters the most now, expect for workloads with very high arithmetic intensity where computation will dominate (in practice: large enough matrix multiplications).
Appreciate the detail! That explains a lot of what is going on.. It also dovetails with some interesting facts I remember reading about the relative power consumption for the zen cores versus the infinity fabric connecting them - The percentage of package power usage simply from running the fabric interconnect was shocking.
Right, but a SIMD single precision mul is linear (or even sub linear) relative to it's scalar cousin. So a 16x32, 512-bit MUL won't be even 16x the cost of a scalar mul, the decoder has to do only the same amount of work for example.
The calculations within each unit may be, true, but routing and data transfer is probably the biggest limiting factor on a modern chip. It should be clear that placing 16x units of non-trivial size means that the average will likely be further away from the data source than a single unit, and transmitting data over distances can have greater-than-linear increasing costs (not just resistance/capacitance losses, but to hit timing targets you need faster switching, which means higher voltages etc.)
Both Intel and AMD to some extent separate the vector ALUs and the register file in 128-bit (or 256-bit?) lanes, across which arithmetic ops won't need to cross at all. Of course loads/stores/shuffles still need to though, making this point somewhat moot.
AFAIK you have to think about how many different 512b paths are being driven when this happens, like each cycle in the steady-state case is simultaneously (in the case where you can do two vfmadd132ps per cycle):
- Capturing 2x512b from the L1D cache
- Sending 2x512b to the vector register file
- Capturing 4x512b values from the vector register file
- Actually multiplying 4x512b values
- Sending 2x512b results to the vector register file
.. and probably more?? That's already like 14*512 wires [switching constantly at 5Ghz!!], and there are probably even more intermediate stages?
> but as opposed to the old intel avx512 cores that got endless (deserved?) bad press for their transition behavior, this is more or less seamless.
The problem with Intel was, the AVX frequencies were secrets. They were never disclosed in later cores where power envelope got tight, and using AVX-512 killed performance throughout the core. This meant that if there was a core using AVX-512, any other cores in the same socket throttled down due to thermal load and power cap on the core. This led to every process on the same socket to suffer. Which is a big no-no for cloud or HPC workloads where nodes are shared by many users.
Secrecy and downplaying of this effect made Intel's AVX-512 frequency and behavior infamous.
Oh, doing your own benchmarks on your own hardware which you paid for and releasing the results to the public was verboten, btw.
To be clear, the problem with the Skylake implementation was that triggering AVX-512 would downclock the entirety of the CPU. It didn’t do anything smart, it was fairly binary.
This AMD implementation instead seems to be better optimized and plug into the normal thermal operations of the CPU for better scaling.
Reading the section under "Load Another FP Pipe?" I'm coming away with the impression that it's not the LSU but rather total overall load that causes trouble. While that section is focused on transition time, the end steady state is also slower…
I haven’t read the article yet, but back when I tried to get to over 100 GB/s IO rate from a bunch of SSDs on Zen4 (just fio direct IO workload without doing anything with the data), I ended up disabling Core Boost states (or maybe something else in BIOS too), to give more thermal allowance for the IO hub on the chip. As RAM load/store traffic goes through the IO hub too, maybe that’s it?
I don't think these things are related, this is talking about the LSU right inside the core. I'd also expect oscillations if there were a thermal problem like you're describing, i.e. core clocks up when IO hub delivers data, IO hub stalls, causes core to stall as well, IO hub can run again delivering data, repeat from beginning.
(Then again, boost clocks are an intentional oscillation anyway…)
Ok, I just read through the article. As I understand, their tests were designed to run entirely on data on the local cores' cache? I only see L1d mentioned there.
Yes, that's my understanding of "Zen 5 also doubles L1D load bandwidth, and I’m exercising that by having each FMA instruction source an input from the data cache." Also, considering the author's other work, I'm pretty sure they can isolate load-store performance from cache performance from memory interface performance.
It seems even more interesting than the power envelope. It looks like the core is limited by the ability of the power supply to ramp up. So the dispatch rate drops momentarily and then goes back up to allow power delivery to catch up.
The fully investigation in this article is really interesting, but the TL;DR is: Light up enough of the core, and frequencies will have to drop to maintain power envelope. The transition period is done very smartly, but it still exists - but as opposed to the old intel avx512 cores that got endless (deserved?) bad press for their transition behavior, this is more or less seamless.