This looks super cool! Looking forward to trying it.
I think a killer feature of these gpu-plotting libraries would be if they could take torch/jax cuda arrays directly and not require a (slow) transfer over cpu.
Thanks! That is a great question and one that I've we've been battling with as well. As far as we know, this is not possible due to the way different contexts are set up on the GPU https://github.com/pygfx/pygfx/issues/510
I've been looking into this issue with Datoviz [1] following a user request. It turns out there may be a way to achieve it using Vulkan [2] (which Datoviz is based on) and CuPy's UnownedMemory [3]. I wrote a simple proof of concept using only Vulkan and CuPy.
I'm now working on a way for users to wrap a Datoviz GPU buffer as a CuPy array that directly references the Datoviz-managed GPU memory. This should, in principle, enable efficient GPU-based array operations on GPU data without any transfers.
This looks cools thanks! Makes me wonder if there's any way to do that with WGPU if WGPU is interfacing with Vulkan, probably not easy if possible I"m guessing.
WGPU has security protections since it's designed for the browser so I'm guessing it's impossible.
Wow. So are you saying that you can have some array on the GPU that you setup with python via CuPy, then you call to the webbrowser and give it the pointer address for that GPU array, and the browser through WASM/WebGPU can access that same array? That sounds like a huge browser security hole.
Yea the security issue is why I'm pretty sure you can't do it on WGPU, but Vulkan and cupy can fully run locally so it doesn't have the same security concern.
Not sure about the memory transfer bottleneck and potential mitigations. But out of interest, how insurmountable would it be to 'retool' fastplotlib to use JAX acceleration instead of wgpu?
I think a killer feature of these gpu-plotting libraries would be if they could take torch/jax cuda arrays directly and not require a (slow) transfer over cpu.