so...my use case for GPU lambdas is for inference. Say 500 inferences/day ~1 sec/inference, has to be real time as the output is user facing.
Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.
For some of my use cases the real-time response isn't essential, just needs one of containers for training. I figured I can just go about ECS, but thought if these were wrapped as lambdas it would just be a matter of toggling the base-image/lambda type.
Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.