While you would never do this kind of training on your user's machines (which takes multiple weeks even with a powerful GPU), you should be able to apply the trained model to a single photo nearly instantaneously. So the real roadblock is mostly that they don't appear to have a included a completely pre-trained model with this release, and it will take you as a developer a lot of GPU time to train one. But your users would not necessarily have a problem captioning images on their machines.
I hadn't considered that (this is really out of my depth). Any ideas on what the actual size of a trained model would be to distribute? Taking 150G on the user's hard drive is out as well, probably.
Depends on the model and dataset, inceptionv3 trained on imagenet is about 150mb but you can quantise the weights to 8bit and prune it much smaller without affecting perf much
