The biggest gripe I have with using Python for rich command-line tool is the startup time. One of the first reason I like to write a nice command-line tool when I start a project, say foo, is to be able to do `foo --help` to quickly see and remember what the project can do (I have a very bad memory and doing this makes it possible for me to jump back faster to a project, even well documented. I can forget what I was doing in just a few days and so I add a lot of small commands).
In short running `foo --help` should be instant and if it loads all its modules to list the different sub-commands and their respective description it is really too slow.
A possibility is to cache some information (e.g. generate a text file or a small Python script).
Hm, it'd be pointless to argue this point (if you think it's slow then it's slow) but I've created many a Python CLI and have never noticed them being slow to start. Perhaps it's a case of importing certain modules globally instead of on a per-function basis?
See my answer to coldtea. Yes you can organize you're code to help making it load faster but I don't think it is fast enough. Maybe my terminal is slow, maybe my machine is slow (and maybe I'm overly sensitive to the problem) but the difference is telling.
>The biggest gripe I have with using Python for rich command-line tool is the startup time.
What startup time?
I even have a python script running on every shell prompt drawing (that checks mercurial on top of starting the python interpreter), and the latency is negligible.
> time python manage.py --help
real 0m0.473s
user 0m0.240s
sys 0m0.148s
Half of a second is very noticeable (and annoying). I had similar perception in my previous work.
You can try to only load enough to display the help text without really loading everything, but that doesn't work that well (you have to organize things differently, and `--help` requires to load a lot of stuff. `subcommand --help` needs less but it still has to see if the subcommand exists).
As a reference:
> time python -c 'print "hello"'
hello
real 0m0.041s
user 0m0.024s
sys 0m0.012s
> time echo hello
hello
real 0m0.000s
user 0m0.000s
sys 0m0.000s
Actually even the first (0.041s) is not instant, while the second one does (I mean as I perceive it visually).
> Half of a second is very noticeable (and annoying). I had similar perception in my previous work.
That's because it has to load django and the whole django project. Here's click:
> time python test.py --help
Usage: test.py [OPTIONS]
Options:
--help Show this message and exit.
python test.py --help 0.06s user 0.02s system 95% cpu 0.078 total
and argparse:
> time python test.py --help
usage: test.py [-h]
optional arguments:
-h, --help show this help message and exit
python test.py --help 0.03s user 0.02s system 93% cpu 0.054 total
reference:
> time python -c 'print "hello"'
hello
python -c 'print "hello"' 0.01s user 0.01s system 88% cpu 0.025 total
Calling a shell built-in isn't really fair -- you should at least compare to /bin/echo (on my system the relevant times are [edit: 0.001s] for echo, 0.007+0.004=0.011 for python -S -c "print 'hello'". Without dropping site-packages (without the -S) python jumps to 0.014+0.012=0.026 -- marginally above 200 ms which I suppose is a perceptible difference between instant, and not-quite-instant).
Sadly, both pypy and python3 are slower than python2.7. Also worth nothing that the sys time fluctates for me -- in other words when it is > 0.00s it doesn't appear to have anything to do with the command run.
Still, your second example is an order of magnitude faster than the example with Django. That's likely because Django loads the world on startup, parses all your model files, checks your database settings, etc.
I'm completely clueless about this, but would it be possible to have a persistent python interpreter always running in the background somehow, and submit the commands to it?
You can keep ipython kernel running in background if you want. Quick unscientific test showed that it didn't reduce ipython console startup time significantly. But it should be possible to make a leaner console to connect to the kernel.
edit: my test was flawed, with more accurate test there is almost a full second time difference:
If loading the rest of the code and initialization time really matters, yes. But then you're back to square one: you now have to create a CLI to access the server and that new CLI must load quickly. Which would also be the case if you just wrote it that way first, then loading the rest of the code once the command is determined.
And if you're in some project similar to Django, you would have first to query the server to know the possible subcommands...
So what you say makes sense when other parts of the code are slow to initialize and must be accessed frequently, not for the CLI itself.
In short running `foo --help` should be instant and if it loads all its modules to list the different sub-commands and their respective description it is really too slow.
A possibility is to cache some information (e.g. generate a text file or a small Python script).