True, it can't compete with AWS/GCP/Azure if you're large scale. But most of us are not large scale, we just need a no frills experience instead of dealing with 27 nested panels just to spin up a VM.
And they're as deterministic as as the underlying thing they're abstracting... which is kinda what makes an abstraction an abstraction.
I get that people love saying LLMs are just compilers from human language to $OUTPUT_FORMAT but... they simply are not except in a stretchy metaphorical sense.
That's only true if you reduce the definition of "compiler" to a narrow `f = In -> Out`. But that is _not_ a compiler. We have a word for that: function. And in LLM's case an impure one.
> this project, even if somewhat spaghettified, will likely take orders of magnitude less time to perfect than it would for someone to create the whole thing from scratch without AI
That's a big leap of faith and... kinda contradicts the article as I understood it.
My experience is entirely opposite (and matches my understanding of the article): vibing from the start makes you take orders of magnitude more time to perfect. AI is a multiplier as an assistant, but a divisor as an engineer.
Both of these are not really the right way to use AI to code with. There are two basic ways to code with AI that work:
1. Autocomplete. Pretty simple; you only accept auto-completes you actually want, as you manually write code.
2. Software engineering design and implementation workflow. The AI makes a plan, with tasks. It commits those plans to files. It starts sub-agents to tackle the tasks. The subagents create tests to validate the code, then writes code to pass the tests. The subagents finish their tasks, and the AI agent does a review of the work to see if it's accurate. Multiple passes find more bugs and fix them in a loop, until there is nothing left to fix.
I'm amazed that nobody thinks the latter is a real thing that works, when Claude fucking Code has been produced this way for like 6 months. There's tens of thousands of people using this completely vibe-coded software. It's not a hoax.
I have worked at companies from startups to fortune 500. They all have garbage code. Who cares? It works anyway. The world is held together with duct tape, and it's unreasonably effective. I don't believe "code quality" can be measured by how it looks. The only meaningful measure of its quality is whether it runs and solves a user's problem.
Get the best programmer in the world. Have them write the most perfect source code in the world. In 10 years, it has to be completely rewritten. Why? The designer chose some advanced design that is conceptually superior, but did not survive the normal and constant churn of advancing technology. Compare that to some junior sysadmin writing a solution in Perl 5.x. It works 30 years later. Everyone would say the Perl solution was of inferior quality, yet it provides 3x more value.
I hear you about "it just works" mattering infinitely more than some arbitrary code quality metric
but I'm not judging Claude Code by how it looks. I kinda like the aesthetics. I'm talking about how slow, resource hungry and finnicky/flickery it is. it's objectively sloppy
> when Claude fucking Code has been produced this way for like 6 months
And people can look at the results (illegally) because that whole bunch of code has been leaked. Let's just say it's not looking good. These are the folks who actually made and trained Claude to begin with, they know the model more than anyone else, and the code is still absolute garbage tier by sensible human-written code quality standards.
Human code quality standards are built around the knowledge that humans prefer polished products that work consistently. You can get away without code quality in the short term, especially if you have no real competitors - to a lot of people, there just aren't any models other than Anthropic's which are particularly useful for software development. But in the long term it gets you into a poor quality trap that's often impossible to escape without starting over from scratch.
(Anthropic, of course, believes that advances in AI capability over the next few years will so radically reshape society that there's no point worrying about the long term.)
How is the time sharing handled? I assume if I submit a unit of work it will load to VRAM and then run (sharing time? how many work units can run in parallel?)
How large is a full context window in MiB and how long does it take to load the buffer? I.e. how many seconds should I expect my worst case wait time to take until I get my first token?
vLLM handles GPU scheduling, not sllm. The model weights stay resident in VRAM permanently so there's no loading/unloading per request. vLLM uses continuous batching, so incoming requests are dynamically added to the running batch every decode step and the GPU is always working on multiple requests simultaneously. There is no "load to VRAM and run" per request; it's more like joining an already-running batch.
TTFT is under 2 seconds average. Worst case is 10-30s.
not original author but batching is one very important trick to make inference efficient, you can reasonably do tens to low hundreds in parallel (depending on model size and gpu size) with very little performance overhead
I've known SSH certs for a while but never went through the effort of migrating away from keys. I'm very frustrated about manually managing my SSH keys across my different servers and devices though.
I assume you gathered a lot of thoughts over these 15 years.
A big problem I have with ssh carts is that they are not universally supported. For me, there is always some device or daemon (for example tinyssh in the initramfs of my gaming pc so that I can unlock it remotely) that only works with “plain old ssh keys”. And if I have to distribute and sync my keys onto a few hosts anyway, it takes away the benefits.
Adding to this: while certs are indeed well-supported by OpenSSH, it's not always the SSH daemon used on alternate or embedded platforms.
For example, OpenWRT used Dropbear [1] instead, which does not support certs. Also, Java programs that implement SSH stuff, like Jenkins, may be doing so using Apache Mina [2] which, though the underlying library supports certs, it is buggy [3] and requires the application to add the UX to also support it.
You can just replace dropbear with openssh on OpenWRT. That was one of the first things I did, since DropBear also doesn't support hardware backed (sk) keys. Just move it to 2222 and disable the service.
I reenabled DB on that alt port when I did the recent major update, just in case, but it wasn't necessary. After the upgrade, OpenSSH was alive and ready.
Might actually be a positive instead of a negative. Gaming use-cases should have not any effect on security policies, these should be as separate as possible, different auth mechanisms for your gaming stuff and your professional stuff ensures nothing gets mixed.
Hah? It being my gaming machine has nothing to do with the problem. It’s also my FPGA development machine, though it gets used less for that. It only happens to be the only Linux workstation in my home (the others are Macs or OpenBSD).
If your use case is such that you are frustrated about managing keys, host or user keys, then yes it does sound like SSH certs would help you. E.g. when you have many users, servers, or high enough cartesian product of the two.
In environment where they don't cause frustration they're not worth it.
Not really more to it than that, from my point of view.
It's no different compared to regular SSH private keys. You need to protect it from compromise.
However, it provides you an additional layer of protection, because it does not need to be on the critical path for every SSH connection. My CA is a Nitrokey HSM, for example. I issue myself temporary certs that are valid only for 6 hours for ephemeral private keys.
Exactly. We'd had discussions about building https://Userify.com (plug!) around SSH certificates, but elected to go with keys instead, because Userify delivers most of the good things around certificates without the jank and insecurity.
It's not that certificates themselves are insecure themselves, it's that the workflows (as the parent points out) are awful. We might still add some automation around that (and I think I saw some competitor tooling out there if you're committed to that path) but I personally feel like it's an answer to the wrong question.
I am keeping an eye on the new (and alpha) Authentik agent which will allow idp based ssh logins. There's also SSSD already supported but it requires glibc (due to needing NSS) meaning it's not available on Alpine.
Can't really speak to the point of the guy you're replying to, but the FreeIPA implementation via SSSD does more than just Kerberos tickets. Actually, I think the Kerberos based stuff as it relates to SSH is GSSAPI as part of sshd itself and has little to do with sssd, though I could be wrong.
That said...
If I'm remembering things correctly (and it's been a looong while since I've played with this), FreeIPA's client configures sshd with an AuthorizedKeysCommand that executes a program that queries sssd for the list of authorized keys for a given user. Sssd then uses a plugin to query the LDAP server @ FreeIPA to get the list of keys.
There's also SSHFP (I think) records in DNS if you're using FreeIPA's DNS servers. These provide the host keys for servers for your ssh client to check against. Not sure if that's integrated into ssh itself or something else -- I can't remember how it's implemented offhand -- but it's fairly nifty since you never see the TOFU prompt (or it would be if DNS was actually secure, anyway).
Yes, FreeIPA is Kerberos+LDAP+X.509 CA, and GSSAPI is in OpenSSH (normally with the key exchange patch). SSSD is a local mechanism, not network authentication. I mentioned authorized keys distribution mechanisms elsewhere, but I was thinking authentication (c.f. OIDC), not authorization.
Yes, but its authN components only act locally, and PAM is optional for sshd. It can/does call out to network services like Kerberos/LDAP given a password, of course, but I was thinking of network authN connected directly with OIDC somehow, for which I don't know a mechanism in vanilla OpenSSH. (I don't know what Authentik does for this -- I could imagine it's behind the scenes somehow.) I should probably look it up sometime.
My understanding is since it's an agent running on the target, possibilities will be quite extensive. But it is relatively new and there is no stable release of it yet.
It depends on what you want to do. CA certs are easy to manage, you just put the CA key instead of the SSH public key in authorized_keys.
They also provide a way to get hardware-backed security without messing with SSH agent forwarding and crappy USB security devices. You can use an HSM to issue a temporary certificate for your (possibly temporary) public key and use it as normal. The certificate can be valid for just 1 hour, enough to not worry about it leaking.
Yes. Caveat: It might not really be worth it if all your infrastructure is managed by these newfangled infrastructure-as-code-things that are quick to roll out (OpenShift/OKD, Talos, etc.) and you have only one repo to change SSH keys (single cluster or single repo for all clusters).
There are some serious security benefits for larger organizations but it does not sound as if you are part of one.
> It's easy to say you're faster if you don't actually support everything or maybe even made a mistake.
> I don't see any tests so I wouldn't use this.
---
> the repo has 5 commits and the first one is from 3 hours ago. "I've been working on" is probably more accurately "this morning I asked an ai to write this for me".
---
> The single-threaded design of redis was specifically so that operations are ordered sequentially, so that the WAL-like log would be replayable and you'd get the exact same state as when shutting down the server.
> Did you take any measures to ensure a sequential order of executed commands?
TCP or UDP are orthogonal to this, so the original comment feels like a non sequitur. These streams are not network streams and could be a file, chunks of procedural audio, or whatever.
I agree, the stream concept should be (and is) very general and ideally cover all these cases - any “bytes producing” source.
I was trying to be open minded about that and conceive a stream API over a UDP socket. It’d work IMHO, but be a little odd compared to an event-like API.
True, it can't compete with AWS/GCP/Azure if you're large scale. But most of us are not large scale, we just need a no frills experience instead of dealing with 27 nested panels just to spin up a VM.
reply