Most of the people working at Ars are the exact same people who have been working there for the better part of their entire existence (source: me) Most of them _are_ experts in their fields, and most are vastly more qualified in their fields than pretty much anyone else publishing online (both now and 20 years ago).
It seems that _certain kinds of individuals_ have had rose-colored glasses on about pretty much everything online, but for Ars especially for some reason.
They detest change in a publication that covers the reality of actual life and technology, rather that commit suicide and stay covering stuff the same way they did in 1997—which 8 people total want to read (and not pay for, by the way).
Ars has been operating at an exceptionally high level for their entire history and have outlasted many other flashes-in-the-pan which are now relegated to the dust bin of history.
Is Ken still actively involved? He seems to appear to clarify something and then disappear into the background until the next major change (I expect the article about this to be bylined to him, as is appropriate).
How are the runs isolated? Can I run multiple parallel sessions on the same repo? What about security / sandboxing to avoid exfiltration of data etc?
Thanks!
Yeah, we have parallel agent functionality, sort of like conductor. This allows you to create worktrees for your repo and run any number of chats per worktree. On your local machine, we don't have any unique sandboxing capabilities, but we reuse your sandboxing settings from Claude or Codex if you have them set. The cloud sandboxing is more isolated, but still has access to the internet.
Yes and no I'd say.
It's still the case that now only by iterating and testing things with the AI you get closer to an actually good solution.
So up front big spec will also not work so well.
The only exception maybe if you already have a very clear understanding and existing tests (like what they did with the Claude's building the rust c compiler to compile the Linux kernel)
Thanks for sharing this! I'm going to put this on my list to play around with. I'm not really an expert in this tech, I come from the audio background, but recently was playing around with streaming Speech-to-Text (using Whisper) / Text-to-Speech (using Kokoro at the time) on a local machine.
The most challenging part in my build was tuning the inference batch sizing here. I was able to get it working well for Speech-to-Text down to batch sizes of 200ms. I even implement a basic local agreement algorithm and it was still very fast (inferencing time, I think, was around 10-20ms?). You're basically limited by the minimum batch size, NOT inference time. Maybe that's a missing "secret sauce" suggested in the original post?
In the use case listed above, the TTS probably isn't a bottleneck as long as OP can generate tokens quickly.
All this being said a wrapped model like this that is able to handle hand-offs between these parts of the process sounds really useful and I'll definitely be interested in seeing how it performs.
Let me know if you guys play with this and find success.
and the "Customer Service - Banking" scenario claims that it demos "accent control" and the prompt gives the agent a definitely non-indian name, yet the agents sounds 100% Indian - I found that hilarious but also isn't it a bad example given they are claiming accent control as a feature?
You mentioned needing 40k tiles and renting a H100 for 3$/hour at 200tiles/hour, so am I right to assume that you spend 200*3=600$ for running the inference?
That also means letting it run 25 nights a 8 hours or so?
Yup back of the napkin is probably about there - also spent a fair bit on the oxen.ai fine-tuning service (worth every penny)... paint ain't free, so to speak
It seems Gemini couples app activity (I can see my chat history) with AI training.
This is horrible...
Any way around it to make Gemini usable without my data being used for training?
reply