Chrony over NTP is capable of incredible accuracy, as shown in the post. Most users who think they need PTP actually just need Chrony and high quality NICs.
Chrony is also much better software than any of the PTP daemons I tested a few years ago (for an onboard autonomous vehicle system).
NTP fundamentally cannot reach the same accuracy as PTP because Ethernet switches introduce jitter due to queueing delays and can report that in PTP but not NTP.
chrony can be configured to encapsulate NTP messages in PTP messages (NTP over PTP) in order to get the delay corrections from switches working as one-step PTP transparent clocks. The current NTPv5 draft specifies an NTP-specific correction field, which switches could support in future if there was a demand for it.
The switches could also implement a proper HW-timestamping NTP server and client to provide an equivalent to a PTP boundary clock.
PTP was based on a broadcast/multicast model to reduce the message rate in order to simplify and reduce the cost of HW support. But that is no longer a concern with modern HW that can timestamp packets at very high rates, so the simpler unicast protocols like NTP and client-server PTP (CSPTP) currently developed by IEEE might be preferable to classic PTP for better security and other advantages.
Correct me if I am wrong, but wouldn't that be true only for testing accross comparable hardware? Would that be true in scenarios like the one that the author describes, where he uses 3 different systems (threadriper cpu, raspberrypi, and LeoNTP GPS-backed NTP server) and architectures?
Yeah, I think "PRAGMA optimize" here on startup is basically a no-op and only costs microseconds but I should just remove it.
Seems like the ideal thing is to spawn a goroutine and run optimize periodically. Many Go daemons I run will have uptime measured in months so running on shutdown isn't super useful there.
Another one I might add is running `PRAGMA wal_checkpoint(PASSIVE)` periodically from a goroutine, which is something I've done in some of my daemons. Because otherwise a random write suffers from increased latency due to having to checkpoint.
Probably makes sense to open a dedicated read/write connection just for running `PRAGMA wal_checkpoint(PASSIVE)` at some regular interval to keep ahead of any autocheckpointing?
> Seems like the ideal thing is to spawn a goroutine and run optimize periodically. Many Go daemons I run will have uptime measured in months so running on shutdown isn't super useful there.
Connections don't necessarily need to be kept forever. Between SetConnMaxLifetime/SetConnMaxIdleTime that's one way of periodically optimizing from “warm” connections. But doing it on a timer from a fresh goroutine is another.
That's a good point too. It seems a bit weird to me to garbage collect connections that I know I'll just recreate in most cases, especially when they're so cheap and limited in number.
Cycling connections may cover for certain classes of bugs but I dislike doing things for this reason, but maybe there are others. I'll put some thinking/testing into it at some point.
Thanks for pointing this out. I don't think it's something this little helper library should try to do. This kind of thing belongs in whatever database abstraction you're building/using, if any. It's also a little trickier than it seems at first glance (RETURNING, etc).
And for many no/low concurrency situations there's no/little benefit.
Added this example to the README.
rwdb, err := sqlitebp.OpenReadWriteCreate("app.db")
if err != nil {
log.Fatal(err)
}
rwdb.SetMaxOpenConns(1)
rwdb.SetMaxIdleConns(1)
// Read-only with default adaptive pool size (2-8 based on GOMAXPROCS)
rodb, err := sqlitebp.OpenReadOnly("app.db")
if err != nil {
log.Fatal(err)
}
That post is quite nitpicky, pointing to edge cases and old version behavior. The essence of it is that writing reliable Bash scripts requires significant domain knowledge, which is true.
Bash scripts are great if you've been doing it for a very long time or you're doing something simple and small (<50 LOC). If it's complicated or large, you should just write it in a proper programming language anyway.
Yeah, that's a good point. It's less about LOC than complexity (data structures, complicated string parsing, handling binary data, etc). And that's actually the advice I follow myself.
Some longish Bash scripts are very maintainable and reliable, because they're still quite regular and simple.
Yeah! I feel like long args very strongly fits with the overall idea of this blog post: when you are running a script you care about different things and want different behaviours than when you’re working interactively.
The short args are great for when you’re typing into a terminal. But when you’re writing and saving a script, be descriptive! One of the most irritating things about bash (and Linux, I guess?) is how magical all the -FEU -lh 3 -B 1 incantations are. They give off a vibe I call “nerd snobbish” where it’s a sort of “oh you don’t know what that all means? How fretful!”
Too damn verbose and you're assuming bash is at /bin. This will cause problems in nix and other environments where bash should be found on the path instead.
Always, always unless the script is guaranteed to not be portable to other platforms:
There are real environments where it's at /bin/env.
Running a script with
bash script.sh
Works on any system with bash in the path, doesn't require that it be executable, and is usually the ideal way to execute them.
I don't really believe in trying to write portable Bash scripts. There's a lot more to it than just getting the shebang right. Systems and commands tend to have subtle differences and flags are often incompatible. Lots of branching logic makes them hideous quickly.
I prefer to write a script for each platform or just write a small portable Go program and compile it for each platform.
I'm almost always writing Bash scripts for Linux that run in Debian-based containers, where using /bin/bash is entirely correct and safe.
Regarding verbosity, most people come to appreciate a little verbosity in exchange for clarity with more experience. Clever and compact code is almost never what you want in production code.
yeah, I agree with this.. all the work that goes into making it portable makes it much harder to read the script as well in most cases.
What I typically do is also have a version check for bash, just to make sure that nothing insane happens.
I also tend to litter asserts all about the beginning of the code in my compiled programs to ensure my environment is sane. because if I make an assumption, I don’t want it to land on someone’s head unexpectedly. (especially my own)
Bluesky is built on atproto, which is designed to be "locked open" in a way that can't be rescinded. That was a core design constraint.
VCs funded Netscape which did more than any other company to launch the web and they made a lot of money without having to destroy the ideals of the web.
The PDSes (personal data servers) can be independantly hosted, but Bluesky itself indexes and presents the messages these servers contain in their social-app. Bluesky also maintains the directory of these servers.
Yes, and running these things is prohibitively difficult such as I've only witnessed two full index attempts and no alternative plc directories.
Bluesky should make these easier so your average Linux admin can attempt to host the full stack, as opposed to only being able to host a PDS. This would eliminate the criticism about Bluesky's design.
AT Proto isn’t really supposed to be about individuals self hosting the whole thing. The system is supposed to be global, distributed, and shared, not isolated to one person self-hosting the whole stack. One person should be able to host a resource and connect it to the network (esp to host their own data). It’s just a different design goal compared to full-stack self-hosting. Fundamentally speaking, you can’t run Twitter at scale on a home laptop. But if lots of people band together their resources by hosting distributed microservices, they can self host it together. That’s what AT Proto is trying to solve
If it was more DNSy it could be hierarchal. I should be able to run my own stack and treat is like an RSS feed, subscribed to the various domains and individuals I'm interested in. Larger (but not huge) players could create plc directories of users and various aggregated feeds. Why have the expectation to host the global firehouse just because you want to customize how relays and appviews work?
If you want a service which indexes every post in the public network, including from folks you don't follow, that is just going to require resources. I think $200/month for a full-network index (as zepplin does) is very reasonable and approachable for organized groups without external funding. Many Mastodon instances cost more than that, and provide a must smaller scope of indexing.
If you want a small scaled down setup for just a small community, which still interoperates with the full network but doesn't have a complete network, there are setups like AppViewLite, which can run on, eg, an old laptop at home: https://github.com/alnkesq/AppViewLite
Personally, I don't think individualist self-hosting is a necessary or helpful goal for indexing the network. Most humans are not interested in spending the time or learning the skills to do this, even if it was as easy as setting up a self-hosted blog with RSS. I think small collectives (orgs, coops, communities, neighborhoods, companies, etc) exist and can fill this role.
Regardless, this is moving the discussion, which was about whether it was possible to decentralize each component the network, not whether it was pragmatic for individuals to self-host the whole thing.
> I think $200/month for a full-network index (as zeppelin does) is very reasonable and approachable for organized groups without external funding.
I didn't know about these recent attempts, they're impressive for sure. However they write[1] about zeppelin:
"The cost to run this is about US $200/mo, primarily due to the 16 terabytes of storage it currrently uses"
So when you here give that $200/mo cost as a price point for "organized groups", you are forecasting that the cost of storage will go down as fast as the BlueSky data size grows? At what rate right now is the data size growing? Because the last numbers I saw were something like 2TB, so it being already 16TB sounds like $200/mo is not going to be enough very soon.
I agree there's a lot of room for improvement in making it easier.
But certain things like full-network relays/app views just have inherent bandwidth/storage/compute costs associated with them but it's definitely something a non-profit (like Internet Archive) could easily afford to do.
The PLC service could likely be hosted for ~$40/mo.
I believe this isn't as much of a problem as it appears to be at first glance because of the scale of social apps like Bluesky.
For example, Wikipedia generates >$180M/yr just by running ads for itself requesting donations. Requesting donations is the least effective monetizing strategy and yet it still works because of scale.
Donations would probably work but Bluesky has additional options. They could create a premium app for power users that just adds nice-to-have features (which may cost real money to provide and maintain), they can resell domain names, they can sell merch, etc.
Bluesky doesn't need to generate billions of dollars to be highly sustainable and profitable. It was built and scaled with fewer than 20 full time employees.
The most important and most difficult part is getting to sufficient scale, and that's mostly a matter of just making the app even better than it is today.
1. I believe they actually could generate (low) billions of dollars without compromising at all, if they manage to reach true mainstream scale (>1 billion MAUs)
2. I really don't care if the investors/shareholders are disappointed as long as the PBC's mission is fulfilled. Also their control is relatively limited.
Maybe I should have written added this:
Disclaimer: I am a shareholder in Bluesky Social, PBC (former employee)
Yeah, I'd love to see growth improve but Bluesky is already in an exclusive club of social apps that have "broken through" in some significant way. It's not going anywhere.
And it's the only open network built on an open protocol to ever do so.
the open network aspect is extremely cool, but it's hard to take seriously considering how difficult it seems to be to set up an alternate host. I do love the idea that you can just build an entirely new UX and bring the social graph for free!
Yeah, that’s more of a hope about the future. It’s up to other people to build it.
I’ve seen posts about some poorly-publicized, proof-of-concept alternative implementations that would probably fall over if they got real attention, but I think that shows that it’s not a problem with the protocol itself.
Good enough, as far as I’m concerned. It’s just about posting comments on the Internet, not bank accounts. If something went fatally wrong, we would move again, just like we moved off previous social networks.
It’s also possible to run an annual event that makes a profit. Which might be something a social network could figure out.
The first club I belonged to as a teenager worked this way. In lieu of high membership dues there was volunteer time spent helping out at or before the event. I was surprised as an adult to learn that some events lose money or only break even.
Their largest operating income thus-far has been from selling two batches of t-shirts. Alongside some minor affiliate revenue from sending people to a domain registrar which they don't advertise anywhere prominent. (all handles are domain names, getting one that doesn't end in .bsky.social means getting one elsewhere)
>For example, Wikipedia generates >$180M/yr just by running ads for itself requesting donations. Requesting donations is the least effective monetizing strategy and yet it still works because of scale.
Plus most of that goes towards other wikimedia projects, wikipedia itself despite being huge and used by most of the internet costs a fraction of that to run yearly.
> plow may not be the best tool that exists but it does make concurrent HTTP requests and generate metrics for them successfully.
HTTP load testing is a problem area that is much more subtle than it seems. I've no doubt that plow does what you're saying here, but, without any tests whatsoever, I have serious doubts that it does so correctly, particularly if/when the load test starts bumping up against any of the numerous bottlenecks that can affect results and their measurements.
> The writes returned 3xx because the handler returns a redirect, so this is expected.
Yeah, but, unless `plow` actually follows that redirect, it's not really measuring the actual end-to-end latency, and further the guestbook.cgi returns 301 See Other for both valid requests (that performed a write) and invalid requests (that didn't).
You have a fair point about plow. I've used it enough to know that it basically works at spamming HTTP requests at a specified concurrency. When I'm doing something where I want better accuracy I've tended to use K6 and Vegeta.
> Yeah, but, unless `plow` actually follows that redirect, it's not really measuring the actual end-to-end latency...
Not in this case, since the purpose here was to measure the POST (write) not the subsequent GET (read) that a browser would do after the redirect.
See as "Benchmarking (writes|reads) using Go net/http"
It was faster but not by very much. Running CGI programs is just forking processes, so Apache's forking model works just about as well as anything else.
Chrony is also much better software than any of the PTP daemons I tested a few years ago (for an onboard autonomous vehicle system).