What I keep running into is that teams know the work is informal at the edge, but systems are designed as if formality can be enforced at capture time. In practice, that just pushes the work elsewhere.
In your experience, where does that mismatch hurt the most: audits, safety reviews, customer disputes, or something else?
Computing and formal mathematics rely on exhaustive case analysis and binary logic. And even with The Excluded Middle, there are unprovable statements.
There are at least two incompatible ternary truth tables (hello there SQL NULL) in formal logic. Then there is fuzzy logic, but that is also a formalization.
(NP-complete problems and uncomputable problems in binary logic are another sore point.)
And for informal systems the best people have found so far is hypothesis testing, which is to say that only the rejection of hypothesis based on measurements works, but not confirming a hypothesis.
Agree with your pov, especially the idea that you can only reject hypotheses, not confirm them.
What I keep seeing operationally is that teams are forced to act before hypotheses can be tested or falsified. For example, inspections completed, assets redeployed, and customers responded to. Only later are they asked to prove correctness.
When that gap shows up for you, what was the concrete trigger? An audit, a customer dispute, a safety review, or something else?
I’m curious in the cases where that 1% mattered, what made the reconstruction painful even when the raw artifacts existed? Was it ordering events, understanding intent/decisions, or just finding the right things under pressure?
And were those moments tied to audits, disputes, safety issues, or something else?
@scott-iii That’s exactly the kind of situation I’m trying to understand better.
Out of curiosity, when you were piecing it together from the camera roll, what was hardest: ordering events, understanding why decisions were made, or just finding the right photos at all?
And was this tied to an audit, a customer issue, or something else?
I know some friends who were put on the RIF list for being critical of AI strategy in a division within a large company, whose CEO is talking about intelligent minds and diffusion.
sorry a bug took me down, so could not reply earlier. Your comment hits the spot.
I agree with you about not just seeing raw numbers or some kind of trend line, but get more analytical insights into what is really happening with the data and the systems feeding that data. However, I am also skeptical that most humans are terrible at understanding stats so any information based on statistical analysis must be dumbed down to ELI5 level so most report consumers understand. How did you approach this problem?
> However, I am also skeptical that most humans are terrible at understanding stats so any information based on statistical analysis must be dumbed down to ELI5 level so most report consumers understand.
For most regular people, I think stats is kinda like code: As soon as they see it, their eyes glaze over and they stop paying attention. I don't think they should even see the underlying math, just the abstracted conclusions. For real-time data, that might be a simple traffic light (green/yellow/red with colorblind-friendly symbology) showing when something is within expected ranges, turns yellow when something is 1 standard deviation away, red when it's 2+, blinking and screaming when it's 3+, etc. The underlying stats have to be carefully analyzed based on the actual use case with proper domain knowledge, but the UI can simplify that down to "everything is ok" vs "keep an eye on this, it's starting to look weird" vs "wake everyone up in the middle of the night to deal with this".
For time-series, maybe it's a colored background range showing the expected the highs/lows for those few months, under the actual data line that plots the real data. If the line is within the range it's fine, if it's far outside the expected for that time period, something might be off.
For deltas, sparklines or font bolding/coloring/sizing can give a visual indication of the magnitude of change, either relative to the previous time period (this metric is 2000% higher than yesterday) and/or relative to other metrics in the dataset (everything else changed +/- 10%, but this one was +50%).
The downside to this approach is that it requires actual domain knowledge, an understanding of each metric and their applicability to the business, a customization for each report viewer based on what their job can do (i.e. what actions can they actually take in response to which metrics), and a lot of filtering, analysis, testing, and further iterations. It's a far cry from a 1-click auto-generated dashboard based on some standard dataset (like web analytics).
In one of my jobs, we used https://lookerstudio.google.com/ quite a lot because it also allowed the report viewers to edit the dashboards on their own (for simpler things, or to change pagination and layout, etc.). But we'd have pipelines in the middle that would ingest raw data and produce statistics for the dashboard. But we approached it like any UX problem: not metrics-first, but user-first. We talked to them about why they wanted a dashboard, how they consume the data, how they triage the metrics, how they respond according to the metrics, how they like to be notified or not, etc. It's a very personalized approach that tries to mimic what a good assistant would do, e.g. "here's today's must-know summaries for you" vs the sysadmin approach ("here's how every CPU core and process is doing").
Fundamentally, it was about finding the very few signals among all the noise, showing as little as possible at a time upfront, but allowing drill-downs where needed. 90% of the time they wouldn't drill down – which to me was a good sign that we were able to customize the dashboard to their everyday needs.
-----
I should note that this is purely anecdotal, based on my experience as a frontend dev who also had to make several dashboards, not a data specialist/data scientist.
Am better today, thank you :-). Your suggestions align with how we are thinking about addressing this problem. I like the suggestion around assistant vs admin. It strongly resonates with me and points us in a direction.
btw, multiple anecdotes are helpful, so thank you once again.
sorry a bug took me down, so could not reply earlier. Thank you for your reply and a few follow-up questions and observations.
1. I do agree with you on the challenge between "feeling" vs "what i actually want". Is this about spending more time with the report consumer to understand what they want to see? Or is there something more?
2. How did you solve this problem? I would love to learn how you addressed this, despite all the struggles.
3. If you were to revisit this same problem today, what would you do differently?
Not just arsenic but also to remove the starch. Most folks don’t cool down the rice before eating so it’s preferable to remove the starch before eating.
That's the opposite of true. Getting light before the start of the waking hours but getting dark before you're even home—i.e., standard time—is unhealthy. Daylight time aligns to natural circadian rhythms better.
Objectively, more people are awake past 5pm than are awake before 7am.
Humans are the exception. Most animals return home as it gets dark. That is the natural pattern, something we are all primed via evolution. Your personal preference to get more light in the evening because you want to spend more time outdoors runs against evolutionary instincts.
People are ruled by the numbers on the clock, so we should shift the daylight hours during those hours people are awake to match the natural needs of the people.
But from where did the habits tied to the numbers come? Were they formed without reference to daylight?
Will most people always sleep before 7am and be awake after 5pm, no matter when there is daylight?
Hmm I would think the problem is changing the time twice a year? Even if permanent DST was a mismatch to our circadian rhythm, wouldn't that completely depend on how close you are to a date line?
They are taking a charge back of $1.2bn with the layoff. Are you suggesting canceling the private performance would have costed a huge amount on similar lines?
No, of course not. But others seem to be. And in my opinion by the time they decided to do layoffs it would’ve been too late to cancel the concert and get their money back, so you might as well enjoy the concert.
What I keep running into is that teams know the work is informal at the edge, but systems are designed as if formality can be enforced at capture time. In practice, that just pushes the work elsewhere.
In your experience, where does that mismatch hurt the most: audits, safety reviews, customer disputes, or something else?