As an outsider (PhD student in a quantitative field, no relation to physics), the experimental physics community really strikes me as a class act. High standards for statistical significance, vigorously working to rule out mundane explanations before publishing data, outlining which statistical tests will be performed before data is collected...I'm a fan.
"In fact, the researchers were so startled to see such a blaring signal in the data that they held off on publishing it for more than a year, looking for all possible alternative explanations for the pattern they found." That's pretty amazing; as far as I can tell, such caution is less typical in e.g. the brain sciences.
The popular media really like to grab any neuroscience paper and twist the hell out of it. Talk to most neuroscientists and they are much more conservative in their leaps and jumps... in reality its moving slowly, but the media wants to portray a Ray Kurzweil reading of every finding.
I think it's a two way relationship, scientists love a chance to reach out to the media too, even if a bit more reserved, you even get that from their press releases. Publicity helps in getting funding in the life sciences, for better or worse, it's part of the system.
The problem is you have multiple amplifiers: The researchers provide the loudest version they feel comfortable with in order to get published. The university PR department makes it a little louder to get a little more attention. Then the media makes it louder yet.
Of course, running a signal through that many amplifiers tends to introduce a lot of distortion.
Something that fascinated me about the LHC results (from my armchair) was how the physics community is seamlessly blending rigorous statistical thinking with state of the art machine learning techniques.
Just the slides in the two big announcement presentations. I don't have links but I imagine they're up on a CERN webpage somewhere.
ML is used pervasively. At the top level they use monte-carlo to simulate the device as well as to train ML classifiers (boosted decision trees of some flavor) that select what data from the detectors can be safely ignored.
My Ph.D thesis was titled "Distributed Machine Learning" and focused on boosted decision trees - this was 1997. Ok, I am small beer and I will get lost in any discussion with the smart folk at Cern in about 2 seconds flat, but I feel it necessary to say whenever I can that I was astonished when I read that the LHC was using a learned model over boosted decision trees as a noise filter. A large number of trees induced in data will find co-incidental outcomes; think about running trials - run 10 and the likelihood of a false positive is low, run 1000 and it's very high! Boost a decision tree and you will bring all sorts of things out of the margins; some of them will even be real. Is the bump in the higgs signal real? There were 15 events that generated that bump, out of n billion. What are the odds of a boosted tree regularizing the noise into that bin - I would say pretty decent.
But I am an old man, who knows little and does less. I shall return to my beer and vegetables.
same here, what also fascinates me is the close interaction between empirical folks and the theoretical ones. there's this understanding that they work in tandem, and the theory guys are eagerly awaiting what the empirical folks have to tell them and vice versa. its rarer than you think.
"In fact, the researchers were so startled to see such a blaring signal in the data that they held off on publishing it for more than a year, looking for all possible alternative explanations for the pattern they found." That's pretty amazing; as far as I can tell, such caution is less typical in e.g. the brain sciences.