I am going to repeat what I always say when these book threads come up:
I love all the recommendations here but please say a word or two about why you recommend said books. Sell them to me, don't make me do the work. Dumb lists of titles are so uninteresting.
2 books I absolutely love and have read cover to cover several times, solved most of the 1000+ problems.
1. Inference - Rohatgi
2. Inference - Stapleton
Why I recommend them ?
The real answer is super long. But the short version is - there are thinkers & there are doers. Basically, the mathematical statistics world has these theory-building Bourbaki type guys who write a LOT, say a LOT, but never get to the fucking point (imho). The opposite view is "math is bunch of tricks. its like chess - more middlegames & endgames you know, higher chance of winning. No real point in learning who originally came up with this particular middle game variation, or why does this opening work etc etc. Just learn the trick & play the game." So that's the eastern (Indian/Chinese) school of thought, which is what I subscribe to.
The 2 inference books listed above are essentially grab-bags of tricks. Do this - it works - now try it on these problems - ok next trick...on & on. So I solved the 1000+ problems & now I know lots of these methods that just work.
eg. recently i was asked - some vc's are evaluating a startup. their valuations are $1 million, $4 M,$10M, $20M, $50M. what's your evaluation & why ?
so i'm thinking - hey isn't this just rohatgi taxicar ? so i quickly said- sum is 85, times 1/5 is 17. Whereas largest observed is 50, times 6/5 is 60, so half is 30. since 50 was max observed, another estimator is half that, ie. 25. if you want doctor's estimate, get rid of 1 and 50, then sum is 34 so times 1/3 is 11.3
so then we have 4 estimators, - the sample mean is 17 million, its the method of moments estimator, clearly unbiased but high mean square error because variance is high. the maximum likelihood estimator is 25 mil, and has smallest variance, but the mse will not be the lowest since it is not unbiased so bias square will add. the 30 mil estimate is also unbiased, but has low variance so it has the lowest mse of the lot. the doctor estimator 11 million is unbiased but high variance and mse is in between. now if you want the absolute lowest mse, i can cook up a 5th estimator which has nonzero bias but mse will be the minimum....
at this point the interviewer interrupts me - you've never seen this problem because we came up with it in our last meeting at our firm. Yet you gave me 4 very good estimators under 2 minutes & want to cook up a 5th one that's even better. And you don't even have a phd. meanwhile i just spoke to an actual phd and asked him this same question, he went on and on for 20 minutes without giving me a single concrete estimator!
so that's the thing. rohatgi, stapleton, these are about real world, down & dirty, how to do stuff. how to solve actual problems.
whereas the gelman bda, the shao, the schervish, the lehman, the bickel & doksum - these were my prescribed textbooks. imho they are absolute garbage, worse than dirt. after the exam i threw them away. such bullcrap. they go on & on without getting anywhere & have practically zero good worked examples.
so that's my 2 cents. i still have the rohatgi & stapleton on my desk. sometimes i tear up when i look at them. they have taught me so, so much!
My personal 2ct: first understand how to solve "real" problems, then move on to the abstract theory. I cannot understand the theoretical mathematics behind a concept if I don't have at least one non-trivial example in my head which I understand completely.
I 100% agree with you. I struggled with math through school. I remember wondering what the heck the point of matrices was when I was first learning about them in 10th grade. I got to grad school and I had a professor who explained everything as a problem we were trying to solve and then gave us the tools and then finally the theory. Math went from a constant fight to a fun experience. Now I run the data science department at a corporation (turns out matrices are really important).
I like how you laid this out as a tension between two worldviews. I'm curious: as a seeming master of the "bag of tricks" worldview, do you feel like having fully absorbed so many "tricks" has given you a kind of empirical insight into things?
In other words: have the tricks, when crashed against the real world over and over like you describe, oozed into something akin to a larger theoretical sense of things? Or do they remain isolated tricks that you apply when they seem to match?
(I wish I could frame this question better, but I don't know this field, sadly.)
i will unfortunately give you a half-assed response. Because the truth will take up a lot of time & space & is very inflammatory. So here's what I honestly believe - there are only isolated tricks. The ones who sit in judgement of us, like savants, and claim some sort of coalesced mastery of these tricks into some larger theoretical construct, are faking. They are good at faking, so we listen to them. They have better rhetoric, better optics. But beneath it all, it is a grab-bag of tricks. I worked with several professors. When trying to prove something they will start with their favorite trick. One guy was like - okay let me see if i can cauchy schwarz this. I'm like, why ? Why cauchy schwarz always? There are million inequality bounds. Why pester cs all the time ? He actually looked at me curiously then said I don't know I just have some favorite tricks it seems to work! Atleast he was honest. So that's the deal. If you probe deeply there are a few isolated tricks. They seem to work well. In reality we don't know why they work. But they do. Its like why do you not use spoon for spaghetti ? Why fork ? Ok you can cut lines into a spoon and call it a spork, but then you can't drink soup with the spork. So you reach for the spoon if you want to sip soup, use your fork for spaghetti. Don't try to design some universal theory of cookware. Just buy 1 spoon & 1 fork. That's basically all of mathematical statistics. Few tricks. They work. Why ? I mean who gives a fuck. Maybe bickel & doksum do but I personally don't.
What about creating/finding tricks? I wonder if there a larger theoretical sense of things involved there? What about in a weaker sense, like: "I think I'll start my trial-and-error with this"?
This is fascinating, going to pick up these books. Do you think one needs a background in formal statistics to understand them? If so, do you have any recommendations for self-learning?
> Basically, the mathematical statistics world has these theory-building Bourbaki type guys who write a LOT, say a LOT, but never get to the fucking point (imho).
I find your comment amusing considering:
1. You just wrote 5 paragraphs of anecdotes.
2. Bourbaki Elements of Mathematics is the driest book you can think about. It’s mostly formal definitions and extremely rigorous demonstrations.
Your snide costs you a lot of credibility at least as far as I’m concerned.
I don't understand your objections here. The anecdotes were the selling that was needed to explain why those books work. That is the point of that comment. It got to it almost immediately.
super interesting. that's pretty much align with brute force memorization that's emphasized in many asian countries.
What's your advice in the self-learning prerequisites knowledge/materials before being able to digest those books?
My last math course was calculus I and II back in first year uni 15 years ago as well as stats I and II on my second year (which I completely bamboozled and forgot the material as soon as I barely pass the class).
The most information-dense form of book recommendation possible is probably a “dumb” list consisting largely of books one knows and appreciates, with one or two new titles in addition. I’ll probably learn more about where a person recommending a book I haven’t read is coming from based on a list of their favorites than I will from the text of a hastily constructed review.
In any case, I wouldn’t dismiss someone out of hand for providing a “dumb” list.
In my line of work it's the Annual Book of ASTM Standards.
In particular the handful of volumes that pertain to my established laboratory & field work, as well as possible aspirational efforts.
It can really pay to keep up to date.
Also, the text/content of all the standards are the work of volunteer technical people, who come to complete consensus before the well-paid journalist professionals at the nonprofit publisher send it to press.
ASTM may contain some of the most statistically documented laboratory procedures for repeatabiliy & reproducibility compared to what you normally find.
A lot of the books people have commented on have been influential in the past and do stand on their own today.
Well even though outdated ASTM standards may have limited value, one real offsetting benefit is the past experience of using them in previous years when they were current and some standards were less fully developed.
So if you think about it, the current year's publication is a snapshot, the majority of your collection from previous years is huge by comparison, and ends up proportionally more helpful on the whole, even though somewhat outdated or even obsolete.
And these books are hefty, they weigh kilos per year and cost the big bucks.
Not really worth it either unless you're really ready to dip deeply into the unique type of bureaucracy associated with these type of efforts.
But it gets much worse, you think ASTM books are boring, how about the Federal Register?
There's another ongoing publication where the bureaucracy is so thick, someone can specialize so highly at navigating it that they can more effectively win bids without any technical qualifications compared to actual practicioners, most of whom don't stand a chance on merit alone. Not for me, but if you're aiming for Uncle Sam's pocketbook you need to up your game here.
I guess there are a lot of other books which might inspire people to take some action of their own, or even build a business around. Not always ones that are intended to be inspirational either.
Some people think Buckminster Fuller was inspirational, one of my handful of books when I had a shelf is Earth, Inc.
Title sounds almost like the name of a business or something:
I'll counter. Don't tell my why you recommend it; tell me briefly what it's about instead and I'll decide for myself whether or not I want to read it.
I like to have moments to myself without feeling like I am being sold something. I feel like I am constantly being sold things on the internet; let me do the discovery myself. Let me decide for myself whether or not it is interesting to me.
If someone says why they really like a book, you can just... skip that part. If you post a list of titles with zero context, the only people you're helping are those who just want a list of titles, which is almost certainly the minority.
I love all the recommendations here but please say a word or two about why you recommend said books. Sell them to me, don't make me do the work. Dumb lists of titles are so uninteresting.