Personal anecdote: every time I quit caffeine, a week later I still had lethargy and headaches. It might take a bit longer than a week for it to pay off. 2-4 weeks and you feel more human
Interesting, I often drop caffeine for a week or so (without any real conceited effort, typically just when I'm sick or on vacation) and any withdrawal seems gone after a few days but then restart it within a week because it's just so damn good, especially when you only sleep 4-6 hours a night which is... every night for me.
I definitely had headaches for a few days, but I really did feel calmer and more relaxed. It's damned HARD to stay caffeine free, though. It's really hard.