I never had that issue... and my .thunderbird folder is ~15gb. Sorting the email (eg. by size) takes a few seconds though, but i don't do that often.

Compared to eg. gmail (web interface), i find the speeds comparable, with the added benefit of having my emails offline (too).