Where are you located?
I'm in Sydney, Australia and for the last 10 months I've had speed issues with the site - it's so "normal" now that I don't even notice them.
See my meta.stackoverflow question heremeta.stackoverflow question here - there's a lot of info on how to diagnose CDN issues in the answers.