I recall seeing a mention a little while back about changes to the SE sites that mean rep adjustments are made "on the fly" and recalcs are no longer necessary. I think it's broken, at least on SF. I don't see my rep changing when I cast downvotes and recently my rep suddenly went down by several hundred points, which could really only be the result of a recalc.
The global reputation recalc is responsible for your rep loss, this happened globally across the network. Previously, reputation was not real-time or anywhere close, so things that negatively affected reputation were not applied and your actual value vs the highly denormalized value you see got way off sync.
This round of changes aimed to fix that, we not keep your reputation in sync, in most cases instantaneously, in other complicated cases, within 5 minutes.
Downvotes are instantly reflected on your reputation, though the confusion here may be that questions do not penalize you 1 rep any more, only downvotes on answers do this.
Down-votes on questions are free these days, which could be part of it.
The drop by several hundred bit is interesting though. Your history doesn't show any recent recalcs that adjusted to anywhere near that magnitude. The last big one was March 2nd, which was a global rep-recalc.
If the rep-drop is more recent then that, I can't dig out the cause from here. Theoretically the only thing that would cause something of that magnitude is a recent post of yours earning a lot of rep that then gets deleted. Historic questions getting shredded no longer dings rep like it used to.
The change to instant rep was rolled out gradually to all users. Yours was applied on the 28th of March, however allegedly your rep only changed by one point (and it went up).
However, downvotes are now free for questions so you won't see any rep change because of that. I just did a test by throwing a -1 on a spam answer, and I saw my rep change immediately after refreshing the page.
Hopefully a dev will see this and cast more light.
Just noticed you did spend 500 rep in September 2011 on a bounty. Kudos to you for throwing out a big bounty...