Implementing this feature will mean that I'll be passing out more "Be Civil!" temporary suspensions, and to names we all see a lot of.
The intentional disconnect between anonymous-criticism and those that desire explanations for the -1 is one of the bigger ways we prevent flame wars. It's bad enough when some 1-rep user starts casting aspersions on the downvoters generically in a comment, but allowing them to do so directly in the inboxes of the downvoters is a level of escalation several of our regulars won't be able to resist.
The only way this would work out would be if it's gated on rep. It doesn't work until you have 500 rep, something big enough to exclude both classes a newbs (101-rep transfers, and 1-rep new users) and have enough rep to perhaps know some of what we like to have in civil discourse.