http://serverfault.com/q/823822/22361

After much troubleshooting the problem turned out to be a hardware fault with one of the network adapters.  For unknown reasons the adapter refused to "see" the peer's advertising of 1Gb speed, connecting at 100Mbps.  Even forcing it to 1Gb and autoneg-off with `ethtool` had no effect.

I replaced the adapter with a newer one based on the same chipset (R8169) and the problem disappeared, now connecting at 1Gbps as expected.

My first reaction is to delete the question, but I can write an answer that explains the resolution as a reminder to future readers to make sure the hardware is OK before blaming the software.

I will defer to the community's wishes on this.