I originally shared the concern that Unix and Linux would start to pull away from SF as well. The main thing that changed my mind is that the Internet is not a zero sum game and that SF keeps growing every week.
There are many more professional system administrators out there that will be interested in participating on the Server Fault and becoming part of the community. Many of them will be Unix experts but consider themselves system administrators first and will want to be part of our community. Some of them will only want to be part of a Unix community, so Unix.se is for them.
Will there be some Unix experts who are also sysadmins that might have been part of Server Fault if it wasn't for Unix.se? Probably, but not enough to loose sleep over I don't think. Since the Internet is so big -- loosing a couple of them to Unix.se is going to harm our community from what I have seen. We already have people that know a lot about Unix and they will keep coming. I also think that there will be plenty of expert Unix questions coming to Server Fault even as Unix.se grows.
When it comes to moving questions -- I wouldn't move them unless you feel that SF is the wrong place.