The comments are stating to be a bit long and I believe the question/statement deserve at least one answer.
A little recap, parent name servers have wrong glue records for ns4.serverfault.com
As it stands, the glue record is wrong and should match the Authoritative Name Servers.
The glue records are the entries that you will find in the Name Servers section when querying the registrar of a domain.
whois serverfault.com | grep -A 4 "Name Servers"
Name Servers:
ns1.serverfault.com
ns3.serverfault.com
ns4.serverfault.com
That's why you never see ns2.serverfault.com in the glue records of the parent name servers. Even if it's allowed to answer queries for serverfault.com with the AA flag.
These records are the delegated name servers.
They ARE NOT the Authoritative Name Servers.
Authoritative Name Servers are the NS RRs found in the zone. As stated in the RFC 1035 (section 3.2.2)
One way to get all the Authoritative Name Servers is to ask for the NS entry of a zone to the SOA of that zone.
dig +short serverfault.com soa
ns1.serverfault.com. sysadmins.stackoverflow.com. 2013020902 600 600 604800 1440
dig +short @ns1.serverfault.com serverfault.com ns
ns3.serverfault.com.
ns1.serverfault.com.
ns2.serverfault.com.
ns4.serverfault.com.
These are the only name servers allowed to answer queries for serverfault.com with the AA flag. (Authoritative Answer). And are not allowed to do recursive query for serverfault.com.
Now, to the trace and glue issue.
If I have no cache of anything with regards to serverfault.com, my resolver will do the equivalent of a dig +trace serverfault.com
As a matter of fact, that command is basically asking your resolver to act as a cache server with an empty cache.
Running dig +trace serverfault.com along with tshark -R "udp.port == 53" -i wlan0
PC ask resolver for all root servers. (Iterative, meaning no RD flag)
m.y.i.p -> m.y.d.ns DNS 59 Standard query NS <Root>
Resolver answers.
0.244995 m.y.d.ns -> m.y.i.p DNS 378 Standard query response NS e.root-servers.net NS f.root-servers.net NS g.root-servers.net NS h.root-servers.net NS i.root-servers.net NS j.root-servers.net NS k.root-servers.net NS l.root-servers.net NS m.root-servers.net NS a.root-servers.net NS b.root-servers.net NS c.root-servers.net NS d.root-servers.net
Then you'll have all the A request for every root servers along with the resolvers answer. But these two lines are important here...(Recursive, with RD flag)
0.331326 m.y.i.p -> m.y.d.ns DNS 78 Standard query A k.root-servers.net
0.344396 m.y.d.ns -> m.y.i.p DNS 408 Standard query response A 193.0.14.129
PC ask k.root-servers.net for A serverfault.com (no RD flag)
0.428328 m.y.i.p -> 193.0.14.129 DNS 75 Standard query A serverfault.com
k.root-servers.net answers with the GTLD servers responsible for .com.
0.479877 193.0.14.129 -> m.y.i.p. DNS 535 Standard query response
Again, my PC will ask for the A record of all the GTLD servers to my resolver. (with RD flag)
0.634209 m.y.i.p -> m.y.d.ns DNS 78 Standard query A l.gtld-servers.net
0.647336 m.y.d.ns -> m.y.i.p DNS 414 Standard query response A 192.41.162.30
PC ask l.gtld-servers.net for A serverfault.com (no RD flag)
0.662461 m.y.i.p -> 192.41.162.30 DNS 75 Standard query A serverfault.com
0.763867 192.41.162.30 -> m.y.i.p DNS 177 Standard query response
l.gtld-servers.net gives the delegated name server and my PC ask my resolver for the IP of each one of them. (with RD flag)
0.765100 m.y.i.p -> m.y.d.ns DNS 79 Standard query A ns1.serverfault.com
0.777483 m.y.d.ns -> m.y.i.p DNS 211 Standard query response A 198.252.206.80
0.777707 m.y.i.p -> m.y.d.ns DNS 79 Standard query A ns3.serverfault.com
0.790625 m.y.d.ns -> m.y.i.p DNS 211 Standard query response A 69.59.196.217
0.791522 m.y.i.p -> m.y.d.ns DNS 79 Standard query A ns4.serverfault.com
0.803938 m.y.d.ns -> m.y.i.p DNS 211 Standard query response A 69.59.196.122
Note that you will always receive these 3 delegated servers in that order.
It will pick one of them and ask for the IP for serverfault.com in my case, ironically enough it picked ns4.serverfault.com. (no RD flag)
0.804629 m.y.i.p -> 69.59.196.122 DNS 75 Standard query A serverfault.com
0.905948 69.59.196.122 -> m.y.i.p DNS 227 Standard query response A 198.252.206.16
Since the last query is with no RD flag and is asked to one of the Authoritative Name Server, a dig +trace will always give you an answer with flag AA...meaning not from your resolvers cache but rather from one of the Authoritative Name Server.
You have just completed the delegation path from Root to Authoritative Name Servers.
The reason why ns4.serverfault.com will resolve properly even if the glue record is wrong is that your resolver, or any resolver, will actually learn about ns1.serverfault.com and ns3.serverfault.com.
Both of them are able to answer queries for serverfault.com and you will know about them first.