Asked a question earlier for a problem I was having. After a few minutes I went back to searching for a solution, and came across

enter image description here

I don't know much about SEO or web-scraping beyond the basics, but is there something that tells the google our content changes often, so check us often?

  • 3
    possible duplicate of Speedy Google indexing – Iain Feb 27 '14 at 22:46
  • 1
    @Iain it appears this is a duplicate, however MDMarra provided a much better answer :-) – MDMoore313 Feb 27 '14 at 23:05
  • Not only does GOOG love StackX properties, they actively harvest the active ones. – quadruplebucky Mar 1 '14 at 1:54
  • 1
    Google doesn't just index sites, they actively follow a gajillion feeds like Twitter, NYTimes, blogs, etc other places to catalog new, public content as quickly as possible. It's publicly known that Google will license (presumably public) feeds from major providers for large sums of money, and this is a significant source of revenue for some providers. Gathering new information quickly is key to Google keeping on top of trends and providing relevant advertisements. They could just be following The SE RSS feeds. – Stefan Lasiewski Mar 4 '14 at 20:05
  • @StefanLasiewski sounds like it would make a decent answer for this meta post. – MDMoore313 Mar 5 '14 at 0:04
up vote 22 down vote accepted

Yes, Google knows which sites generate a lot of valuable content and index accordingly. If they didn't, they wouldn't be a very good search engine :)

In fact, I believe that Google worked directly with SE when they moved from serverfault.com style-naming to site.stackexchange.com style naming for SE 2.0 sites to ensure that there wasn't a major impact to search results. As to how Google does this - that may be a question for Google.

  • 13
    Google loves us, This I Know - For the search stats tell me so! (without revealing anything too specific and getting beaten up by the community team, the VAST MAJORITY (by multiple orders of magnitude) of our traffic comes from Google). Stack Exchange sites in general, and the Trilogy Sites (SO/SF/SU) in particular, are "high impact" and frequently-linked which drives up our search rankings ; plus I'm relatively sure there's some back-end magic biasing searches toward us for some keyword/keyphrase combinations. – voretaq7 Feb 27 '14 at 20:06
  • I seem to remember from one of the older podcasts that Google has a feed they watch to be notified of new and updated pages. I can't find it now. – longneck Feb 27 '14 at 20:11
  • I'm pretty sure key phrases such as "voretaq7", "mdmarra", and "cloud/butt" drive a lot of traffic here. – TheCleaner Feb 27 '14 at 21:39
  • 5
    So, what can the internet tell me about voretaq7's cloud butt? Enquiring minds need to know... – MadHatter Feb 27 '14 at 22:08
  • 4
    @MadHatter: The internet doesn't have sufficient Xs to rate that. – Iain Feb 27 '14 at 22:47
  • I think "Jon Skeet" is also a prominent keyword. – Agi Hammerthief Mar 10 '14 at 18:12

Update 2018

Google announced that they have dropped this feature. Sitemaps are still going to work, but will of course be slower.


Websites can "Ping" search engines to notify them when new content is available. For my WordPress instance I use a sitemap generator plugin that also uses this method to notify Google, Yahoo, Bing etc. When I publish a new post it is available there instantly.

I would assume that the StackExchange network uses a similar function.

Quote from the plugin faq:

This plugin can automatically notify Google and YAHOO when the content of your blog changes. This service is free to use, YAHOO just requires an API key which can be freely obtained here. After the search engines recieved the "ping" they may come and crawl your site again. Since the sitemap files contain the last change of every post or page, the spiders should just retrieve the changed ones and save your traffic.

Google adheres to Sitemap Protocol 0.9 as defined by sitemaps.org, and as a result, you can instruct Google's crawler how often your site updates. If they figure out that your site is popular, and you claim that your content updates often, they will prioritize you and crawl on your site more often. Often enough, apparently, to make questions and answers here show up within minutes.

You can't expect a low-traffic blog that states that it updates "always" to get the same attention from the Google crawler, obviously.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .