24

Asked a question earlier for a problem I was having. After a few minutes I went back to searching for a solution, and came across

enter image description here

I don't know much about SEO or web-scraping beyond the basics, but is there something that tells the google our content changes often, so check us often?

5
  • 3
    possible duplicate of Speedy Google indexing
    – user9517
    Commented Feb 27, 2014 at 22:46
  • 1
    @Iain it appears this is a duplicate, however MDMarra provided a much better answer :-)
    – MDMoore313
    Commented Feb 27, 2014 at 23:05
  • Not only does GOOG love StackX properties, they actively harvest the active ones.
    – quadruplebucky
    Commented Mar 1, 2014 at 1:54
  • 1
    Google doesn't just index sites, they actively follow a gajillion feeds like Twitter, NYTimes, blogs, etc other places to catalog new, public content as quickly as possible. It's publicly known that Google will license (presumably public) feeds from major providers for large sums of money, and this is a significant source of revenue for some providers. Gathering new information quickly is key to Google keeping on top of trends and providing relevant advertisements. They could just be following The SE RSS feeds.
    – Stefan Lasiewski
    Commented Mar 4, 2014 at 20:05
  • @StefanLasiewski sounds like it would make a decent answer for this meta post.
    – MDMoore313
    Commented Mar 5, 2014 at 0:04

3 Answers 3

Reset to default
21

Yes, Google knows which sites generate a lot of valuable content and index accordingly. If they didn't, they wouldn't be a very good search engine :)

In fact, I believe that Google worked directly with SE when they moved from serverfault.com style-naming to site.stackexchange.com style naming for SE 2.0 sites to ensure that there wasn't a major impact to search results. As to how Google does this - that may be a question for Google.

6
  • 13
    Google loves us, This I Know - For the search stats tell me so! (without revealing anything too specific and getting beaten up by the community team, the VAST MAJORITY (by multiple orders of magnitude) of our traffic comes from Google). Stack Exchange sites in general, and the Trilogy Sites (SO/SF/SU) in particular, are "high impact" and frequently-linked which drives up our search rankings ; plus I'm relatively sure there's some back-end magic biasing searches toward us for some keyword/keyphrase combinations.
    – voretaq7 Mod
    Commented Feb 27, 2014 at 20:06
  • I seem to remember from one of the older podcasts that Google has a feed they watch to be notified of new and updated pages. I can't find it now.
    – longneck
    Commented Feb 27, 2014 at 20:11
  • I'm pretty sure key phrases such as "voretaq7", "mdmarra", and "cloud/butt" drive a lot of traffic here.
    – TheCleaner
    Commented Feb 27, 2014 at 21:39
  • 5
    So, what can the internet tell me about voretaq7's cloud butt? Enquiring minds need to know...
    – MadHatter
    Commented Feb 27, 2014 at 22:08
  • 4
    @MadHatter: The internet doesn't have sufficient Xs to rate that.
    – user9517
    Commented Feb 27, 2014 at 22:47
  • I think "Jon Skeet" is also a prominent keyword.
    – Agi Hammerthief
    Commented Mar 10, 2014 at 18:12
13

Update 2018

Google announced that they have dropped this feature. Sitemaps are still going to work, but will of course be slower.

Websites can "Ping" search engines to notify them when new content is available. For my WordPress instance I use a sitemap generator plugin that also uses this method to notify Google, Yahoo, Bing etc. When I publish a new post it is available there instantly.

I would assume that the StackExchange network uses a similar function.

Quote from the plugin faq:

This plugin can automatically notify Google and YAHOO when the content of your blog changes. This service is free to use, YAHOO just requires an API key which can be freely obtained here. After the search engines recieved the "ping" they may come and crawl your site again. Since the sitemap files contain the last change of every post or page, the spiders should just retrieve the changed ones and save your traffic.

6

Google adheres to Sitemap Protocol 0.9 as defined by sitemaps.org, and as a result, you can instruct Google's crawler how often your site updates. If they figure out that your site is popular, and you claim that your content updates often, they will prioritize you and crawl on your site more often. Often enough, apparently, to make questions and answers here show up within minutes.

You can't expect a low-traffic blog that states that it updates "always" to get the same attention from the Google crawler, obviously.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .