Skip to main content
edited body
Source Link
Chris S Mod
  • 78.3k
  • 36
  • 72

Eh, sorta... You can set robots.txt to not index the transcripts of particular rooms or the whilewhole site. But:

  1. That doesn't stop anyone from copying the content, and allowing it to be indexed from their site.
  2. Google's a bit slow at updating the "rules" from robots.txt if it's already indexed a site.
  3. Rule #5 - The Internet Never Forgives or Forgets

Eh, sorta... You can set robots.txt to not index the transcripts of particular rooms or the while site. But:

  1. That doesn't stop anyone from copying the content, and allowing it to be indexed from their site.
  2. Google's a bit slow at updating the "rules" from robots.txt if it's already indexed a site.
  3. Rule #5 - The Internet Never Forgives or Forgets

Eh, sorta... You can set robots.txt to not index the transcripts of particular rooms or the whole site. But:

  1. That doesn't stop anyone from copying the content, and allowing it to be indexed from their site.
  2. Google's a bit slow at updating the "rules" from robots.txt if it's already indexed a site.
  3. Rule #5 - The Internet Never Forgives or Forgets
Source Link
Chris S Mod
  • 78.3k
  • 36
  • 72

Eh, sorta... You can set robots.txt to not index the transcripts of particular rooms or the while site. But:

  1. That doesn't stop anyone from copying the content, and allowing it to be indexed from their site.
  2. Google's a bit slow at updating the "rules" from robots.txt if it's already indexed a site.
  3. Rule #5 - The Internet Never Forgives or Forgets