Stay organized with collections
Save and categorize content based on your preferences.
Wednesday, January 09, 2008
Confused about the best uses of robots.txt, nofollow, URL removal tool? Wondering
how to keep some of your pages off the web? Our webspam lead, Matt Cutts, talks about the best
ways to stop Google from crawling your content, and how to remove content from the Google index
once we've crawled it.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],[],[[["Matt Cutts explains the optimal methods for preventing Google from crawling specific website content, including using robots.txt, `nofollow`, and the URL removal tool."],["The video addresses how to remove already-crawled content from Google's index."],["Viewers are encouraged to provide feedback on the video through the Google Webmaster Help Group."],["While the video mentions robots.txt originating in 2006, a note clarifies its actual inception was around 1996."],["Additional resources and support are available through Google's Help Center articles on content removal."]]],[]]
