Great articles on Search engines & their respective Bots (why Yahoo.com named theirs "Slurp" is completely beyond me):
http://ghita.org/search-engines-dynamic-content-issues.html
http://en.wikipedia.org/wiki/Robots_exclusion_standard
Posting these as I'm assisting in some SEO/online reputation work now :)
More to come!
Still finding that excel is the best top-level site-content parser, especially for search engines - they only have a few variable responses (probably tested and limited because they're trying to find the balance between stopping automated querying of their services (much like the old API they scrapped - by-passed ads and their revenue) and confusing their users with too many different cases.
Tuesday, April 27, 2010
Subscribe to:
Posts (Atom)
