I am starting to think that the best way to get a new site into Google fast is to link to it from a weblog. I linked to brutecreations.com (a friend’s site) a few days ago, and she’s in Google now. Nr. 1 for handmade colombia jewelry, even though the site is pretty much all images.
My logs show the Googlebot indexed 5 pages, the MSNBot indexed 4 pages.
What would a search-engine friendly robots.txt file look like?
February 21st, 2004 at 6:56 pm
Yes, it’s long been known that a link from a frequently indexed page will get you into Google very quickly.
A search-engine friendly robots.txt would be empty or only contain directives that limit bot access to pages you want indexed.
February 23rd, 2004 at 7:37 pm
(using: http://www.webtoolcentral.com/webmaster/tools/robots_txt_file_generator/ )
your robots.txt file:
# All robots will spider the domain
User-agent: *
Disallow: