Tuesday, July 05, 2005

How do I know when Google hits my site?

My favorite way to track this is to use a default 404 file not found script that tracks access to robots.txt.

My experience has been that Google sometimes visits sites without using Googlebot as the user agent. However, they always seem to check for robots.txt. I know this because I track the IP and I can reverse trace the IP address back to Google.

http://www.nirsoft.net/utils/ipnetinfo.html

What I do is create a script that will handle all 404 - file not found requests. When I detect that the requested page is robots.txt, I write this information to a file.

I can use this information for creating cloaked pages (which I think are old school and I don't do anymore) or I can use this information for learning about how google visits my sites.

8 Comments:

Blogger John said...

coach outlet store online
coach outlet online
birkenstock
fit flops
ray ban glasses
longchamp handbags
canada goose pas cher
louis vuitton outlet store
michael kors
canada goose jackets
20161015yuanyuan

Sunday, October 16, 2016 5:14:00 AM  
Blogger Unknown said...

michael kors handbags
cheap nfl jersey
ralph lauren uk
tiffany jewellery
michael kors outlet
ugg boots
longchamp handbags
nike trainers uk
kate spade uk
nfl jerseys wholesale
20170114caihuali

Saturday, January 14, 2017 1:56:00 AM  
Blogger raybanoutlet001 said...

birkenstock sandals
bears jerseys
asics shoes
reebok shoes
chaussure louboutin
cheap michael kors handbags
nike huarache trainers
nike store uk
fitflops sale clearance
jordan shoes

Tuesday, January 17, 2017 2:20:00 AM  
Blogger chenmeinv0 said...

the north face outlet
polo shirts
burberry outlet online
mont blanc mountain pen
ray ban wayfarer
louis vuitton handbags
oakley sunglasses
ugg outlet
abercrombie and fitch
ralph lauren outlet
hzx20170120

Friday, January 20, 2017 1:41:00 AM  
Blogger Unknown said...

jianbin0317
ralph lauren shirts
ray ban sunglasses
rolex outlet
salomon shoes
michael kors outlet online
ferragamo outlet
oakley sunglasses wholesale
christian louboutin outlet
cazal outlet
jerseys wholesale

Thursday, March 16, 2017 8:40:00 PM  
Blogger Admin said...

Thank you for your support who has read our article. I'm very happy if you can share our article below

5 Tanaman Herbal Untuk Menyembuhkan Maag
Cara Meneyembuhkan Flu Singapura
Obat Dermatitis Paling Ampuh
Obat Sariawan Pada Lidah
Obat Untuk Gusi Bengkak

Monday, May 21, 2018 11:35:00 PM  
Anonymous Pengobatan Kejang Demam Pada Anak said...

If the disease is not treated immediately, then everything will result in harm to the health of the body. Thank you for sharing the information
Pengobatan Patah Tulang
Obat Ampuh Atasi Penyakit Malaria
Cara Mengobati Ginjal Bengkak
Obat Ginjal Bocor Untuk Semua Usia
Obat Wasir Tanpa Operasi
Pengobatan Diare Paling Ampuh Sampai Sembuh

Monday, August 13, 2018 3:00:00 AM  
Anonymous Nicholas B said...

I liked your blog thanks for sharing this.

Wednesday, July 03, 2024 9:37:00 AM  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home