Comments on: Debugging The Network Unreachable / Robots. txt Unreachable Error http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/ Working at home... just me being me.. Fri, 30 Jul 2010 01:56:17 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Valeria Kelly http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-784 Valeria Kelly Thu, 29 Jul 2010 17:34:47 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-784 are there any good seo forum which discusses more about keyword research?""" are there any good seo forum which discusses more about keyword research?"""

]]> By: Xavier Watson http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-761 Xavier Watson Thu, 24 Jun 2010 04:47:14 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-761 What is the best SEO Forum these days?'~" What is the best SEO Forum these days?'~"

]]> By: Salena Amigon http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-750 Salena Amigon Tue, 15 Jun 2010 14:59:15 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-750 Observed your site on del.icio.us these daysand really liked it.. i bookmarked it and will likely be back again to catchit out some more later .. Observed your site on del.icio.us these daysand really liked it.. i bookmarked it and will likely be back again to catchit out some more later ..

]]> By: Yohan Perera http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-731 Yohan Perera Thu, 13 May 2010 03:23:18 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-731 Actually your article helped me to solve a different kind of problem. I am using Pingdom to monitor my blog. Since the day I started using Pingdom with it’s default 5 minutes check, I started to receive frequent down alerts for my blog :sad: . I checked with the hosting company several times but they denied any down time :evil: . Later on I found out that the firewall guarding my web server has a automatic mechanism which blocks the IP address of any outside host that makes concurrent connections in a short period of time :mrgreen: . Since I was using 5 minutes check resolution Pingdom was making concurrent connections to my web server once in every 5 minutes. The firewall misunderstood this to be a suspicious act and blocked pingdom frequently. Pingdom had no idea that it’s being blocked by a firewall and can’t get to the site. Thus reported to me as the site is down :lol: . Actually your article helped me to solve a different kind of problem.

I am using Pingdom to monitor my blog. Since the day I started using Pingdom with it’s default 5 minutes check, I started to receive frequent down alerts for my blog :sad: . I checked with the hosting company several times but they denied any down time :evil: .

Later on I found out that the firewall guarding my web server has a automatic mechanism which blocks the IP address of any outside host that makes concurrent connections in a short period of time :mrgreen: .

Since I was using 5 minutes check resolution Pingdom was making concurrent connections to my web server once in every 5 minutes. The firewall misunderstood this to be a suspicious act and blocked pingdom frequently.

Pingdom had no idea that it’s being blocked by a firewall and can’t get to the site. Thus reported to me as the site is down :lol: .

]]> By: Hridoy http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-725 Hridoy Thu, 11 Mar 2010 18:05:10 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-725 This post helped me a lot to see solve a problem related to robots.txt. Thanks :razz: This post helped me a lot to see solve a problem related to robots.txt. Thanks :razz:

]]> By: Sharath http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-719 Sharath Thu, 31 Dec 2009 18:43:25 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-719 I too had the same problem. I added the following meta tags to all my pages and it all worked out fine. meta name="GOOGLEBOT" content="INDEX, FOLLOW" / meta name="robots" content="noodp,index,follow" / meta name="revisit-after" content="7 days" / I too had the same problem. I added the following meta tags to all my pages and it all worked out fine.

meta name="GOOGLEBOT" content="INDEX, FOLLOW" /
meta name="robots" content="noodp,index,follow" /
meta name="revisit-after" content="7 days" /

]]> By: Sharath http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-718 Sharath Thu, 31 Dec 2009 18:41:10 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-718 Hi, Nice post. I too had a similar problem. I added the following Meta Tags on all my pages and it worked out fine. Hi, Nice post. I too had a similar problem. I added the following Meta Tags on all my pages and it worked out fine.

]]> By: Engineering Colleges in Punjab http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-702 Engineering Colleges in Punjab Sat, 17 Oct 2009 05:01:10 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-702 to my mind, network unreachable problem means when google bots tries to make so many concurrent connection, website server temporarily disallows access to them. then google reports it as network unreachable. to my mind, network unreachable problem means when google bots tries to make so many concurrent connection, website server temporarily disallows access to them. then google reports it as network unreachable.

]]> By: Arthur Coleman http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-693 Arthur Coleman Wed, 16 Sep 2009 18:07:29 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-693 Great post. I am experiencing the exact same problem with only a few of my pages (as compared to all pages, which makes it a bit more difficult to debog) and this was very helpful. Great post. I am experiencing the exact same problem with only a few of my pages (as compared to all pages, which makes it a bit more difficult to debog) and this was very helpful.

]]> By: ankit http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-686 ankit Tue, 08 Sep 2009 04:13:30 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-686 I was facing same error and i also contacted my host..but they said they are not blocking any IP range..i found that due to my default wp privacy settings..it was blocking search engines to crawl my site..but i changed that option to allow all search engines to crawl my site..and other seach engines like yahoo, bing are crawling too but not google..even google ads are not showing properly..they are showing public service ads on my site ..where as my content is original and its not adult too.. I was facing same error and i also contacted my host..but they said they are not blocking any IP range..i found that due to my default wp privacy settings..it was blocking search engines to crawl my site..but i changed that option to allow all search engines to crawl my site..and other seach engines like yahoo, bing are crawling too but not google..even google ads are not showing properly..they are showing public service ads on my site ..where as my content is original and its not adult too..

]]> By: Search Engine Optimization Dallas - Fixing my Google Sitemap Errors so I show up in the search engine results | Swanson SEO Services Blog http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-678 Search Engine Optimization Dallas - Fixing my Google Sitemap Errors so I show up in the search engine results | Swanson SEO Services Blog Fri, 21 Aug 2009 20:27:03 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-678 [...] Debugging the Network unreachable / Robots.txt unreachable error [...] [...] Debugging the Network unreachable / Robots.txt unreachable error [...]

]]> By: Sitemap Google webmaster tool « MaseBlog http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-672 Sitemap Google webmaster tool « MaseBlog Mon, 03 Aug 2009 14:34:40 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-672 [...] aver letto un interessante post sull’argomento datato però 2007 ho trovato diverse risposte utili all’interno del forum di google ed in [...] [...] aver letto un interessante post sull’argomento datato però 2007 ho trovato diverse risposte utili all’interno del forum di google ed in [...]

]]> By: Ger ke http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-666 Ger ke Tue, 23 Jun 2009 15:22:45 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-666 Hi Andrew, Great post, thanks for this! I am experiencing the same problem. I am in contact with my ISP now and they want to know the google network range so they can whitelist it. Do you know how I can provide them with this? Hi Andrew, Great post, thanks for this! I am experiencing the same problem. I am in contact with my ISP now and they want to know the google network range so they can whitelist it. Do you know how I can provide them with this?

]]> By: Jasper http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-649 Jasper Sun, 01 Mar 2009 20:49:35 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-649 Hey Andrew...thank you very much for your excellent advice. I am definitely going to take it... starting right now. The latest update from my current host, HostGator, is that they suggest I try to verify my site from a different location using a different ISP. They suggest that maybe my current ISP, Verizon, is dropping packets in the transfer. I suspect they believe this because they do not block any bots or spiders coming from Google IPs, and they have been able to verify my sites without an issue. I have three PCs at home--two that connect wirelessly and one direct-connected. I've tried all three PCs and used both Firefox and IE Explorer, all without success. The odd thing is that a month ago, I had no problem verifying my sites in Webmaster Tools. And a strange new update is this: HostGator verified several of my sites for me, and these sites showed 'no errors' for the sitemap.xml. Google has since spidered the file and now reports errors. Some of these sites only have two or three pages, and I had rebuilt and resent the sitemap.xml file prior to HostGator verifying it, so this can't be a problem on my end. This has to be some problem in communication occurring between Google and HostGator. Well, anyway...I'm going to test the waters with a new host, as per your suggestion, and see how that goes. This thread, by the way, is excellent and I really appreciate your rapid feedback, Andrew! Much appreciated, mate! Hey Andrew...thank you very much for your excellent advice. I am definitely going to take it... starting right now. The latest update from my current host, HostGator, is that they suggest I try to verify my site from a different location using a different ISP. They suggest that maybe my current ISP, Verizon, is dropping packets in the transfer. I suspect they believe this because they do not block any bots or spiders coming from Google IPs, and they have been able to verify my sites without an issue. I have three PCs at home--two that connect wirelessly and one direct-connected. I've tried all three PCs and used both Firefox and IE Explorer, all without success. The odd thing is that a month ago, I had no problem verifying my sites in Webmaster Tools. And a strange new update is this: HostGator verified several of my sites for me, and these sites showed 'no errors' for the sitemap.xml. Google has since spidered the file and now reports errors. Some of these sites only have two or three pages, and I had rebuilt and resent the sitemap.xml file prior to HostGator verifying it, so this can't be a problem on my end. This has to be some problem in communication occurring between Google and HostGator. Well, anyway...I'm going to test the waters with a new host, as per your suggestion, and see how that goes. This thread, by the way, is excellent and I really appreciate your rapid feedback, Andrew! Much appreciated, mate!

]]> By: Andrew http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-647 Andrew Sun, 01 Mar 2009 14:00:55 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-647 Wow Jasper... 25 blogs and no visitors. That is really the pits. To be fair, I have NEVER met a System Admin who admits the fault could be on their end until they exhaust all possible options. By then, it may be too late to say "I told you so". Let's get to down to your prob. If I were you, I would first transfer ONE or TWO domain(s) out to any other host. Then you know the drill. Put out some links or get your pals to put out some links pointing to your site. Hard as it may be, sit tight and wait for Googlebot. Continue to monitor the situation for a couple of days (yeah... it's nerve wracking). If you're sure that Googlebot is spidering your (transfered) blog without problems, then chances are you are right and there is a problem over at your present host that they may be unable (or unwilling) to change. However, IF you still can't get Googlebot to visit after your domain transfer, you will need to check a 1001 other things that may have gone wrong. As far as I'm concerned (from the way they handled my problem above), my present webhost is as reliable and professional as I can expect. You might want to give them a try. Your 2 options are : 1. The banner on the right leading to Exabytes.com. This is an affiliate ad. If you prefer not to go via my affiliate link, feel free to 2. Go direct to http://www.exabytes.com I really hope you get out of this mess soon Jasper! Wow Jasper... 25 blogs and no visitors. That is really the pits. To be fair, I have NEVER met a System Admin who admits the fault could be on their end until they exhaust all possible options. By then, it may be too late to say "I told you so".

Let's get to down to your prob. If I were you, I would first transfer ONE or TWO domain(s) out to any other host. Then you know the drill. Put out some links or get your pals to put out some links pointing to your site. Hard as it may be, sit tight and wait for Googlebot. Continue to monitor the situation for a couple of days (yeah... it's nerve wracking). If you're sure that Googlebot is spidering your (transfered) blog without problems, then chances are you are right and there is a problem over at your present host that they may be unable (or unwilling) to change. However, IF you still can't get Googlebot to visit after your domain transfer, you will need to check a 1001 other things that may have gone wrong.

As far as I'm concerned (from the way they handled my problem above), my present webhost is as reliable and professional as I can expect. You might want to give them a try. Your 2 options are :

1. The banner on the right leading to Exabytes.com. This is an affiliate ad. If you prefer not to go via my affiliate link, feel free to

2. Go direct to http://www.exabytes.com

I really hope you get out of this mess soon Jasper!

]]> By: Jasper http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-646 Jasper Sun, 01 Mar 2009 08:25:16 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-646 Andrew, this thread has been my living nightmare for the past 10 days. I have axperienced all the problems above and then some. I have done everything to try and get Google and my miserable host--HostGator--to play nicely and help me work through a solution, but they're acting like spoiled brats, blaming each other and pointing the finger. I'm at the end of my rope and I want OUT!!!! I cannot run my business this way. I do not want to spend the next week teaching HostGator how to do their job. I have simply given up and want an new Host. PLEASE!!!! Tell me a Host that most people like and has already resolved the issues illustrated here. If anyone knows a decent host, please let me know. I've got 25 blogs that are sitting there with no traffic because Googlebot can't access my plain-as-day, well-written sitemap.xml file (server time-outs) and breaks when it sees my simple, generic Google-supplied robots.txt file. Desperate in Boston. Andrew, this thread has been my living nightmare for the past 10 days. I have axperienced all the problems above and then some. I have done everything to try and get Google and my miserable host--HostGator--to play nicely and help me work through a solution, but they're acting like spoiled brats, blaming each other and pointing the finger. I'm at the end of my rope and I want OUT!!!! I cannot run my business this way. I do not want to spend the next week teaching HostGator how to do their job. I have simply given up and want an new Host. PLEASE!!!! Tell me a Host that most people like and has already resolved the issues illustrated here. If anyone knows a decent host, please let me know. I've got 25 blogs that are sitting there with no traffic because Googlebot can't access my plain-as-day, well-written sitemap.xml file (server time-outs) and breaks when it sees my simple, generic Google-supplied robots.txt file.

Desperate in Boston.

]]> By: Google Can't find my site - Thirty Day Challenge Forums http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-633 Google Can't find my site - Thirty Day Challenge Forums Thu, 08 Jan 2009 05:04:18 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-633 [...] may have to contact your web hosting provider but read this first Debugging The Network Unreachable / Robots. txt Unreachable Error - HomeWithAndrew.com __________________ Devanand [...] [...] may have to contact your web hosting provider but read this first Debugging The Network Unreachable / Robots. txt Unreachable Error - HomeWithAndrew.com __________________ Devanand [...]

]]> By: Andrew http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-614 Andrew Wed, 26 Nov 2008 09:54:56 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-614 hi hard money... glad you found the post helpful. I now make it a habit to visit Webmaster Tools to check my sites' status and possible errors. Doing that daily is a heck of a lot better than getting one gigantic heart attack when traffic suddenly stops! Adsense doesn't do too well on HomeWithAndrew.com so I pulled them out long ago. But your kind intentions have made me think about putting a "tipping jar" script so's nice folks llike you can buy me a Big Mac! Cheers! hi hard money... glad you found the post helpful.

I now make it a habit to visit Webmaster Tools to check my sites' status and possible errors. Doing that daily is a heck of a lot better than getting one gigantic heart attack when traffic suddenly stops!

Adsense doesn't do too well on HomeWithAndrew.com so I pulled them out long ago. But your kind intentions have made me think about putting a "tipping jar" script so's nice folks llike you can buy me a Big Mac! Cheers!

]]> By: hard money http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-613 hard money Wed, 26 Nov 2008 08:15:49 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-613 Hey Andrew, Your post really helped me! I have a lot of sites and was getting the exact "network unreachabel" and "sitemap error bwecause of robots.txt" that you mentioned. I just forwarded this url to my webhost and hopefully they can fix it in their firewall rules. Your analysis was spot on and I really appreciate you spelling it all out. If you had some adsense ads on here I'd click them just to make you some money, but alas you only have the one webhosting ad and I've already got that... ;) Hey Andrew,

Your post really helped me! I have a lot of sites and was getting the exact "network unreachabel" and "sitemap error bwecause of robots.txt" that you mentioned. I just forwarded this url to my webhost and hopefully they can fix it in their firewall rules. Your analysis was spot on and I really appreciate you spelling it all out. If you had some adsense ads on here I'd click them just to make you some money, but alas you only have the one webhosting ad and I've already got that... ;)

]]> By: Andrew http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/comment-page-1/#comment-590 Andrew Sun, 12 Oct 2008 22:24:54 +0000 http://www.homewithandrew.com/index.php/debugging-the-network-unreachable-robots-txt-unreachable-error/#comment-590 Hi Avery... There is a difference between an ordinary sitemap and an XML sitemap. An ordinary sitemap is just an ordinary (HTML) page that list out the links to subsections and pages. An "XML sitemap" would normally be referred to as an RSS feed. This is a page that conforms to a specific format usually written in XML. You can learn more about creating your own RSS XML feed if you Google "how to create your own xml rss feed". This RSS feed is basically used by RSS aggregators to "feed" web users with your content without them having to actually visit your site. In order to do this, web users will need to use an RSS reader. You can Google "RSS readers" to learn more about RSS feeds. As far as you site is concerned, I searched for "snipersuitghillie" and found 38 entries in Google, with your Home page at PR4. This means that you HAVE indeed been indexed by Google, so you are doing something right! Don't worry. Getting the hang of SEO and Google takes some time. All the best. Hi Avery...
There is a difference between an ordinary sitemap and an XML sitemap. An ordinary sitemap is just an ordinary (HTML) page that list out the links to subsections and pages. An "XML sitemap" would normally be referred to as an RSS feed. This is a page that conforms to a specific format usually written in XML. You can learn more about creating your own RSS XML feed if you Google "how to create your own xml rss feed". This RSS feed is basically used by RSS aggregators to "feed" web users with your content without them having to actually visit your site. In order to do this, web users will need to use an RSS reader. You can Google "RSS readers" to learn more about RSS feeds.

As far as you site is concerned, I searched for "snipersuitghillie" and found 38 entries in Google, with your Home page at PR4. This means that you HAVE indeed been indexed by Google, so you are doing something right! Don't worry. Getting the hang of SEO and Google takes some time.

All the best.

]]>