Comments on: Debugging The Network Unreachable / Robots. txt Unreachable Error

By: john

john — Mon, 20 Dec 2010 21:04:03 +0000

does anyone know the exact time seonetlinks is coming out? i heard it's an amazing way to generate tons of traffic

By: Erin Sparks

Erin Sparks — Mon, 29 Nov 2010 21:48:22 +0000

This information is incredibly helpful. Can you provide any sites that test out a specific subdomain's issues?

By: Flannel Sheets

Flannel Sheets — Wed, 13 Oct 2010 18:25:43 +0000

i always sign up on SEO forums because i want to learn more about SEO and internet marketing"*"

By: private hard money lenders

private hard money lenders — Wed, 11 Aug 2010 15:21:01 +0000

@Valerie, there are so many search engine optimization forums out there that could help you in keyword research.

By: Xavier Watson

Xavier Watson — Thu, 24 Jun 2010 04:47:14 +0000

What is the best SEO Forum these days?'~"

By: Salena Amigon

Salena Amigon — Tue, 15 Jun 2010 14:59:15 +0000

Observed your site on del.icio.us these daysand really liked it.. i bookmarked it and will likely be back again to catchit out some more later ..

By: Yohan Perera

Yohan Perera — Thu, 13 May 2010 03:23:18 +0000

Actually your article helped me to solve a different kind of problem.

I am using Pingdom to monitor my blog. Since the day I started using Pingdom with it’s default 5 minutes check, I started to receive frequent down alerts for my blog . I checked with the hosting company several times but they denied any down time .

Later on I found out that the firewall guarding my web server has a automatic mechanism which blocks the IP address of any outside host that makes concurrent connections in a short period of time .

Since I was using 5 minutes check resolution Pingdom was making concurrent connections to my web server once in every 5 minutes. The firewall misunderstood this to be a suspicious act and blocked pingdom frequently.

Pingdom had no idea that it’s being blocked by a firewall and can’t get to the site. Thus reported to me as the site is down .

By: Hridoy

Hridoy — Thu, 11 Mar 2010 18:05:10 +0000

This post helped me a lot to see solve a problem related to robots.txt. Thanks

By: Sharath

Sharath — Thu, 31 Dec 2009 18:43:25 +0000

I too had the same problem. I added the following meta tags to all my pages and it all worked out fine. meta name="GOOGLEBOT" content="INDEX, FOLLOW" / meta name="robots" content="noodp,index,follow" / meta name="revisit-after" content="7 days" /

By: Sharath

Sharath — Thu, 31 Dec 2009 18:41:10 +0000

Hi, Nice post. I too had a similar problem. I added the following Meta Tags on all my pages and it worked out fine.

By: Engineering Colleges in Punjab

Engineering Colleges in Punjab — Sat, 17 Oct 2009 05:01:10 +0000

to my mind, network unreachable problem means when google bots tries to make so many concurrent connection, website server temporarily disallows access to them. then google reports it as network unreachable.

By: Arthur Coleman

Arthur Coleman — Wed, 16 Sep 2009 18:07:29 +0000

Great post. I am experiencing the exact same problem with only a few of my pages (as compared to all pages, which makes it a bit more difficult to debog) and this was very helpful.

By: ankit

ankit — Tue, 08 Sep 2009 04:13:30 +0000

I was facing same error and i also contacted my host..but they said they are not blocking any IP range..i found that due to my default wp privacy settings..it was blocking search engines to crawl my site..but i changed that option to allow all search engines to crawl my site..and other seach engines like yahoo, bing are crawling too but not google..even google ads are not showing properly..they are showing public service ads on my site ..where as my content is original and its not adult too..

By: Search Engine Optimization Dallas - Fixing my Google Sitemap Errors so I show up in the search engine results | Swanson SEO Services Blog

Fri, 21 Aug 2009 20:27:03 +0000

[...] Debugging the Network unreachable / Robots.txt unreachable error [...]

By: Sitemap Google webmaster tool « MaseBlog

Sitemap Google webmaster tool « MaseBlog — Mon, 03 Aug 2009 14:34:40 +0000

[...] aver letto un interessante post sull’argomento datato però 2007 ho trovato diverse risposte utili all’interno del forum di google ed in [...]

By: Ger ke

Ger ke — Tue, 23 Jun 2009 15:22:45 +0000

Hi Andrew, Great post, thanks for this! I am experiencing the same problem. I am in contact with my ISP now and they want to know the google network range so they can whitelist it. Do you know how I can provide them with this?

By: Jasper

Jasper — Sun, 01 Mar 2009 20:49:35 +0000

Hey Andrew...thank you very much for your excellent advice. I am definitely going to take it... starting right now. The latest update from my current host, HostGator, is that they suggest I try to verify my site from a different location using a different ISP. They suggest that maybe my current ISP, Verizon, is dropping packets in the transfer. I suspect they believe this because they do not block any bots or spiders coming from Google IPs, and they have been able to verify my sites without an issue. I have three PCs at home--two that connect wirelessly and one direct-connected. I've tried all three PCs and used both Firefox and IE Explorer, all without success. The odd thing is that a month ago, I had no problem verifying my sites in Webmaster Tools. And a strange new update is this: HostGator verified several of my sites for me, and these sites showed 'no errors' for the sitemap.xml. Google has since spidered the file and now reports errors. Some of these sites only have two or three pages, and I had rebuilt and resent the sitemap.xml file prior to HostGator verifying it, so this can't be a problem on my end. This has to be some problem in communication occurring between Google and HostGator. Well, anyway...I'm going to test the waters with a new host, as per your suggestion, and see how that goes. This thread, by the way, is excellent and I really appreciate your rapid feedback, Andrew! Much appreciated, mate!

By: Andrew

Andrew — Sun, 01 Mar 2009 14:00:55 +0000

Wow Jasper... 25 blogs and no visitors. That is really the pits. To be fair, I have NEVER met a System Admin who admits the fault could be on their end until they exhaust all possible options. By then, it may be too late to say "I told you so".

Let's get to down to your prob. If I were you, I would first transfer ONE or TWO domain(s) out to any other host. Then you know the drill. Put out some links or get your pals to put out some links pointing to your site. Hard as it may be, sit tight and wait for Googlebot. Continue to monitor the situation for a couple of days (yeah... it's nerve wracking). If you're sure that Googlebot is spidering your (transfered) blog without problems, then chances are you are right and there is a problem over at your present host that they may be unable (or unwilling) to change. However, IF you still can't get Googlebot to visit after your domain transfer, you will need to check a 1001 other things that may have gone wrong.

As far as I'm concerned (from the way they handled my problem above), my present webhost is as reliable and professional as I can expect. You might want to give them a try. Your 2 options are :

1. The banner on the right leading to Exabytes.com. This is an affiliate ad. If you prefer not to go via my affiliate link, feel free to

2. Go direct to http://www.exabytes.com

I really hope you get out of this mess soon Jasper!

By: Jasper

Jasper — Sun, 01 Mar 2009 08:25:16 +0000

Andrew, this thread has been my living nightmare for the past 10 days. I have axperienced all the problems above and then some. I have done everything to try and get Google and my miserable host--HostGator--to play nicely and help me work through a solution, but they're acting like spoiled brats, blaming each other and pointing the finger. I'm at the end of my rope and I want OUT!!!! I cannot run my business this way. I do not want to spend the next week teaching HostGator how to do their job. I have simply given up and want an new Host. PLEASE!!!! Tell me a Host that most people like and has already resolved the issues illustrated here. If anyone knows a decent host, please let me know. I've got 25 blogs that are sitting there with no traffic because Googlebot can't access my plain-as-day, well-written sitemap.xml file (server time-outs) and breaks when it sees my simple, generic Google-supplied robots.txt file.

Desperate in Boston.

By: Google Can't find my site - Thirty Day Challenge Forums

Google Can't find my site - Thirty Day Challenge Forums — Thu, 08 Jan 2009 05:04:18 +0000

[...] may have to contact your web hosting provider but read this first Debugging The Network Unreachable / Robots. txt Unreachable Error - HomeWithAndrew.com __________________ Devanand [...]