Comments on: Is this really asynchronous? http://blog.mischel.com/2008/06/10/is-this-really-asynchronous/ Musings on technology and life Tue, 06 Jan 2009 13:50:31 +0000 http://wordpress.org/?v=2.7 hourly 1 By: Darrin Chandler http://blog.mischel.com/2008/06/10/is-this-really-asynchronous/comment-page-1/#comment-2399 Darrin Chandler Wed, 11 Jun 2008 14:19:24 +0000 http://blog.mischel.com/?p=125#comment-2399 It's only obvious in hindsight of course. :-) This has to be one of the reasons all the big crawlers hit my site in one big run. Nothing for days, then they crawl pretty much the whole site. Because no matter how you slice it, the DNS lookup MUST be done and that takes time. It's true whether it's threaded, in the background, or both. So why not use the result while you have it? It’s only obvious in hindsight of course. :-)

This has to be one of the reasons all the big crawlers hit my site in one big run. Nothing for days, then they crawl pretty much the whole site. Because no matter how you slice it, the DNS lookup MUST be done and that takes time. It’s true whether it’s threaded, in the background, or both. So why not use the result while you have it?

]]>