How to reduce dns lookup time?

If you are writing a program to resolve massive domain names such as an email address verifier, you must have headache about the dns lookup time. The dns lookup is very slow by default. The resolve of a domain name may only timeout after 30 seconds, which is intolerable. How to speedup the lookup process? There are some options in /etc/resolv.conf that control the dns lookup process. You can adjust the time spent in resolving the names with the timeout and attempts parameters.

The default value for timeout is 5(seconds). The default value for attempts is 2 (times). What do timeout and attempts mean? Let’s see first what the shit manual doc says about the two options:

Sets the amount of time the resolver will wait for a
response from a remote name server before retrying the
query via a different name server. This may not be the
total time taken by any resolver API call and there is
no guarantee that a single resolver API call maps to a
single timeout.  Measured in seconds, the default is
RES_TIMEOUT (currently 5, see <resolv.h>).  The value
for this option is silently capped to 30.

Sets the number of times the resolver will send a query
to its name servers before giving up and returning an
error to the calling application.  The default is
RES_DFLRETRY (currently 2, see <resolv.h>).  The value
for this option is silently capped to 5.

What is it talking about? The resolver waits for timeout before retrying the second server, then if the second server fails to response, retries the first server the second time, then retries the second server again, and repeats the process? The doc is trying to confuse you about the real meaning of timeout and attempts. You can’t understand what timeout and attempts mean unless by digging into the source code of a resolver. In fact, the concept of timeout and attempts is very simple: the resolver queries the first dns server and wait its response for timeout seconds, if it does not get the response in timeout seconds, it queries the first server again. The process is repeated for attempts times and if it still can’t get the response, it fails over to the second server and queries the second server for attempts times as it did for the first server. So in our example, the timeout would be 1*1*2=2 seconds, while it takes 5*2*2=20 seconds to timeout originally. (see also The options parameters can be written on multiple lines as follows:



Posted in tips of hosting