[HOWTO] - Speed up DNS lookups in OSX

[Darkshadow]

This will explain how to speed up DNS lookups. This is useful for contacting servers much quicker than normal, plus not waiting around forever before a lookup fails. I would just like to note at the beginning here that this will not load web pages any faster, or let you download anything faster, but it will contact web servers & such much, much quicker than the default.

———————————————
Part 1 - Speeding up DNS lookups in NetInfo

I will be giving the commands to do this for the terminal. All of the commands I list must be run as root. Either use su to log in as root, or append sudo to the beginning of the commands.

First, we'll create the locations/lookupd "folder" that NetInfo uses (actually, lookupd uses, but anyway...). Note that if this already exists, it won't overwrite anything. This is just in case it doesn't exist yet.

niutil -create . /locations/lookupd

Next, we'll create another location for hosts:

niutil -create . /locations/lookupd/hosts

Next, we add some parameters for hosts - this will basically be telling lookupd to not validate the cache for hosts, and to time out after 30 seconds if there is no response:

niutil -createprop . /locations/lookupd/hosts LookupOrder CacheAgent NIAgent DNSAgent NILAgent
niutil -createprop . /locations/lookupd/hosts Timeout 30
niutil -createprop . /locations/lookupd/hosts ValidateCache NO

Now we create a few more locations for the agents DNSAgent and NILAgent. DNSAgent is the DNS lookup, of course, and NILAgent is sort of a negative reply agent - any search that doesn't come up with a result will be remembered by NILAgent, and if another search is done for the same host, item, whatever else, NILAgent will kick and and pretty much say "Don't bother, it doesn't exist." Meaning you won't have to sit there and wait for it to timeout again. Here's the stuff for this:

niutil -create . /locations/lookupd/agents
niutil -create . /locations/lookupd/agents/DNSAgent
niutil -create . /locations/lookupd/agents/NILAgent

Ok, now we'll add in the parameters for NILAgent first. These tell it to keep an item in its database for 1 hour, and for the agent itself to timeout after 30 seconds:

niutil -createprop . /locations/lookupd/agents/NILAgent TimeToLive 3600
niutil -createprop . /locations/lookupd/agents/NILAgent Timeout 30

Finally, we'll do the parameters for DNSAgent. This tells the agent to timeout after 15 seconds, and to retry a lookup 3 times:

niutil -createprop . /locations/lookupd/agents/DNSAgent Timeout 15
niutil -createprop . /locations/lookupd/agents/DNSAgent Retries 3


And that's that. Now do this command to restart NetInfo and lookupd:

/System/Library/SystemConfiguration/Kicker.bundle/Resources/restart-NetInfo

———————————————
Part 2 - Creating a caching-only nameserver

This next part describes how to create your own nameserver. Note that this isn't a fully-fledged nameserver, it only caches servers you connect to for a short while, it doesn't provide a DNS server for your own domain. You could do this, yes, but you would still need to apply for a domain, have a static IP number, yadda yadda yadda...and none of that is needed for this anyway.


Why create your own nameserver? Well, having your own nameserver on your own machine greatly shortens the time spent on DNS lookups. Plus, you aren't one of many accessing the DNS server, you are the only one.

You won't need to install any additional software to get this working, everything needed is already installed. As with the first part, these will all be terminal commands (except for one small step), and you will need to be root/use sudo as before to edit/create these files.

First we'll edit the file /etc/named.conf. Named is the program that runs the nameserver, FYI. Enter this command, then enter the text as shown:

pico /etc/named.conf

// Config file for caching only name server

options { 
            directory "/var/named";

            // Uncomment next line if you are behind a firewall of any sort
            // query-source port 53;
};

zone "." {
            type hint;
            file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
            type master;
            file "pz/127.0.0";
}; 

(type control-o to save it, then control-x to exit pico)

This file tells named where to find all the stuff it needs to set up the nameserver.

Next we need to create the folder named will look into to find it's config files:

mkdir /var/named

Now we travel to the newly created directory and create another one:

cd /var/named
mkdir pz

Now, we need to create a file for named to use to do its DNS lookups. This is the file called root.hints. These are the root servers that supply DNS lookups to the majority of DNS servers out there. Note that you will need to maintain this file, I'll explain that after we set it up. You will need to be online to do these commands:

first type dig to get the file according to your current DNS server. See all those lines that start with some letter (i.e. A or G or M) with .ROOT-SERVERS.NET. after it? You need to query one to get the root hints file. After you pick one, do this command (replace the root server I listed with the one you choose)

dig @m.root-server.net . ns > root.hints

This will create the root.hints file. Now, like I mentioned, you will need to maintain this file. The reason is because the root servers do change from time to time, and you will need to keep up to date with the changes. Maintaining it is just redoing the above commands to create a new root.hints file (done once a month).

Next we'll go into the pz folder we created and make a new file there. This is the file that loads in all the DNS info for you:

cd pz
pico 127.0.0

@     IN     SOA     ns.mynameserver.com. hostmaster.mynameserver.com (
                            1     ;  Serial
                            8H   ; Refresh
                            2H   ; Retry
                            1W  ; Expire
                            1D) ; Minimum TTL
               NS       ns.mynameserver.com.
1              PTR      localhost.
 

(again, control-o to save it, and control-x to exit pico)

Next we need to tell OS X to use itself as the DNS server. You do this via the Network Preferences in System Preferences. Load it up, click on the TCP/IP tab, and under Domain Name Servers, type in 127.0.0.1 Then click save at the bottom and quit.

Be sure to replace anything that was already there - we don't want to use any other DNS but our own.

Now we start named. The command for this is pretty simple.

ndc start

Now, we need to do one of two things to actually get the connection to use the new name server. You can either disconnect and reconnect, or if you would rather not disconnect, edit /var/run/resolv.conf. If you wish to do the latter, do

pico /var/run/resolv.conf

nameserver: 127.0.0.1

(control-o, control-x)

Be sure to replace anything else that was already in the resolv.conf.

Now you need to test it to make sure everything is working right. The easiest way is to use nslookup. Type that in, and the first two lines you should see are

Default Server: localhost
Address: 127.0.0.1

If it doesn't, then you have an error in the config files somewhere - go back and make sure they're typed in exactly as I have them.

The last thing we need to do is get named to launch at boot. You'll need named to start before you try to use your connection.

Good thing there's a (pretty) simple way - make a startup item for named.

Create a folder named Named in /Library/StartupItems (you may need to create the folder StartupItems in /Library first):

mkdir /Library/StartupItems/Named
cd /Library/StartupItems/Named

Now we create two files to start named at boot. One of the files is always named the same as the folder name - so if you create the folder with some other than Named, then save this file as that same name. Otherwise, do this:

pico Named

#!/bin/sh

. /etc/rc.common

if [ "${DNSSERVER:=-NO-}" = "-YES-" ]; then

ConsoleMessage "Starting named"

/usr/sbin/ndc start
fi

(control-o, control-x)

Now type chmod +x Named to make it executable (needed!).

Now we edit the other file used for startup items - StartupParameters.plist. This one is named the same for every startup item. Here's what to put in it:

pico StartupParameters.plist

{
Description = "caching-only nameserver";
Provides = ("named");
Requires = ("Disks", "Resolver", "Desktop DB", "Input Managers");
OrderPreference = "None";
Messages =
{
start = "Starting named...";
stop = "Stopping named...";
};
}

Now, one last little thing. Edit /etc/hostconfig, and add in this line:

DNSSERVER=-YES-

to the end of it..

[kilowatt]

Wow, thanks alot! Thats pretty darn cool.

I have a few questions, though.

I'm at a university, behind a firewall, and the dns server they defaultly assign me is behind it as well, its a 10.x.x.x host.

That internal dns server provides dns information about other subnetted hosts. Infact, if your computer's name is BigMac, your ip might resolve to BigMac.danh.uc.edu or something.

Anyway, I don't get this information anymore (Well, I added the internal dns server after 127.0.0.1 in the network control panel). Before I did that, ping BigMac.card.uc.edu would really ping it, but with just 127.0.0.1 in there it does not.

Is there anything I can do to make my personal dns server contain the information within the sub net's dns server?

Also, in the /var/named/pz/127.0.0 file, I see stuff like ns.mynameserver.com and hostmaster.mynameserver.com. Should I leave them like that, or change them (it just doesn't feel right...)

Thanks for the instructions, it was very helpful!

[Darkshadow]

You can add this to the named.conf file:

add into the options { } section

forward first;
forwarders {
     DNS server #1
     DNS server #2
     ...
     DNS server #n
};
 

This will query the DNS servers there as well. After changing the config file, type ndc reload and it should all be fine.

Adding the DNS server to the Network Prefs does pretty much the same thing as this, so you could do whichever you were most comfortable with.

You can leave the ns.mynameserver.com and hostmaster.mynameserver.com as they are if you wish, or change them to something else. Since this is a caching-only nameserver, and not a full on DNS, it doesn't matter overmuch. If it were a full on DNS, you would need to put your actual domain name there. Personally, I use dreamstatic.org, the name I've given my localhost.

And you're welcome. I really should have posted that sooner! I've actually been running that config for a few months now It wasn't until the network forum opened that I thought "Hey, other people would like doing this too..."

[Trip]

That was incredibly long.
But also very helpful! Thanks!

[boysimple]

First off: Excellent tutorial, thank you very much. i've been having so many problems with slow DNS lookups - just what I needed.

Also - if you have an 'always on' internet connections, and want to automate the updating of root.hints add this to the /etc/crontab file (it's 2 lines, may wrap a lil cause they are long lines):

# update DNS cache root.hints on the first of every month at noon                
0       12      1       *       *       root dig @h.root-servers.net . ns > /var/named/root.hints

of course, replace h.root-servers.net with the name of the server that you are using.

once the file is saved cron will restart itself when it sees the newer file date.

that's it. It'll take care of itself forever now.

E

[kilowatt]

Is the output from dig @h.root-servers.net. .ns supposed to look like this:
[user@mach4]$ dig @h.root-servers.net. .ns

; <<>> DiG 8.3 <<>> @h.root-servers.net. .ns
; (1 server found)
;; res_nmkquery: buffer too small

[user@mach4]$

Thats all I get when I run it into the file ( the &lt stuff).

[Darkshadow]

boysimple - yes, that's the location of Apple's startup items. However, they wish us third party people to place startup items in /Library/StartupItems. That's why I specified that location. And I did say that you may need to create the StatupItems folder in /Library. Plus you don't need to be root to add stuff to /Library, you can do it as your regular user.

I didn't tell people to set that up as a cron job because some people out there are using modem connections still (I happen to be one of the unlucky few), and if you're not online when the cron job runs, it won't get updated. Plus people don't always have their computer running when the cron job is supposed to be run, and it would be skipped, or if they used MacJanitor, they would inadvertantly be updating the root.hints file more than they need to. Which isn't a bad thing, but I try not to go for overkill.

Also, you don't need to restart cron after you change its config file. It checks its config file *every minute* and if the file has a modification date later than the one it last saw (a whole minute ago), it reloads the config file. You can see this for yourself by opening up the system log in the console, editing the /etc/crontab file, and waiting a minute. You'll get a message about cron reloading it.

kilowatt - You're almost there! It would be dig @h.root-servers.net. .[space]ns, not .ns You do get that stuff when you do it this way, but you also get the servers as well. The .ns without the space was confusing dig as to what you wanted, so it wasn't giving you anything.

[boysimple]

I stand corrected. Long may the shadow wander...

E