[HOWTO] - Speed up DNS lookups in OSX

Darkshadow

wandering shadow
This will explain how to speed up DNS lookups. This is useful for contacting servers much quicker than normal, plus not waiting around forever before a lookup fails. I would just like to note at the beginning here that this will not load web pages any faster, or let you download anything faster, but it will contact web servers & such much, much quicker than the default.

———————————————
Part 1 - Speeding up DNS lookups in NetInfo

I will be giving the commands to do this for the terminal. All of the commands I list must be run as root. Either use su to log in as root, or append sudo to the beginning of the commands.

First, we'll create the locations/lookupd "folder" that NetInfo uses (actually, lookupd uses, but anyway...). Note that if this already exists, it won't overwrite anything. This is just in case it doesn't exist yet.

niutil -create . /locations/lookupd

Next, we'll create another location for hosts:

niutil -create . /locations/lookupd/hosts

Next, we add some parameters for hosts - this will basically be telling lookupd to not validate the cache for hosts, and to time out after 30 seconds if there is no response:

niutil -createprop . /locations/lookupd/hosts LookupOrder CacheAgent NIAgent DNSAgent NILAgent
niutil -createprop . /locations/lookupd/hosts Timeout 30
niutil -createprop . /locations/lookupd/hosts ValidateCache NO

Now we create a few more locations for the agents DNSAgent and NILAgent. DNSAgent is the DNS lookup, of course, and NILAgent is sort of a negative reply agent - any search that doesn't come up with a result will be remembered by NILAgent, and if another search is done for the same host, item, whatever else, NILAgent will kick and and pretty much say "Don't bother, it doesn't exist." Meaning you won't have to sit there and wait for it to timeout again. Here's the stuff for this:

niutil -create . /locations/lookupd/agents
niutil -create . /locations/lookupd/agents/DNSAgent
niutil -create . /locations/lookupd/agents/NILAgent

Ok, now we'll add in the parameters for NILAgent first. These tell it to keep an item in its database for 1 hour, and for the agent itself to timeout after 30 seconds:

niutil -createprop . /locations/lookupd/agents/NILAgent TimeToLive 3600
niutil -createprop . /locations/lookupd/agents/NILAgent Timeout 30

Finally, we'll do the parameters for DNSAgent. This tells the agent to timeout after 15 seconds, and to retry a lookup 3 times:

niutil -createprop . /locations/lookupd/agents/DNSAgent Timeout 15
niutil -createprop . /locations/lookupd/agents/DNSAgent Retries 3


And that's that. Now do this command to restart NetInfo and lookupd:

/System/Library/SystemConfiguration/Kicker.bundle/Resources/restart-NetInfo

———————————————
Part 2 - Creating a caching-only nameserver

This next part describes how to create your own nameserver. Note that this isn't a fully-fledged nameserver, it only caches servers you connect to for a short while, it doesn't provide a DNS server for your own domain. You could do this, yes, but you would still need to apply for a domain, have a static IP number, yadda yadda yadda...and none of that is needed for this anyway. ;)


Why create your own nameserver? Well, having your own nameserver on your own machine greatly shortens the time spent on DNS lookups. Plus, you aren't one of many accessing the DNS server, you are the only one.

You won't need to install any additional software to get this working, everything needed is already installed. As with the first part, these will all be terminal commands (except for one small step), and you will need to be root/use sudo as before to edit/create these files.

First we'll edit the file /etc/named.conf. Named is the program that runs the nameserver, FYI. Enter this command, then enter the text as shown:

pico /etc/named.conf

Code:
[color=red]// Config file for caching only name server

options { 
            directory "/var/named";

            // Uncomment next line if you are behind a firewall of any sort
            // query-source port 53;
};

zone "." {
            type hint;
            file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
            type master;
            file "pz/127.0.0";
};[/color]
(type control-o to save it, then control-x to exit pico)

This file tells named where to find all the stuff it needs to set up the nameserver.

Next we need to create the folder named will look into to find it's config files:

mkdir /var/named

Now we travel to the newly created directory and create another one:

cd /var/named
mkdir pz


Now, we need to create a file for named to use to do its DNS lookups. This is the file called root.hints. These are the root servers that supply DNS lookups to the majority of DNS servers out there. Note that you will need to maintain this file, I'll explain that after we set it up. You will need to be online to do these commands:

first type dig to get the file according to your current DNS server. See all those lines that start with some letter (i.e. A or G or M) with .ROOT-SERVERS.NET. after it? You need to query one to get the root hints file. After you pick one, do this command (replace the root server I listed with the one you choose)

dig @m.root-server.net . ns > root.hints

This will create the root.hints file. Now, like I mentioned, you will need to maintain this file. The reason is because the root servers do change from time to time, and you will need to keep up to date with the changes. Maintaining it is just redoing the above commands to create a new root.hints file (done once a month).

Next we'll go into the pz folder we created and make a new file there. This is the file that loads in all the DNS info for you:

cd pz
pico 127.0.0


Code:
[color=red]
@     IN     SOA     ns.mynameserver.com. hostmaster.mynameserver.com (
                            1     ;  Serial
                            8H   ; Refresh
                            2H   ; Retry
                            1W  ; Expire
                            1D) ; Minimum TTL
               NS       ns.mynameserver.com.
1              PTR      localhost.
[/color]
(again, control-o to save it, and control-x to exit pico)

Next we need to tell OS X to use itself as the DNS server. You do this via the Network Preferences in System Preferences. Load it up, click on the TCP/IP tab, and under Domain Name Servers, type in 127.0.0.1 Then click save at the bottom and quit.

Be sure to replace anything that was already there - we don't want to use any other DNS but our own.

Now we start named. The command for this is pretty simple.

ndc start

Now, we need to do one of two things to actually get the connection to use the new name server. You can either disconnect and reconnect, or if you would rather not disconnect, edit /var/run/resolv.conf. If you wish to do the latter, do

pico /var/run/resolv.conf

nameserver: 127.0.0.1

(control-o, control-x)

Be sure to replace anything else that was already in the resolv.conf.

Now you need to test it to make sure everything is working right. The easiest way is to use nslookup. Type that in, and the first two lines you should see are

Default Server: localhost
Address: 127.0.0.1

If it doesn't, then you have an error in the config files somewhere - go back and make sure they're typed in exactly as I have them.

The last thing we need to do is get named to launch at boot. You'll need named to start before you try to use your connection.

Good thing there's a (pretty) simple way - make a startup item for named.

Create a folder named Named in /Library/StartupItems (you may need to create the folder StartupItems in /Library first):

mkdir /Library/StartupItems/Named
cd /Library/StartupItems/Named


Now we create two files to start named at boot. One of the files is always named the same as the folder name - so if you create the folder with some other than Named, then save this file as that same name. Otherwise, do this:

pico Named

#!/bin/sh

. /etc/rc.common

if [ "${DNSSERVER:=-NO-}" = "-YES-" ]; then

ConsoleMessage "Starting named"

/usr/sbin/ndc start
fi


(control-o, control-x)

Now type chmod +x Named to make it executable (needed!).

Now we edit the other file used for startup items - StartupParameters.plist. This one is named the same for every startup item. Here's what to put in it:

pico StartupParameters.plist

{
Description = "caching-only nameserver";
Provides = ("named");
Requires = ("Disks", "Resolver", "Desktop DB", "Input Managers");
OrderPreference = "None";
Messages =
{
start = "Starting named...";
stop = "Stopping named...";
};
}


Now, one last little thing. Edit /etc/hostconfig, and add in this line:

DNSSERVER=-YES-

to the end of it..
 

kilowatt

mach-o mach-o man
Wow, thanks alot! Thats pretty darn cool.

I have a few questions, though.

I'm at a university, behind a firewall, and the dns server they defaultly assign me is behind it as well, its a 10.x.x.x host.

That internal dns server provides dns information about other subnetted hosts. Infact, if your computer's name is BigMac, your ip might resolve to BigMac.danh.uc.edu or something.

Anyway, I don't get this information anymore (Well, I added the internal dns server after 127.0.0.1 in the network control panel). Before I did that, ping BigMac.card.uc.edu would really ping it, but with just 127.0.0.1 in there it does not.

Is there anything I can do to make my personal dns server contain the information within the sub net's dns server?

Also, in the /var/named/pz/127.0.0 file, I see stuff like ns.mynameserver.com and hostmaster.mynameserver.com. Should I leave them like that, or change them (it just doesn't feel right...)

Thanks for the instructions, it was very helpful!
 

Darkshadow

wandering shadow
You can add this to the named.conf file:

add into the options { } section

Code:
[color=red]forward first;
forwarders {
     [i]DNS server #1[/i]
     [i]DNS server #2[/i]
     ...
     [i]DNS server #n[/i]
};
[/color]
This will query the DNS servers there as well. After changing the config file, type ndc reload and it should all be fine.

Adding the DNS server to the Network Prefs does pretty much the same thing as this, so you could do whichever you were most comfortable with.

You can leave the ns.mynameserver.com and hostmaster.mynameserver.com as they are if you wish, or change them to something else. Since this is a caching-only nameserver, and not a full on DNS, it doesn't matter overmuch. If it were a full on DNS, you would need to put your actual domain name there. Personally, I use dreamstatic.org, the name I've given my localhost.

And you're welcome. I really should have posted that sooner! I've actually been running that config for a few months now :D It wasn't until the network forum opened that I thought "Hey, other people would like doing this too..."
 

boysimple

Some guy
First off: Excellent tutorial, thank you very much. i've been having so many problems with slow DNS lookups - just what I needed.

Also - if you have an 'always on' internet connections, and want to automate the updating of root.hints add this to the /etc/crontab file (it's 2 lines, may wrap a lil cause they are long lines):

Code:
# update DNS cache root.hints on the first of every month at noon                
0       12      1       *       *       root dig @h.root-servers.net . ns > /var/named/root.hints
of course, replace h.root-servers.net with the name of the server that you are using.

once the file is saved cron will restart itself when it sees the newer file date.

that's it. It'll take care of itself forever now. :)

E
 

kilowatt

mach-o mach-o man
Is the output from dig @h.root-servers.net. .ns supposed to look like this:
[user@mach4]$ dig @h.root-servers.net. .ns

; <<>> DiG 8.3 <<>> @h.root-servers.net. .ns
; (1 server found)
;; res_nmkquery: buffer too small

[user@mach4]$

Thats all I get when I run it into the file ( the &lt stuff).
 

Darkshadow

wandering shadow
boysimple - yes, that's the location of Apple's startup items. However, they wish us third party people to place startup items in /Library/StartupItems. That's why I specified that location. And I did say that you may need to create the StatupItems folder in /Library. Plus you don't need to be root to add stuff to /Library, you can do it as your regular user.

I didn't tell people to set that up as a cron job because some people out there are using modem connections still (I happen to be one of the unlucky few), and if you're not online when the cron job runs, it won't get updated. Plus people don't always have their computer running when the cron job is supposed to be run, and it would be skipped, or if they used MacJanitor, they would inadvertantly be updating the root.hints file more than they need to. Which isn't a bad thing, but I try not to go for overkill. :D

Also, you don't need to restart cron after you change its config file. It checks its config file *every minute* and if the file has a modification date later than the one it last saw (a whole minute ago), it reloads the config file. You can see this for yourself by opening up the system log in the console, editing the /etc/crontab file, and waiting a minute. You'll get a message about cron reloading it.

kilowatt - You're almost there! It would be dig @h.root-servers.net. .[space]ns, not .ns You do get that stuff when you do it this way, but you also get the servers as well. The .ns without the space was confusing dig as to what you wanted, so it wasn't giving you anything.
 

beef

Dinner
First of all, thanks for the tip...

everything's working... but when I reboot, it kinda gets stuck at "Starting Directory Services". The same thing happened when I was attempting to make changes to netinfo database before, so I guess it's for the same reason...

Anyway... it got stuck there long enough for me to do the dishes...

I don't need to reboot all that much, so it's not a big problem, but it's rather annoying when you have to go back to 9 then back to X, etc...

Am I the only one experiencing this? If anyone knows the solution, let me know.
 

Darkshadow

wandering shadow
Hmm...Directory Services is what starts NetInfo and lookupd. It sounds like you may have a config problem.

Can you go to the terminal and type nidump -r / / and send me the text you get back from that? It spouts out your entire NetInfo domain. You can edit it to delete the user stuff from it - this will dump all your users, and while the passwords are encrypted, you probably don't want to send that part out.

Ya can either post it here, or PM me, or send it to me in an email. I'll take a look and see if I can figure out what may be going wrong.
 

~vert

It's short for vertigo...
Darkshadow, Try as I might, I can't get past the "nslookup" stage. It always returns:

[localhost:/var/named/pz] root# nslookup
Default Server: localhost
Address: 0.0.0.0


I also wonder if you have a typo in the directions. At the stage where it says:
niutil createprop . /locations/lookupd/agents/NILAgent TimeToLive 3600
niutil createprop. /locations/lookupd/agents/NILAgent Timeout 30
Should the second line read "createprop[space]. /locations..."? I tried it both ways but neither solved my problem.

Since I don't know what to do next, is there any harm in leaving all the config files and folders? Do I need to clean up to make sure that things work as smoothly as possible?

Thanks for the tutorial, wish it worked on my setup!!
 

beef

Dinner
well... sorta typo in the instruction, I guess.. but it's long, bound to have a typo here and there...

some of the lines are missing "-" in front of the option "createprop"
 

Darkshadow

wandering shadow
Whoops...you're right, that's a typo. I'll fix it - I'll also fix those missing "-"'s.

~vert - Hmm...did you restart NetInfo after the first part, and either disconnect/reconnect or edit the /var/run/resolv.conf file? You'll need to do both of those before it will work. If those don't fix it, go back and make sure that everything is exactly right. You can view the stuff you added into NetInfo by doing this: nidump -r /locations/lookupd / - that'll print out everything that you've added. Make sure everything is right there. If it is, then go and check all the config files you made for named. Make sure it's all correctly written down.

You can also open up Console.app and type ndc reload in the terminal. You'll see named starting up. If it spouts out any errors, check out what it tells you (it should inform you what part of the config files it's chocking on).
 

~vert

It's short for vertigo...
Darkshadow - It seems that the typos were the problem, although I also found there may be one in the dig command. When I run dig I get:

C.ROOT-SERVERS.NET. ...[et al]

'SERVERS' is plural. I also found that it didn't work unless I typed the part shown above in capitals. I didn't think that would matter, but it did in my case.

Regardless, it is up and working - Thanks for your assistance! It's fun having stuff to mess with like this in a Mac. I'm beginning to become an 'aesthetic geek', to coin a phrase...

I also tried editing the crontab file, so we will see how that goes.
 

beef

Dinner
hmm...

I just did

dig @a.root-servers.net. ns > file1
dig @A.ROOT-SERVERS.NET. ns > file2

I don't see any difference in the files besides the query time and date.
 

beef

Dinner
well, it's been awhile... anyway, I recently upgraded my Powerbook to 10.1, so I decided to give this tip a try on that machine.

(Note: I had problem with this on my desktop. It worked, but "Starting Directory Services" took hell of a wrong time...)

Anyway, to make the story short, as we all know, there's nothing wrong with the instruction given here, and it worked without a problem on my PB.

I wondered what the difference between my PB and DT is, and well, basically, I've made modifications to Netinfo on my DT system.

I had added some info under "machines", and although it was working, it seems that I did it in a wrong manner, and that didn't go well with the changes I made following this tip.

So I removed all the changes I made, then tried this again, and now it works great on both machines.

Looks like I need to spend more time reading about Netinfo...
 

zootbobbalu

Registered
After following this trick to speed up DNS lookups, none of my Classic apps can see my internet connection.

Anyone else having this problem?
 

zootbobbalu

Registered
Setting up OS X to be a DNS might disable Classic mode's ability to connect to the internet. Since I didn't do much internet stuff in Classic mode, this didn't bother me until I wanted to use AIM again. AIM and Netscape in Classic mode wouldn't connect to the internet. This drove me crazy for about an hour. Apple helped me figure out that all my settings were ok, but still no solution. I remembered that the last thing I tinkered with related to my internet settings was speeding up my DNS loopups with the tip posted here. The only thing that Classic mode doesn't like about this hack is adding your own computer as a DNS (127.0.0.1) in the network settings in the system preferences panel. All I had to do to get Classic mode's ability to connect to the internet was remove my computer as a DNS and relaunch Classic. If anyone knows how I can keep my computer as a DNS in OS X and keep Classic happy, please let me know.......
 

macxtra

Registered
beef
Dinner
I experience the same dalay in starting up "beef" talked about ->


> everything's working... but when I reboot, it kinda gets stuck at "Starting
> Directory Services". The same thing happened when I was attempting to
> make changes to netinfo database before, so I guess it's for the same
> reason...

> Anyway... it got stuck there long enough for me to do the dishes...

> I don't need to reboot all that much, so it's not a big problem, but it's rather > annoying when you have to go back to 9 then back to X, etc...

> Am I the only one experiencing this? If anyone knows the solution, let me
> know.

any explains ....
 
Top