Where has all my ram gone?

kilowatt

kilowatt

mach-o mach-o man
On my G4 733 with 640 megs of ram, I have just run it down to the last 33.8megs. Which means that while it sits at the login window (because I ssh'd in from a linux box), the system needs OVER 600 MEGS to just freakin sit there and do nothing. And thats not including paged ram...

I could be wrong about this, I thought it was low because all my apps started running much slower than I thought they should be. Here's my output from top:
(hope this isn't formatted funny..)


Processes: 40 total, 2 running, 38 sleeping... 95 threads 19:11:40
Load Avg: 0.29, 0.05, 0.01 CPU usage: 0.0% user, 100.0% sys, 0.0% idle
SharedLibs: num = 137, resident = 17.6M code, 1.38M data, 4.09M LinkEdit
MemRegions: num = 1342, resident = 9.62M + 1.54M private, 15.3M shared
PhysMem: 57.8M wired, 46.7M active, 502M inactive, 606M used, 33.8M free
VM: 828M + 56.5M 34887(34887) pageins, 33838(33838) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
2681 top 0.0% 0:00.12 1 15 14 172K 276K 408K 1.37M
2676 tcsh 0.0% 0:00.11 1 17 16 476K 604K 944K 5.74M
2675 sshd 0.0% 0:00.16 1 9 13 168K 472K 428K 1.50M
2674 top 0.0% 0:23.09 1 15 14 216K 276K 452K 1.37M
2670 loginwindo 0.0% 0:00.81 1 57 80 5.55M 12.6M 15.4M 76.6M
2669 Window Man 0.0% 0:00.18 3 66 42 332K 8.66M 9.20M 43.6M
2664 tcsh 0.0% 0:00.10 1 20 16 476K 604K 940K 5.74M
2655 tcsh 0.0% 0:00.11 1 17 16 476K 604K 924K 5.74M
2654 sshd 0.0% 0:00.30 1 9 13 168K 472K 424K 1.50M
854 CCacheServ 0.0% 45:25.51 1 21 20 4K 1.21M 56K 5.36M
279 cron 0.0% 0:00.79 1 9 15 24K 252K 92K 1.52M
275 sshd 0.0% 0:00.75 1 9 14 56K 472K 164K 1.50M
269 SecuritySe 0.0% 0:02.14 2 54 23 380K 464K 612K 2.94M
264 httpd 0.0% 0:00.00 1 9 169 4K 252K 60K 6.57M
260 httpd 0.0% 0:16.01 1 9 169 16K 252K 84K 6.57M
259 slpd 0.0% 0:47.07 8 29 28 88K 352K 260K 4.99M
252 automount 0.0% 0:00.01 2 11 18 48K 320K 132K 2.14M
249 nfsiod 0.0% 0:00.01 1 9 12 4K 232K 36K 1.25M
247 nfsiod 0.0% 0:00.00 1 9 12 4K 232K 36K 1.25M
246 nfsiod 0.0% 0:00.00 1 9 12 4K 232K 36K 1.25M
245 nfsiod 0.0% 0:00.00 1 9 12 4K 232K 36K 1.25M
232 DirectoryS 0.0% 0:00.14 3 36 41 4K 232K 44K 3.62M
231 named 0.0% 0:02.16 1 9 21 396K 520K 672K 2.73M
223 inetd 0.0% 0:00.00 1 9 13 4K 232K 36K 1.26M
216 coreservic 0.0% 0:04.41 3 61 30 416K 288K 496K 2.74M
212 AppleFileS 0.0% 0:34.00 12 89 50 76K 340K 248K 8.75M
203 ntpd 0.0% 0:25.82 1 9 15 60K 308K 172K 1.49M
193 lookupd 0.0% 0:12.81 2 37 20 176K 380K 392K 2.39M
186 netinfod 0.0% 0:03.68 1 8 17 200K 296K 308K 1.54M
164 CrashRepor 0.0% 0:00.01 1 25 16 4K 232K 52K 1.59M
143 syslogd 0.0% 0:01.61 1 8 13 48K 244K 104K 1.26M
115 configd 0.0% 0:03.78 4 102 111 404K 372K 572K 3.72M
92 autodiskmo 0.0% 0:01.72 2 40 19 88K 300K 204K 2.34M
69 dynamic_pa 0.0% 0:00.86 1 22 15 24K 240K 68K 1.27M
66 update 0.0% 1:05.14 1 8 13 12K 236K 60K 1.25M
62 ATSServer 0.0% 0:33.64 1 31 69 600K 2.18M 1.59M 50.9M
41 kextd 0.0% 0:00.02 2 16 126 4K 232K 36K 3.69M
2 mach_init 0.0% 0:14.83 1 267 13 52K 252K 120K 1.27M
1 init 0.0% 0:00.08 1 20 12 52K 252K 108K 1.26M
0 kernel_tas 0.0% 28:52.54 24 0 - - - 37.5M 555M
0 idle_threa 0.0% 80:06:06



So wtf is going on? I tried kill -HUP on the window server to see if maybe it was just stuck on something and needed a little kick in the ass. It didn't help. My performa from back in the day came with 8 megs of ram. And I could surf the net, use aim, and play games... (now it has 40megs, and nobody uses it...).

I'm going to ruin my uptime now, and reboot to see if this changes my ram useage.

somebody please shed some light on this... I hope I'm reading top wrong
 
ok, now what? I reboot, and I've got over 500megs avaliable... something is fishy. Do you think my +4 day uptime has revieled a memory leak or something?

I've had osx up longer than that before (something like 24 days), and it wasn't running out of ram then (but I was moving out of my dorm room so I had to, sniff sniff, pull the plug for a bit).

Here's my top output now:

Processes: 35 total, 2 running, 33 sleeping... 95 threads 19:24:19
Load Avg: 0.13, 0.32, 0.25 CPU usage: 0.0% user, 100.0% sys, 0.0% idle
SharedLibs: num = 67, resident = 14.0M code, 1.14M data, 3.79M LinkEdit
MemRegions: num = 1244, resident = 12.5M + 2.47M private, 16.6M shared
PhysMem: 42.8M wired, 30.8M active, 37.2M inactive, 111M used, 529M free
VM: 765M + 32.8M 2091(2091) pageins, 0(0) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
299 top 0.0% 0:00.13 1 15 14 172K 292K 408K 1.37M
293 tcsh 0.0% 0:00.14 1 17 16 468K 620K 944K 5.74M
292 sshd 0.0% 0:00.18 1 9 13 168K 512K 420K 1.50M
291 loginwindo 0.0% 0:00.92 1 57 65 5.94M 9.76M 15.4M 76.5M
283 cron 0.0% 0:00.00 1 9 15 72K 284K 148K 1.52M
279 sshd 0.0% 0:00.50 1 9 14 56K 512K 188K 1.50M
273 SecuritySe 0.0% 0:00.02 1 15 17 124K 532K 260K 2.02M
266 httpd 0.0% 0:00.00 1 9 169 56K 2.19M 276K 6.57M
264 httpd 0.0% 0:00.05 1 9 169 44K 2.19M 1.56M 6.57M
259 slpd 0.0% 0:00.01 8 29 28 172K 440K 520K 4.99M
256 automount 0.0% 0:00.01 2 11 18 184K 356K 320K 2.14M
253 nfsiod 0.0% 0:00.00 1 9 12 12K 284K 88K 1.25M
251 nfsiod 0.0% 0:00.00 1 9 12 12K 284K 88K 1.25M
250 nfsiod 0.0% 0:00.00 1 9 12 12K 284K 88K 1.25M
249 nfsiod 0.0% 0:00.00 1 9 12 12K 284K 88K 1.25M
236 named 0.0% 0:00.01 1 9 18 688K 804K 608K 2.48M
235 DirectoryS 0.0% 0:00.14 9 43 52 568K 844K 1.12M 6.62M
227 inetd 0.0% 0:00.01 1 9 13 68K 272K 128K 1.26M
220 coreservic 0.0% 0:00.06 3 38 30 476K 324K 740K 2.56M
216 AppleFileS 0.0% 0:07.04 12 86 49 636K 852K 1.42M 8.75M
207 ntpd 0.0% 0:00.05 1 9 14 88K 440K 288K 1.49M
197 lookupd 0.0% 0:00.15 2 11 19 232K 512K 464K 2.39M
190 netinfod 0.0% 0:00.14 1 8 17 204K 368K 360K 1.54M
168 CrashRepor 0.0% 0:00.01 1 16 16 160K 280K 120K 1.57M
147 syslogd 0.0% 0:00.05 1 8 13 80K 272K 152K 1.26M
119 configd 0.0% 0:00.56 4 70 107 476K 820K 1.36M 3.47M
96 autodiskmo 0.0% 0:00.10 2 25 18 188K 380K 340K 2.28M
73 dynamic_pa 0.0% 0:00.00 1 9 15 72K 264K 108K 1.27M
70 update 0.0% 0:00.01 1 8 13 56K 260K 96K 1.25M
68 Window Man 0.0% 0:01.99 3 56 43 332K 8.68M 9.07M 43.6M
66 ATSServer 0.0% 0:00.09 1 27 32 644K 852K 900K 14.5M
41 kextd 0.0% 0:00.23 2 16 165 2.54M 284K 1.82M 4.07M
2 mach_init 0.0% 0:00.55 1 34 12 44K 284K 128K 1.27M
1 init 0.0% 0:00.02 1 20 12 48K 272K 256K 1.26M
0 kernel_tas 0.0% 0:01.40 24 0 - - - 26.0M 547M
0 idle_threa 0.0% 1:27.31
 
The move to a modern vm system is a huge conceptual leap for people used to the old macos model, so this is going to continue to be an issue for many people for years.

Which is not helped by the fact that it takes a long time to explain. So please forgive me if I ramble on:


The first issue you're seeing here is that os10 (like essentially all unices) uses any ram which is not otherwise occupied as a filesystem cache. When the machine has recently been booted, you haven't done enough filesystem access to have much in there. Over a short amount of time, normal disk access will populate the cache. If processes want to actually use that memory, the system will repurpose the least-recently-used pages of the cache to active duty.

This is a great model. Remember the old macos disk cache setting, which had to be manually fixed per boot? Which was pretty much always either too big or too small? This does the same thing, except that all "free" ram automatically gets shuffled in and out of this duty.

In the general case, you actually want to have extremely little ram "free". "free" means it's just sitting around uselessly, when it could be providing you some service.

Two big semi-related issues:

"I have plenty of ram. Why am I using swap?"

There are some objects which have been or theoretically could need to be accessed in memory, but which basically never are, or never are after a single event. For example, many things used for the boot process are never touched again.

Your system does a pretty good job of identifying such things, and swapping them out, even if it's not in dire need of more ram. This tends to not be a problem.

The above two together mean that you'll frequently have some parts of the filesystem which have been cached in ram, and some parts of memory which have been shipped out to the filesystem. This seems odd, but it works out well.


"Hey, if you add up the memory each process claims to be using, it comes to like ten times as much ram as I have!"

Yet another wonderful effect of the "virtual" in "virtual memory". The memory each process is using does not always have a fixed, one-to-one relationship with actual bits of real ram. The most common instance of this is multiple programs which have all loaded the same library: only one copy of the library will reside in real ram; each process will count it as being part of its memory footprint, and they have no knowledge of what if anything else is using it.

(As an aside, no, this doesn't mean they'll step on one another if they try making changes to that object after they've loaded it. If any of them writes to its "copy", the system will automatically copy the changed pages to new real ram. So they don't hurt one another, and it copies only the bits that're changed, not even the whole library. Again, very clever.)

It's actually not possible for a process to know how much memory it could have available, or whether its memory is currently swapped out to disk, or whether its memory is shared at all with any other processes. Those all happen at a much lower layer, and the process essentially doesn't need to know.


So what's the upshot of all this? Throw away all your memory-measuring utilities. Don't try and micromanage how your system uses memory; it makes complex decisions which are almost guaranteed to be better than you would do.

The only thing about which you'll ever care is if your system is actively swapping in and out a lot.
And if that starts happening, don't worry; you'll know.
 
Thanks for the info, however, it appears to be more than micromanagement I'm up to.

For example, I downloaded the BSD 4.4/intel binary iso file, which is arround 650megs, and omniweb begins to take on HUGE ram useage. I mean, omniweb shot up to 779 megs or ram durring the download (and it increased porportionally to the download, too). Omniweb was almost un-useable durring this. And, for some reason, aim shot up to 91 megs, and iTunes to 90. I've been using linux for a few years, and I'm used to having the system maxamize its resources - an un-used processor is a wasted processor, right? Well, here's part of top in linux. Netscape, gaim, and xmms are using LESS than 40 megs of ram. The entire system only has 128 physical and 128 swap.

2:22am up 27 min, 3 users, load average: 0.20, 0.15, 0.09
75 processes: 74 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 4.3% user, 1.6% system, 0.0% nice, 93.9% idle
Mem: 126720K av, 124032K used, 2688K free, 0K shrd, 5284K buff
Swap: 265032K av, 96K used, 264936K free 56512K cached

I really don't think its normall for omni web to allocate over SEVEN HUNDREAD MEGS for its self. I mean, that is just crazy.

Not to mention, the finder was using over 100. On a normall day, I've got between 500 and 300 megs of ram avaliable, which is fine with me. But when top uses 2.5megs, bash uses 5megs, and aim (AI FREAKIN' M) uses 90megs, things get harry.


My system got to the point where I couldn't bare to use it any more. I just rebooted, and it seams to be ok again...

Oh, and while omni web was using +700megs of ram, I launched a few other programs. They each grabbed +100 megs each, and omni's share didn't change at all. So I don't think the system was dynamically sharing the ram... seams like it flipped out and pagged everything.
 
PhysMem: 57.8M wired, 46.7M active, 502M inactive, 606M used, 33.8M free

Wired memory is non-pageable kernel memory; it consists of the kernel text and, most probably, network buffers and VM page tables.

Active memory is the mapped pageable memory the contents of which is in use by running processes; this memory may need/will be paged out. IIRC, active memory includes both clean and dirty pages.

Inactive memory is mapped pageable memory which has not been changed from the time it was mapped; this memory includes filesystem cache.

Free memory is pageable memory without a valid mapping; it is possible to reuse it immediately.

It is advantageous to keep the mapping to a (filesystem) object alive even after demise of the process which required it, since another process may need access to the same object, which can be satisfied by assigning a mapping to inactive memory.

Since inactive memory is not assigned to any running process, in case of free memory shortage, inactive memory can be freed on a moment's notice (using some kind of weighted LRU algorithm)
 
Well, the linux behaviour is actually consistent with this:

Mem: 126720K av, 124032K used, 2688K free, 0K shrd, 5284K buff

It's still basically using all but the last couple megs of ram, that just happens to be a much lower total amount.

And omniweb's choice to map the whole file it's transferring into ram is sadly unsurprising. I love omniweb, I'm typing into it right now, but its usage of memory tends to be... rather poorly thought out. I usually prefer using something like wget for grabbing huge files.

As for why all other processes started using drastically more ram when omniweb was sucking.... well, I'm afraid I have no idea. I've never knowingly run into that situation, nor can I think of any good reason it'd be so.

Sorry to not have a better answer on the last point.
 
Ok, I'm having the same exact problem. I have 1.2GB of ram and after a few days my system starts swapping like a mofo and slows way the hell down. If I quit every single application that I have running, I still can have 900+ MB of "active" memory.

My question is, why is it when an application is shut down that the memory it was using doesn't become "inactive"? Right now I'm running IE [just launched] terminal and the finder and here is what top says:

PhysMem: 113M wired, 721M active, 405M inactive, 1.21G used, 41.7M free

this is after a reboot yesterday. i'm sure that after a few days i'll have to reboot again so that I can use Maya reliably [maya tends to crash when active memory gets up near 1gb and the system has to swap like crazy].

isnt there some way to manually free up all of the "active memory" that really isn't active any more? or am I not understanding something...
 
You must log in or register to reply here.
Back
Top