It all started on Friday night when I was doing a secure erase of the Trash. My first mistake was in not realizing one of the folders I had trashed must have had a LOT more stuff in it than I realized. The secure erase was obviously going to take a while. OOOPS.
However, for some inane reason I decided to let it run, rather than clicking the stop button and trying to sort through the Trash and only secure-erase the few files I really wanted to wipe. That was mistake number two.
At some point during the wipe, Time Machine started up. I had previously noticed that my Time Machine drive was almost full, and, having just moved around a lot of stuff, I also knew that what little space was left was not going to be enough for the next backup. Time Machine was going to need to do a pre-backup “thinning” procedure in order to make room for all the stuff I had moved.
So, Time Machine was cranking away with its thinning process, while at the same time, the Finder was trying to do a secure delete on a whole mess of stuff. Theoretically, it’s supposed to be possible to do these two things at the same time. In practice, something went haywire. After Time Machine had been running for a ridiculously long time, I decided to stop it, turn it off, and allow the wipe operation to complete. I would then turn Time Machine back on so it could do its backup in peace. I suppose this was another mistake, although in retrospect I don’t know if stopping the wipe operation would have prevented the subsequent problems, given that it was already obvious that something was wrong.
My Time Machine drive is an external 1 terabyte Firewire 800 drive, which, as I said, was almost full. After the wipe was complete, I turned Time Machine on again and initiated a backup. It did the backup, and went into the thinning process again. It thinned and thinned and thinned, until it had deleted over 600 gigabytes of files.
I could see no legitimate reason for why it would delete that many files. I also figured that, with 2/3 of my backups deleted, I was basically fucked and the only thing to do was reinitialize the Time Machine drive and start over. However, it was already very late that night, so I put that off until Saturday.
Saturday evening, I reinitialized the Time Machine volume (not the whole drive, just the volume, which is a subtle but, as it turns out, crucial distinction–i.e., mistake number four). I allowed Time Machine to start up again, telling it this time to ignore all but my internal drive and one small external with about 80 gigs of material for backing up. It did the external drive first, and everything worked just fine, apparently. Then it started copying files from the internal drive, and something seriously wrong started to happen, again. Progress slowed to a glacial pace. I would estimate it took two or three minutes to back up 100 megabytes of data, and since I had another 250 to 300 gigabytes to go, this was not acceptable. My only possible saving grace at that point might have been if the slowness was a result of it getting bogged down in the thousands of teensy little files in the depths of the System folder. I decided to let it run overnight.
On Sunday, however, after a lengthy night’s sleep (truthfully, I did not want to get out of bed and deal with this shit), only 40 additional gigabytes had been copied. The backup had been running for over 12 hours and wasn’t even half done. I soon decided that more waiting was pointless. I did a bit of Googling and found some tips that looked like they might help.
Here’s what worked: I stopped the backup again, and turned Time Machine off. I went into my Spotlight preferences and discovered that Spotlight had somehow not bothered to exclude the Time Machine drive from its indexing process the way it did the previous times I had set it up. That was undoubtedly a factor. However, it didn’t prove to be the primary factor. I also reinitialized the drive, and this time I told Disk Utility to redo the entire partition map, not just the partition itself. More importantly, I had it format the partition using a “GUID” partition map, which is the default for Leopard, and which Time Machine supposedly prefers. Previously, it had been formatted with an Apple Partition map, presumably left over from my previous Tiger system. I then sacrificed a chicken, prayed to all the gods in Valhalla (Loki in particular), and told Time Machine to make another go at it.
Well, the GUID partition map really seems to have been the magic bullet. Not only did the initial backup work perfectly, the remaining drives backed up without a hitch as well, at the point when I re-included them. What’s even more amazing to me, though, is that Time Machine is now performing at about six times the level of efficiency it was before all this started. It’s substantially faster, and it uses only about 1/6 the RAM it did before. Before this, backups were an irritating drag on the system, so annoying that I would often turn them off to alleviate the frustration. They would also hog close to 350 megabytes of RAM, meaning that every hour some idle application got shunted off into virtual memory. This was a severe annoyance with programs that utilize a lot of RAM. Now it uses a mere 55 megs, it’s backing up the same amount of data, and doing the whole shebang in about 30 seconds, unless there’s a bigger file that needs to be backed up. Note that it’s backing up four drives, totaling about 750 gigabytes of data, with what must be half a million files at least. In less than a minute. That, in my opinion, is how things ought to work. ;)
So, even though this whole experience was rather nightmarish, especially when I began to wonder if my internal hard drive was on the verge of failure, I ended up learning a thing or two about Time Machine optimization. If you’re having Time Machine problems, seriously check the format of that partition map and make sure it’s GUID. If it’s not, and if you find Time Machine backups to be an incessant bother, it will probably be worth your while to nuke that partition and replace it with a GUID partition. Also, double check to make sure Spotlight isn’t indexing your Time Machine drive. Normally, it will not index that drive, however if something gets messed up, it’s possible Spotlight will not “realize” it’s attempting to index a Time Machine drive at the same time that Time Machine itself is trying to do its initial backup.