[ Previous Tip ] [ Mac Tips ] [ Next Tip ]

• Introduction
• Backup Methods and Media
• Software
• Backups Gone Overboard
• Time Machine
• Local Backups7 Feb 12
• Time Machine Issues
• Time Machine Disk Sizing 26 May 12
• Time Machine Performance
• AirDisk Issues 26 May 12

The subject of this page is backups, why they should be done, how they should be done, and Time Machine, a new feature introduced in Mac OS 10.5.

If there is any data at all on your computer that you care about keeping, then you need to implement a backup strategy. There are lots of ways to do this, some easier than others. Each way has its own advantages and disadvantage. The message I hope to deliver on this page is that NO ONE METHOD does it all. You need to implement at least an archival backup and a "real time" backup.

Computers are electro-mechanical hardware, and as such, have it within their birthright to crap out at any time and for any of thousands of reasons. They can also be lost, destroyed or stolen if they don't actually break on their own. Oftentimes, the result of one of these bad events is that the data stored on that computer is lost. If that data is not kept in another place, then the data can be lost forever.

This is bad.

Those of you that have experienced loss of some irreplaceable data, such as family photos, may have found salvation via a near religious conversion to a sect called BABU, or Born Again Back Up. Those that haven't been converted yet will be sometime in the future.

There are many methods that people use to back up their data. These include, but are not limited to:

• Manual file copies in the file system
• Use of commercial, shareware or freeware software designed to do backups
• Time Machine (on the Macintosh anyway)

Whatever method works for you, you must at least use it and use it often. You actually should use at least TWO methods. I use all three.

Just copying a file into another place on the same computer or disk does NOT help much at all. Your data should be copied, by some method, onto physically separate media. Further, that media should not be stored in the same location as the computer. A fire, for example, could destroy both the computer and the backup media at the same time. Backups should be implemented such that TWO copies are made, one to store in convenient location, and one to store at a remote location such that one disaster won't take out both backups.

The media that you select should have a very long storage life, at least as long as your life. I back up to CD and DVD and store them in binders, but piling them back on a CD spindle is a good storage method too. Optical media will not last forever, but stored properly, it'll last longer than I do. Tape probably has the best track record for longevity, simply because tape has been around longer than the rest and, bit rot^* aside, tape as withstood some tests of time.

There are two general types of backups, archival and real time.

An archival backup usually records the backup data on write-once media such as DVD-R, but it could also be tape or even other hard disks. However hard disks are not well suited to archival backup as their electronics are more prone to fail than a write-once media. Even write-once media has limitations. The media itself may fail due to external environmental problems or by degradation of the media itself over time. Also, the hardware needed to read an old backup may not exist when the backup is to be read. There are many examples in recent history of backup tapes an disks that were in otherwise good condition that could not be read because there was no working hardware left that was capable of reading that media. Hopefully CD and DVD hardware will be around long enough to read my media when it is needed.

Archival backups are generally done in a batch form when enough data has been collected to make up a block of data that efficiently fits on the archival media. In this regard, smaller media, such as CD, is actually better because they hold less data and therefore have to be written more often.

Real time backups are necessary to fill in the time gaps between archival backups. If you only do an archival backup every month, then you have an average of two weeks of exposure to loss of data that hasn't made it onto the archival media yet.

Real time backups are usually implemented with special hardware, such as a RAID disk array, or special software, such as Time Machine, that creates a backup in near real time.

A RAID disk array can be arranged with TWO or more disks where the data is written to BOTH of them at the same time. These are often used in server farms were total redundancy is required. If one of the disks fails, the system can continue to run with the remaining disk until the failed one is replaced. This is a pretty expensive solution that is overkill for most home uses although it is very effective.

Software based semi real time backup can be implemented with the inclusion of a cheap external USB or FireWire hard disk and the Time Machine feature of Mac OS X. Time Machine is a good tool for keeping the first level backup, but it CANNOT replace an archival method. The Time Machine disk is usually collocated with the computer AND it will have limited capacity. Eventually, it will fill up. To keep running with current backups, Time Machine begins to "thin" or delete the oldest files that are not currently on the computer to make room for newer files. Therefore Time Machine CANNOT BE USED for archival storage. Time Machine backups also tend to be local so that a local disaster can take out the Time Machine backup as well. Time Machine using a local disk, however, is effective against hardware failures or user errors. If the Time Machine disk itself is the item that fails, a new disk can be substituted for it without loss of any current data. What is lost is some history of older data.

On-line backups to some remote server can be done in real time AND are remote by their nature. They are also expensive. However, when outfits like Google offer terabytes of storage cheap, or even free with advertising attached somehow, then on-line backups can take center stage. However, when you do on-line backups, you don't control how and where your data is stored. You don't even control the data, it can be scanned and searched, maybe to help focus those targeted ads that pay for the space.

Semi-real time backups can also be done with a variety software that stores a current image of your disk. These work too, but do not provide any history. If you make a mistake and delete some important data, and then do a backup, that data on the backup device may be erased in the process.

There is lots of software out there at reasonable cost to implement a solid archival backup strategy. Some of it is also good for real time backups. I have used only a few types, but all of them have worked.

CarbonCopyCloner and SuperDuper! are both excellent shareware programs that will clone a whole disk so that the clone is actually bootable. They will also do incremental backups, saving only those files that have changed since the last backup in order to save time. Some of these methods will also save older versions of backed up data, however none are as easy to use as Time Machine. I use CCC to make bootable clones.

Synchronize!Plus and Synchronize!Pro are intended to do incremental backups, the Pro version will also make a bootable clone. I use Synchronize!Plus for my routine incremental backups primarily because it shows me what it is going to do before it does it. That way, if I don't like what I see, I can change it before it messes with my backup disk.

Finder. All of those programs generally replace older versions of a file with newer versions, they often do not keep a version history. None are particularly suitable for archival backups. For that, I use the Finder. I collect the data that I am going to want to archive into a folder (700 Mbyte for a CD or 4.28 Gbyte for a DVD, these numbers are about 7% larger in Lion due to the way a GB is defined) and when the folder grows to the appropriate size, I burn two copies of the folder, one to keep at home and the second to keep at a remote location. The folder is then either fully or partially cleaned out to make room for new stuff.

Once the data is recorded onto an archival backup media, it does little good if you cannot find the data again. There is where cataloging tools come in handy. I use simple brute force tools called AutoCat or Alias Disk to create a directory structure identical to the backup media, but built only from aliases. I keep these aliases on my hard disk so that Spotlight can index at least the file names. Each disk is numbered when burned so that when I do a Spotlight search and find something, Spotlight shows me the path which also contains the number of the disk that the data was burned on. A quick flip through my archive will find that disk number and the file is ready as soon as the disk is inserted. There are many other programs that do cataloging in a more graceful fashion, but I find that the simple method works quite well.

I may tend to go a little over the top in doing backups, but this is what I have done on a recent Antarctic cruise. I had my PowerBook with me, it has an 80 G internal disk. For backing up, I have:

On a saner note, this is my backup method for my iMac which has a 250 GB internal disk.

All this discussion leads to Time Machine, a feature of OS X. Time Machine is an incremental backup tool that also keeps a history of the complete state of the disk at discrete times in the past. If a software install goes awry, Time Machine can be used to restore the disk state as it existed BEFORE the problem occurred. Individual files, emails, address book entries and other pieces of data can be plucked from the past as well.

I am not going to try to write a description of Time Machine, I intend to write mostly about what hasn't been written already. If you haven't done so already, please read Apple's product description of Time Machine. For those who want more details, read Ars Technia's review of Time Machine. For those that want to know more about the underlying process that Time Machine uses, read Ars Technia's description of FSEvents.

Time Machine makes heavy use of hard links on the backup disk. Hard links reside in the disk directory and are pointers to the actual file. One or more hard links can point to the same file. EVERY directory entry that you see on any disk is a hard link, but usually there is only one per file. When the last hard link to a file is removed from the directory, the file itself, in effect, no longer exists. Time Machine uses many hard links to each file. Every time stamped folder in the Time Machine backup set consists of a complete set of hard links to every file that existed when that backup set was written. If the file was new, then there is just the one hard link in the directory and the file exists somewhere out there on the disk. If the file was pre-existing then Time Machine writes only the hard link pointing to an older instance of the file. However, if you look at the directory of a given backup, it still LOOKS like every file that existed at the time of the backup is there in each backup folder.

Time Machine uses an external disk to keep it's backups. This disk can be connected directly to the computer being backed up, or it can be connected to a computer acting as a server or it can be a network attached storage device. Time Machine keeps the remote backups in such as fashion so that several computers can be backed up on the same disk without interference, WITHIN DISK CAPACITY LIMITS. When Time Machine uses a network connected disk, it keeps it's backup in a "sparsebundle" file which is scrambled so that it is not readable except from the computer that wrote it.

My experience has shown me that the backup disk should be AT LEAST twice as large as the internal disk to be backed up, and preferably be larger than that. The bigger the disk is, the longer Time Machine can go on keeping a record of every file that has ever existed. My backup disk is 3 times as large as the internal hard disk of my iMac. I whacked on it pretty hard over the period of just two weeks but I actually filled it up. Not everybody is going to move 400 GB of data around in addition to having the internal disk nearly full most of the time in just two weeks, but I did.

Message: BIGGER IS BETTER.

Very large external FireWire and USB disks are available at relatively low cost. As of early 2008, 750 G disks seem to be a commodity, running less than $200 with multiple interfaces. 1 TB disks are also coming available at reasonable prices. USB only disks are even cheaper. Spend the money and get a big disk.

If several computers are backed up on the same backup disk, then the backup should be 3x the size of all of the computer's disks put together.

Before March 19, 2008, Time Machine could not use an AirDisk (because the AirDisk function of the Airport simply didn't work well enough) or iPods (which are not designed to run continuously). The Time Capsule (Airport base station with a built in disk) is just becoming available and it does work. On March 19, Apple released a Time Machine and Airport update (including Airport firmware 7.3.1) that upgrades the older 802.11n Airport base stations so that they will work with Time Machine. Note that the capability to use an AirDisk for a Time Machine backup is currently unsupported although it does appear to work.

Time Machine will not run automatically on a laptop running from battery power, it will simply put itself to sleep until the laptop is connected to it's battery charger. This is probably a good thing.

Time Machine is also pretty graceful when it loses connection to it's backup disk. If a directly connected disk is ejected, TM will wait patiently until the disk is reconnected. For a connection via a server, TM will find the server and mount the disk when it needs it. When it is done, it will dismount the disk and disconnect from the server. If Time Machine cannot find the server, it will declare an error and then wait until it can find the server again.

If Time Machine is stopped, or if the computer is shut down, or if the backup disk becomes unavailable DURING a backup, Time Machine will declare and error and pick up where is left off when it can find the backup disk again.

Time Machine does backups by file. If a single bit in a large file is changed, the WHOLE file is backed up again. This is a problem for virtual disk files used with Parallels or the data base that Entourage uses. These can be excluded but the better solution is to get a bigger backup disk that can handle the abuse of very large files changing fairly often.

Files or folders that are simply moved or renamed are counted as NEW files or folders. If you rename any file or folder, Time Machine will back up the ENTIRE file or folder again no matter how big or small it is.

Time Machine allows exclusions. If a file is causing problems, then it, or any folder that contains it, can be excluded from the backup set and Time Machine will simply ignore it. There are also built in exclusions. Time Machine does not back up the Trash, caches or other transitory data that isn't needed to restore the state of a disk. These built in exclusions cannot be easily changed.

Time Machine is primarily a scheduled process. Nominally, every hour, Time Machine starts itself up, backs up the disk and shuts itself down. However, there are many factors that can impact the timing.

• After computer wakes from sleep, Time Machine will often schedule a backup at the next appointed time, but often when this time comes, Time Machine will simply reschedule for the NEXT hour. I think that this is prevent TM from starting a backup immediately to avoid possible conflicts with Spotlight and foreground process and because there might not be enough new stuff to make a backup worthwhile.
• When ANY disk is attached Time Machine will start up immediately. This includes memory sticks, iPods set for disk use, some cameras, and servers.
• When a user requests it

The exact timing of the backups depends on several factors. Time Machine will typically schedule the backups starting at an hour since the last boot. It will then backup every hour past that. However, if the computer sleeps, this can change.

When Time Machine does start up, it goes through several steps. First it does a preparation step, then it determines if enough space is available, then it copies changed files, then it deletes expired backups.

During the preparation step it checks the FSEvents log for consistency. If it determines that something isn't quite right it has to rescan the whole disk. This can take quite awhile. A full rescan is always triggered by a crash, an unplanned shut down event or by booting from some other bootable disk between backups.

If Time Machine determines that enough space is available on the backup disk for the data to be copied plus some padding, it proceeds to backup the necessary data. If there is NOT enough room, Time Machine has to make some room by deleting older backups. This process is called pre-thinning and is described in some more detail below.

The copy process is pretty straight forward and can happen in two passes if changes happen DURING a backup. This part can go quickly or slowly, sometimes for no obvious reason. It can also rip through lots of data, then slow to a crawl for awhile and then start to rip along again. If a backup is interrupted for any reason, the next backup will usually go very slowly but it will eventually finish.

The last part of the process is post thinning where backups that have expired are deleted. A description of what causes a backup to expire is found below.

Time Machine keeps snapshots of the disk at particular times, each snapshot has different lifetime in the backup set. Time Machine figures that files that exist for only a short time are probably not too important, it'll remember them for less time. Files that hang around for awhile are remembered longer.

These are the rules for how long a file is remembered in the backup set.

• If a file is created and deleted such that it does not exist during an hourly backup, the file will not be remembered by Time Machine at all.

• If a file is created and deleted such that it does exist in an hourly backup but does not exist at the time of the first backup after midnight, it will be remembered for only 24 hours.

• If a file DOES exist at the time of the first backup made after midnight, it will be remembered for at least 30 days.

• If a file DOES exist during the first backup after midnight of each day of the week since the backup set started, it will be remembered at least until the Time Machine disk fills up. Older files are forgotten first.

• If a file CURRENTLY exists on the computer's disk, it will be represented in the "Latest" backup set and it will not be thinned.

The backup designated as the weekly backup can change. Normally, it is the first backup after midnight every week on the same day of the week as the first backup. However, if there are NO backups on that day, Time Machine will use the next one after midnight. It will then use THAT day of the week instead for subsequent weekly backups.

When the Time Machine disk does fill up, Time Machine must do something to keep working. The first time Time Machine is run, it copies the WHOLE disk less exclusions. After that, it copies only new or changed files. When Time Machine is faced with a full backup disk, it figures that the oldest stuff is not important enough to keep and it DELETES the original backup set. Files that are in that backup that aren't represented by hard links in a more recent backup are deleted. This means that history of user deleted files will start to GO AWAY. If an archival system is not implemented in addition to Time Machine, old but possibly valuable files will be lost. Time Machine was never intended to do archival backups, it keeps the recent changes that may not be covered in a proper archival backup.

During each subsequent backup where adequate space cannot be allocated, Time Machine deletes more stuff starting with the oldest backup in the backup set.

Time Machine will operate as long as the computer is on and not sleeping, or in the case of a laptop, only when plugged into AC power. Unless told otherwise, Time Machine backs up the whole disk, system, applications and all users. A user login is not required, Time Machine will operate behind the login screen. Unless excluded, all users are backed up, even ones that are not currently logged in.

The Time Machine process is triggered by a cron event every hour. At that time Time Machine looks at the FSEvents log to determine what parts of the file system have changed. It then scans all the changed parts to get the details of the changes and figures out what it is going to do. This all happens during the "Preparing" phase. This can take awhile. In the case where an unplanned shutdown has occurred since the last backup, the preparing phase can take a long time.

The Time Machine Preference Pane provides very little detail about what is going on while Time Machine is running. If you want to see more detail, or need to see any error messages generated, then you'll need to use the Console, see some more details below.

If the computer is off or sleeping, Time Machine will not run. This is not a problem as files are not being changed during these times. If the Time Machine process is interrupted for any reason, Time Machine will figure this out the next time and pick up where it left off, maybe slowly, but it will do it.

When Time Machine does it's "post-thinning" operation, it may find that more than one backup has expired. It will delete up to five at a time until it has caught up deleting expired backups.

If you should want to do some massive rearrangement of your disk, Time Machine will interpret the rearranged files as new files and back them up again in their new locations. Just renaming a folder will cause this to happen. This is ok if you've got lots of room on your backup disk. Eventually, Time Machine will thin those backups and the space consumed will be recovered. However, if you really want recover the space in the backup volume immediately, you can. To do this, bring a Finder window to the front and then click the Time Machine icon on the dock. This will activate the Time Machine user interface. Navigate back in time to where the old stuff exists and select it. Then pull down the "action" menu (the gear thing) and select "delete all backups" and the older stuff vanishes.

If your backup volume is a locally connected disk, the space will be recovered immediately and this will show up in the Finder's reports of the amount of data on the disk. If the backup is to a sparsebundle on a remote disk, the bundle won't appear to get any smaller. Once a bundle grows to an given size, it never shrinks. What you don't see is that "white space" is created inside the bundle which Time Machine can reuse so that the bundle won't grow for awhile as more stuff is added. When all the white space is finally consumed, the bundle will start to grow again.

If a Time Machine backup fails, Time Machine will abandon the backup, it will not show up as a backup folder but the portion of the failed backup that did get copied will cause the sparsebundle to grow leaving the room it consumed before abandonment as white space. This can be an issue if the failed backup was the initial one and Time Machine got pretty far before it barfed. At this point, the sparcebundle will have a significant size even when it appears to be empty. Time Machine should be able to reclaim this white space later before it starts growing the sparsebundle again.

This characteristic of a sparsebundle will have some interesting impacts on a shared disk, such as a Time Capsule or an AirDisk. Most desktop machines should probably have a dedicated, directly connected Time Machine disk as this is cheap and easy. However, a dedicated disk sort of works against the concept of a wirelessly connected laptop. With several laptops using the same wirelessly connected disk, competition for disk space will occur and the sparsebundle will play into how this works in the endgame. Assuming that there are no other users of the Time Capsule or AirDisk other than several laptops using Time Machine, then each sparsebundle will slowly grow until one of them finally uses the last of the disk. At this point, each sparsebundle is "pinned" in size. They are functionally equivalent to partitions that have naturally grown in proportion to the needs of each user. The heavier users with the bigger disks will tend to have larger sparsebundles. At this point, each instance of Time Machine will need to manage its own space within the fixed constraints of it's own sparsebundle. If one instance of Time Machine needs to pre-thin, it will reclaim some space for itself, but its sparsebundle will not shrink and allow another instance of Time Machine to gobble up that space.

If there are other users with more conventional file structures on the shared disk, then when one of those users frees up some space, the next instance of Time Machine may suck it up, never to be seen by the non-Time Machine users again. If you intend to allow regular file sharing users and remote Time Machine instances to use the same disk, it might be best to pre-partition the disk to separate the types of users or plug in a 2nd disk, one for Time Machine and one for everything else.

In Mac OS X Lion, a new function of Time Machine was added, the Local Backup (or local snapshots). This is a parallel backup that is automatically stored on the users own internal disk. They follow the same rules as regular Time Machine backups but have a shorter lifespan.

Local backups are intended to help mobile users where access to their Time Machine disk may not be available, such as when they are on the road. They provide no protection against hardware failures but do provide some protection against user failures such as by accidentally deleting files.

Local backups DO consume space on the user's disk and will do so as long as there is room. However, when the disk utilization gets to 80%, Time Machine will start prethinning backups IMMEDIATELY to maintain 20% free disk space. Further, the backups are prethinned after just a week so that the local backup date range is short.

There is one thing VERY IMPORTANT to note. If you delete a file and it remains in the Trash, it will still be recoverable via Time Machine. However, if you empty the trash, Time Machine will delete it from the backup set immediately. DO NOT EMPTY THE TRASH WHILE YOU ARE ON THE ROAD.

In the Time Machine browser (the star field thing), local backups appear in white in the timeline at the right side of your screen, regular Time Machine backups appear in pink. While you are on the road, you won't see ANY pink ones.

The version of Time Machine provided with OS 10.5.1 still had some issues, maybe even bugs, that needed to be fixed. As most software goes, those are being sorted out over time. There are still some issues to look out for.

The initial Time Machine backup can go extremely slowly. Wireless connections don't help this so the initial backup can be helped along via an Ethernet connection. Figure on starting it at bedtime and letting it run overnight so you won't fret about it's apparent lack of progress. Subsequent backups often happen very quickly. If the initial backup is interrupted or fails for some reason, then Time Machine will resume where it left off on the next try, BUT, it will do it EXTREMELY slowly as it verifies what it already did. The best data rates to a local FireWire disk that I have seen while Time Machine is running are on the order of 30 MBytes/sec, however, they are often much less and fluctuate quite a lot. Further, Time Machine can appear to completely stop, usually 10 or 15 MB into the backup. This is probably due to a conflict with mdworker. It seems that when TM is just sitting there, mdworker is issuing messages to the Console. When the messages cease for awhile, the TM will pick back up.

Time Machine will also slow down dramatically when it is chewing through a large number of very small files. I have a chunk of about 400,000 aliases in one folder structure. When TM hits this part, it grinds to an apparent halt.

Immediately after a system installation, upgrade, or Archive and Install, Spotlight will often be madly re-indexing the disk. It is best to wait until Spotlight has finished before launching a Time Machine initial backup.

For some reason, earlier versions of Time Machine did not like computer names with anything but alphanumeric characters in them (fixed in 10.5.2). Go to the Sharing Preference Pane and rename the computer using only characters in the range of a-z, A-Z or 0-9. No spaces, punctuation or any other funny characters are allowed.

Time Machine is intolerant of disks partitioned as Master Boot Record (MBR). This applies to almost any generic disk that you can buy because MBR is a Windows/DOS format. Virtually everybody will have to open Disk Utility and repartition the disk as AFP or GUID. It doesn't really matter which one because the Time Machine disk will not be bootable anyway. AFP allows a disk to boot a PowerPC, GUID allows the disk to boot an Intel processor but both are easily digestible by Time Machine on either kind of processor.

There was a pretty obvious bug with certain obscure files, although they were not obscure enough because I had 35 of them. Time Machine would generate and error in the console that looks like this:

PM Error: (-8088) copying /Users/Shared/george's public stuff/archives/dad's user archive/dad's .......

The -8088 error is particularly bothersome because Time Machine would trip on this error and then quit but declare success. Any file that would have been backed up after this error occurs will simply not be in the backups set. Fortunately, the Console error message provides the ENTIRE path to the offending file and it can be easily located and deleted or excluded. However, since Time Machine trips on the FIRST bad file that it finds, it has to be run again and again to grind through all of them until it completes without errors. Note, than in 10.5.2, Time Machine may declare a -8088 error, but at least it doesn't choke on it.

In order to see these error messages, the Console (or some other method) must be used to examine the logs. The Console is located in the Utilities folder. A typical Time Machine run looks like this:

Note that "ALL MESSAGES" has been selected in the left column (if the column doesn't show, click the icon right above the column) and that "backupd" has been entered into the Filter bubble. This down selects the messages to those mostly generated by Time Machine. This same data is recorded in system.log.

The messages themselves are pretty self explanatory and this backup is normal except that it actually copied data in two blocks. This is because things were changing while Time Machine was running and it picked up those changes in a second pass.

Time Machine allocates more space for it's backup than it really needs. I've seen it allocate from 1.2 times to 100,000 times more disk space than it eventually uses. Fortunately, when it badly over-allocates space, it does it from a very small basis so that the actual allocation request doesn't seem to get outlandish. The 1.2 times case can be seen in the Console display above. An example of a wild case is shown below:

2/29/08 12:37:13 PM /System/Library/CoreServices/backupd[766] No pre-backup thinning needed: 363.7 MB requested (including padding), 291.97 GB available
2/29/08 12:37:17 PM /System/Library/CoreServices/backupd[766] Copied 507 files (3 KB) from volume dadsiMac.

If Time Machine seems to be misbehaving, then the Console is about the only place that you can get the data that you need to figure out what is going on. I've seen Time Machine issue just four kinds of errors.

• The -8088 error MUST be addressed or Time Machine just won't work unless you have upgraded to 10.5.2.
• A -43 error means "file not found" (I think). Don't worry too much about this one, it just means that Time Machine had expected to find a file there but something else had probably deleted it or tied it up before Time Machine got there. Sometimes Time Machine will note this error and skip right by it without difficulty. Sometimes Time Machine will post a dialog box stating that had an error and quit. The next time Time Machine runs, it'll complete the backup. These errors happen on my machine several times a week but never seem to be a permanent problem.
• Sometimes Time Machine will declare a -36 error for a file and then immediately declare a -8062 error for the same file and then quit. These don't seem to be repeatable for the same file on subsequent runs so they might be related to something else that happened while that particular file was being processed.

When Time Machine backs up to a disk attached to a server, it saves it's data in a "sparsebundle" which is a disk image as opposed to the regular looking file structure used on a locally mounted disk. The sparsebundle looks like just one file to the server's file system but Time Machine and Spotlight can work inside the sparsebundle and do whatever they want, even if the native file system on the server won't support Apple Filing Protocols. This allows the disk image to be moved from one server to another, even of a different kind. When Time Machine is pointed back to it's new home, it'll pick up from where it left off. Initial indications from Time Capsule users is that the regular file structure on a local disk cannot be moved to a server. If you've been using a local disk for backup and want to switch to a server, or go the other way, you'll have to start a new backup set in the new location.

The sparsebundle is all that is mounted during a remote Time Machine run. It will contain its own Spotlight index. Therefore it has to be mounted for Spotlight to be able to see it. If Time Machine detects that the sparsebundle is being indexed by Spotlight, Time Machine will wait until Spotlight is done. Herein lies a problem.

If the Spotlight index gets corrupted for some reason and Spotlight is trying to fix it over a remote connection, Spotlight can take hours to complete. All the while Time Machine is waiting and issuing messages to that effect in the Console every 30 seconds. If the user has detected that things are going really slowly and tried to fix it by putting the remote volume in Spotlight's privacy list, then Spotlight will quit indexing, but it may leave behind a partial or corrupt index. Time Machine may STILL wait for Spotlight to finish but it never will.

Deadlock

Time Machine can operate without a Spotlight index in the sparsebundle, but Time Machine seems to have problems with a partial or corrupted index. Either the index needs to be completely deleted or Spotlight should be allowed the time it takes to fix the index. Then Time Machine can proceed through its "preparing" phase without waiting for Spotlight. This can still take many hours, but at least it will eventually finish.

To allow Spotlight fix the index, follow these steps:

1. Click the Time Machine application on the Dock. This causes Time Machine to mount the disk image
2. Cancel out of Time Machine, this leaves the disk image mounted.
3. Turn Time Machine off in the Time Machine Preference Pane, this gets Time Machine out of the way.
4. Remove the disk image from the Spotlight Privacy list if it is there
5. Let Spotlight grind away, this could take many hours
6. When Spotlight is done (no pulsing dot in the Spotlight icon on the menu bar), turn Time Machine back on
7. Let Time Machine finish, this could again take many hours if it has to do a deep traversal, which is likely

If you apply the Time Machine and Airport update V1.0 (released March 19, 2008) you also need to update the firmware in the base station to the latest version. A prompt to do this SHOULD happen automatically when the Airport Utility is opened, but sometimes it does not. In that case, select "Check for Updates" from the Airport Utility menu and the update will be located and downloaded. You can then apply the firmware update.

NOTE: There have been Airport firmware updates that have broken TM's ability to back up to an AirDisk. The v7.6 version broke the Fast Ethernet version while v7.5.2 and v7.6.1 worked. Don't update the firmware lightly. If it is working, it may be better to leave it alone or be prepared to backdate the firmware if the reliability degrades.

Time Machine can also have difficulties if the sparsebundle (which represents a disk to the computer being backed up) has directory errors. If Time Machine isn't doing what it should, then you may want to use Disk Utility to repair the disk. However first you have to mount the sparsebundle. To do this, open Time Machine from the Dock icon and then cancel out. This will leave the disk image mounted in a fashion that is actually readable. Then open Disk Utility and Repair the disk as you would any other. Be prepared to wait for awhile as Time Machine backup volumes have lots of multi-linked files and Disk Utility has to check each one. If Disk Utility can repair the volume, Verify it again just to be sure. Sometimes Disk Utility says that it has repaired a volume but the problem isn't fixed. If Disk Utility cannot actually repair the problem, then it best just to delete the sparsebundle and start over with a new initial backup. However BEFORE you do that, you should check the whole backup disk with Disk Utility. There may be problems outside the sparsebundle as well.

In general, the TM backup disk should be at least twice as large as the disk being backed up, full or not. This is to prevent a gridlock situation that CAN occur when TM decides, for whatever reason, that it needs to do another full backup. If the internal disk is nearly full, the initial TM backup will take half of the TM disk. Then add some normal backups on that and then TM decides that it is uncertain of the validity of the backup set and wants to make another full backup. It can't because there isn't enough room for TWO full backups and some other stuff. It won't delete the initial backup if it can't be sure that a full backup is actually there. If it manages to complete the 2nd full backup, then it can delete much of the 1st one and then there is plenty of room again.

In my experience, if the TM disk is 4x the size of a roughly half full internal disk, and really large files are not moved or changed often, then the life of the backup set is about 2 years before it has to prethin. As new files are changed/added, the oldest backups will be removed to make room for the new stuff.

The whole initial backup is not removed. Only files that exist ONLY in the oldest set are removed. If any file is present in any newer backup set, it is not thinned until such time as the only set that represents it is the oldest set. This means that, in the example above, only files that haven't been there in two years are subject to thinning.

To TM, a file is current if the name, location and date have not been changed. Once a file is updated, renamed or moved to a new location, it is a NEW file to TM and it is subject to backup. Therefore in the case of a large VM (virtual machine) file, every time you use it, it becomes a new file. TM will keep a series of them dating back as far as the oldest backup set.

If you use lots of large files and change or move them every day, this can materially shorten the life of a backup set because you might have many copies of versions of the file. If you use Parallels every hour of every day, there are 24 hourly copies of the VM, all distinct in TM. Then there are 26 daily backups (4 of them will become weekly backups and are counted with the weeklies) and if your backup set is a year old, there are 52 weekly backups. This is 102 distinct copies of the VM. If the VM is 20 GB, that makes 2 TB worth of them. It is true that every hour when you create a new one, a previous daily has been thinned so that there is always just enough room for the new version of the VM for the next backup. The total number of VMs stored won't grow because every full day 24 old ones have been thinned and 24 new ones added. Also on that day, an old daily has been post-thinned making room for one of the hourly backups to be increased in rank to a daily backup. Once a week, one of the weekly backups must be thinned to make room for a new weekly. If the backup disk is indeed full, this is when pre-thinning happens. The oldest backup in the set with the oldest VM in it is deleted. Removal of the oldest VM makes room for a new VM assuming that the new VM is the same size as the oldest one. As the VM size increases, TM may have to pre-thin the two oldest weekly backups with smaller VMs to make room for one larger VM.

Keeping a rotating set of 102 VM's totaling 2 TB means that the backup disk must be MUCH larger than 2 TB. If it isn't, TM won't be able to keep anything else and it will keep pre-thinning until there is room for the rest of the stuff, resulting in a much shorter backup set life.

I keep several VMs on my Mac, totaling now just over 100 GB. However, they don't get modified more often than once a week in general so that a TM disk 4x the size of my 3/4 full internal disk resulted in a backup set life of about 6 months. I have bumped the TM disk size to 6x but I started a new backup set in the process so that it will take some time to see how long that lasts in practice.

The bottom line is that if you use TM and you use large files, usually VM or DV, and you modify them fairly often, then get a really big backup disk. If you do not use lots of really large files, many GB each, then you can get away with a TM disk that is 3 or 4 x the size of your internal disk. If you don't let the volume of data on your internal disk to grow to nearly full, then you can get away with a smaller TM disk, maybe 4x the size of your data.

Disks are cheap now, roughly $0.05 per GB.

Time Machine is basically slow, and depending on conditions, it can get even slower. For a locally connected disk, figure on rates of about 15 GB/hour at best. For an ethernet connected disk, half that. For a wirelessly connected disk, half that (or worse) again. Do your initial backup to a remote disk via ethernet if possible, THEN let it do the incremental backups wirelessly.

It is not particularly important that Time Machine runs quickly. What is important is that it runs at all. As long as the Time Machine process doesn't materially impact the foreground processes that the user experiences, then it can run at practically any speed. The AirDisk function of an Airport Extreme Base Station, or a Time Capsule, or even a server connected disk are never going to nearly match the speed of a locally connected disk so that they may not be acceptable for storage and retrieval of user data, especially in large quantities. But for Time Machine these remotely connected disks are good enough.

The performance and obtrusiveness of Time Machine can vary by a great deal depending on the capability of the computer and connectivity of the external disk to the computer being backed up. In the better cases, Time Machine isn't even noticeable. In less than ideal cases, it can take forever to do it's job and materially impact the user experience by hogging resources for a long period of time. It is important that Time Machine be non-obtrusive or it will simply be turned off.

The "better" case is a late model Mac with a dual core CPU and a directly connected (USB or FireWire) backup drive. In this case, TM can run without even being noticed.

The "less than ideal" case involves one or more of these things. The more of them there are, the worse it gets.

• Minimally supported Macintosh (~1 GHz G4)
• Remotely connected disk (Time Capsule, AirDrive or disk connected to a server)
• Connection via Airport (especially 802.11g)

If all three of these conditions are present, Time Machine can be a pain. Remove any one condition, and Time Machine seems to work well enough.

This describes the user of a G4 iBook using a wireless connection to a remote disk. In this case, Time Machine can appear to not work at all. It can take many hours, or even days, to do the initial backup and even small incremental backups can suck up so much CPU that there is little left for the user. Further, even moderate backups can take many minutes and when Time Machine is running under these conditions the computer itself can be dog slow from the user's perspective for an unacceptable period of time.

However, all is not lost. Since a user with a slow computer is not likely to have the option of increasing the performance of the computer (without dropping a grand or more), then the obvious thing to do is to work on the other bottlenecks. Simply plugging the computer into Ethernet when a large backup is expected makes a serious difference. However, if a laptop user is tethered by a wire, then why not just dedicate a disk to the laptop and connect it via FireWire or USB once in a while? Even a marginally supported Macintosh will do fine with a directly connected disk. Time Machine then won't do backups every hour, but it will wait patiently for the disk to be reconnected so that it can do backups during windows of opportunity.

I ran a simple test of remote disk connectivity on a 1.25 GHz G4 PowerBook. Connected wirelessly to a remote disk connected to a server, it took exactly 20 minutes to back up 51.6 MB of data. Connected via Gigabit Ethernet to the same server, it did a backup of a copy of the same data set in 80 seconds. These times were the start to finish time including finding and mounting the remote disk, preparing, doing the backup, cleaning up and disconnecting the remote disk.

Another example of the degradation due to a wireless connections is the "preparing" phase. After a crash, Time Machine must scan the whole disk to determine what it has to do because it cannot trust the information that it left behind. Connected via an Ethernet connection, this phase would typically take 20 minutes on an older PowerBook. Being connected wirelessly stretches this phase out to over 3 hours.

Upgrading the computer itself solves this performance problem in another way. Time Machine can eat a 1 GHz iBook G4 alive. It will consume 80 to 100% of the CPU and make the computer very sluggish. If the user experience is impacted, which it would be, then the backup has to happen quickly. However, while doing the same work on a 2 GHz dual core Intel CPU Macintosh, Time Machine doesn't soak up enough CPU to notice even if the backup does run for a long time due to a constrained pipe to the disk.

Time Machine can perform it's backups to a locally connected USB or FireWire disk (the preferred method) or to a remote disk (often a more convenient method for laptop users). This remote disk can be Macintosh file server, a Windows file server, a Linux file server, a network attached storage device or Apple's own Time Capsule (an Airport eXtreme Base Station with an integral disk). It can also be an AirDisk. Time Machine can handle all these different server formats because when it saves backups to a remote disk, it only creates one file that the server can recognize so that it fits well into a variety of file systems. This file is called a "sparsebundle." To a Macintosh, a sparsebundle represents a disk image of entire Macintosh disk so that the Mac OS can do anything it wants WITHIN the sparsebundle without the file server, whatever it is, being any the wiser.

The AirDisk is the subject of this section because that is what I use and it is what I have experience with. I haven't used the other forms of remote servers so I cannot comment on those.

An AirDisk is a remote disk connected to an Airport eXtreme Base Station (the squarish ones). When Leopard and Time Machine were first released, Time Machine didn't even work with an AirDisk. Later, Apple released the Time Capsule and that did work with Time Machine BUT the AirDisk still did not. Sometime later, a firmware upgrade was issued that allowed Time Machine to work with an AirDisk BUT this feature was "unsupported." The term "unsupported" generally means "it sort of more-or-less works but there are known problems that we don't know how to/can't/won't fix because we've got other things to do. Use it at your own risk."

The AirDisk does indeed work, but it also seems to have problems in some circumstances. I am not sure if this is a basic fault with the AirDisk or just an issue with wireless connectivity. I have six computers that back up to a 1 TB AirDisk on an older "fast" ethernet AXBS. Sometimes problems develop but they seem be be related to the way that the computer connects to the disk.

Sometimes, TM on some computers just refuses to work properly. This may be the result of corruption in the directories of either the AirDisk itself or of the directories within a particular sparsebundle. When these problem occur, it is sometimes possible to fix them. Sometimes the only solution is to deleted the whole backup set for the impacted computer and start over.

This is my procedure for testing and repairing an AirDisk. It uses only Apple tools and is easy to do if not a little time consuming. I don't know if the Time Capsule suffers these same problems or the exact procedure for recovering from those problems if they do occur. This procedure for fixing an AirDisk is somewhat abbreviated as it assumes that you are familiar with the tools, the Airport Utility and Disk Utility.

AirDisk Repair Procedure

1. Launch the Airport Utility and Disconnect All Users from the AirDisk
2. Unplug the AirDisk from the AXBS and plug it directly into a Macintosh
3. Launch Disk Utility and Verify (or Repair if desired) the top level disk
4. If a repair was done, repeat until a Verify indicates that the disk is clean
5. Double click on a sparsebundle within the AirDisk
6. Either wait (for hours maybe) for the disk to mount or press "Skip" to get by the automatic disk check
7. Wait until the sparsebundle mounts as a disk image in Disk Utility
8. Either Verify or Repair the sparsebundle
9. If a Verify was successful, the sparsebundle is good, dismount it and move on
10. If a repair is necessary, it can take hours
11. If the repair is declared as successful, then VERIFY again. Sometimes Disk Utility will declare a successful repair but it didn't actually fix the problem.
12. After a good Verify, move on
13. If the repair or subsequent verify fails, especially if fatal errors were reported, the sparsebundle is toast. It will have to be deleted and Time Machine will later be used to create a new one. All history of past backups is lost for the impacted computer.
14. Once each sparcebundle is either good or has been nuked, eject the backup disk and reconnect it to the AXBS
15. Run Time Machine on each computer (connected via Ethernet) that lost it's sparsebundle and create a new backup. Do this overnight.

I recently ran this process because my wife's MacBook refused to back up properly. I went through all six sparsebundles on my backup disk and I noticed some things related to the way that Time Machine is used that seem to have some bearing on damage to the individual sparsebundles.

If you note the connection method and the status of the sparsebundles, all of the computers that connect via ethernet have had no problems. All of the ones that connect wirelessly seem to be having difficulties.

Update 7 Feb 12

Nearly four years have gone by since the last update to this page and I am still using the AirDisk backup method successfully. I now have a 4th generation Airport Extreme Base Station (simultaneous dual band II) base station with a Belkin USB hub and two large disks (2 TB and 3 TB) disks attached. The firmware version is v7.6.1. I also still have the "fast ethernet" version (mentioned above) in service at another location with a 2 TB disk attach (also v7.6.1). Both systems work reliably with Leopard, Snow Leopard and Lion.

Some changes have been made in Snow Leopard and even more changes in Lion that reduce the probability that a sparsebundle going bad without you knowing about it. In Snow Leopard, if the sparsebundle clearly had problems, TM will declare the backup no good and it is toast. You will have to start another one, but at least you aren't backing up to a sparsebundle that you may not be able to use when you need it. Lion further adds a periodic (roughly monthly) "verification" of remote backups. This verification takes some time, but it runs in the background and you may not notice it.

Update 26 Mar 12

Some months back, Apple updated the firmware for the "square" Airport Base Stations to v7.6.1. This firmware update seems to be good for the 1st generation square AEBS (Fast Ethernet) through the 4th generation (simulatious dual band) versions at least. There has been no corruption of TM sparsebundles for any computer that I still have using TM, Leopard onward.

[ Previous Tip ] [ Mac Tips ] [ Next Tip ]