Disk cloning

Disk cloning is the process of copying the contents of one computer hard disk to another disk or to an "image" file. This may be done straight from one disk to another, but more often, the contents of the first disk are written to an image file as an intermediate step, then the second disk is loaded with the contents of the image. Typically, this is done for archiving purposes, to restore lost or damaged data, or to move wanted data into a new disk, though other reasons also exist.

Unlike standard copying functions, disk cloning involves copying hidden and in-use files, and thus presents special challenges, as those types of files are typically not available for copying. Additional complications arise when the process is used for networked computers, as the network must be able to distinguish between different computers. Post-cloning operations may be necessary to address these and other issues.

Common uses

There are a number of notable uses for disk cloning software. These include:

• Reboot and restore – a technique in which the disk of a computer is automatically wiped and restored from a "clean", master image, which should be in full working order and should have been swept for viruses. This is used by some cybercafes and some training and educational institutes, and helps ensure that even if a user does misconfigure something, downloads inappropriate content or programs, or infects a computer with a virus, the computer will be restored to a clean, working state. The reboot and restore process can either take place irregularly when a computer shows signs of malfunctioning, on a regular basis (e.g., nightly) or even, in some cases, every time a user logs off, which is the safest approach (although that does involve some downtime). Notable third party applications that provide reboot and restore are Drive Vaccine and Deep Freeze.
• Provisioning new computers – Provisioning with a standard set of software so that a new user is ready to go straight away with a complete application suite and does not have to waste time installing individual applications. This is often done by original equipment manufacturers and larger companies.
• Hard drive upgrade – An individual user may use disk copying (cloning) to upgrade to a new, usually larger, hard disk. Recently more users use cloning to move to the faster Solid State Drive SSD. Disk cloning is also useful when reconditioning an SSD.
• Full system backup – A user may create a comprehensive backup of their operating system and installed software.
• System recovery – An OEM can provide media that can restore a computer to its original factory software configuration.
• Transfer to another user – A system sold or given to another person may be reset by reloading a known, previously-saved image that contains no personal files or information.

How it works

This article is specific to disk cloning on the x86 and PowerPC platform; specific details may not apply to other platforms.

To provision the hard disk of a computer without using disk cloning software, the following steps are generally required for each computer:

1. Create one or more partitions on the disk
2. Format each partition to create a file system on it
3. Install the operating system
4. Install device drivers for the particular hardware
5. Install application software

With disk cloning, this is simplified to:

1. Install the first computer, as above.
2. Create an image of the hard disk (optional)
3. Clone the first disk, or its image, to the remaining computers.

This can be referred to simply as a recovery disc.

History

Before Windows 95, some computer manufacturers used hardware disk copying machines to copy software. This had the disadvantages of copying not just the used data on the disk, but also unused sectors, as the hardware used was not aware of the structures on the disks. A larger hard disk could not be copied to a smaller one, and copying a smaller one to a larger left the remaining space on the new disk unused. The two disks required identical geometries.

Other manufacturers and companies partitioned and formatted disks manually, then used file copy utilities or archiving utilities, such as tar or zip to copy files. It is not sufficient simply to copy all files from one disk to another, because there are special boot files or boot tracks which must be specifically placed for an operating system to run, so additional manual steps were required.

Windows 95 compounded the problems because it was larger than earlier popular operating systems, and thus took more time to install. The long filenames added to the FAT filesystem by Microsoft in Windows 95 were not supported by most copy programs, and the introduction of the FAT32 filesystem in 1997 caused problems for others. The growth of the personal computer market at this time also made a more efficient solution desirable.

Ghost was introduced in 1996 by Binary Research. It initially supported only FAT filesystems directly, but it could copy but not resize other filesystems by performing a sector copy on them. Ghost added support for the NTFS filesystem later that year, and also provided a program to change the Security Identifier (SID) which made Windows NT systems distinguishable from each other. Support for the ext2 filesystem was added in 1999.

Competitors to Ghost soon arose, and a features war has carried on to the present day. Many disk cloning programs now offer features which go beyond simple disk cloning, such as asset management and user settings migration.

On UNIX based computer systems, dd was more commonplace due to the lack of filesystem support in Ghost.

Post-cloning operations

If cloned disks are deployed to multiple machines, for example a standard PC build within a corporation that is officially supported by that organisation's IT department, then there are a few issues to consider. Most of these issues are either to do with the problem of ensuring that machines can be uniquely identified on a local area network, or to do with hardware differences.

Uniqueness within a network

The use of DHCP can assist in assigning unique IP addresses to otherwise identical machines. DHCP can also allocate a unique hostname. Machines can be given persistent identities from DHCP by identifying the network hardware's unique MAC addresses.

Sometimes it is easier to allocate a unique hostname in a post-cloning operation. Under most variants of Linux, this is simply a case of echoing a unique name to the /etc/hostname file. For Microsoft Windows machines, programs such as Sysprep can perform these kinds of post-cloning tasks.

Microsoft Active Directory and network domains

For Windows NT and its successors, having two computers with identical machine SIDs (Security Identifiers) was considered to be serious security risk for many years, without actual proof of this being true. Problems With Duplicate SIDs. This has changed over time and now duplicated machine SIDs are not considered to be a problem. Nevertheless, many Windows disk cloning programs will change the machine SID of clones because of the previous belief that this was necessary.

When it comes to "Domain SID", the Domain SID is recomputed each time a computer enters a domain. Thus, all the "post-cloning operations" that are based on "leave the domain and then rejoin the domain" will actually cause a re-creation of the Domain SID for the computer that joins the domain. In other words, duplicated SIDs are usually not a problem with Microsoft Windows systems

Hardware and storage differences

Cloned operating system images from one machine may not operate completely correctly on another machine if that machine has different hardware.

By far the easiest solution to this problem is to use identical hardware. Corporate IT departments often buy large numbers of identical machines for this reason.

Some operating systems are not well suited to changes in hardware, so that a clone of Windows XP for example may object to being booted on a machine with a different motherboard, graphics card and network card, especially if non-generic drivers are used. Microsoft's solution to this is Sysprep, a utility which runs hardware detection scans and sets the SID and computer name freshly when the machine boots. Microsoft recommends that Sysprep be set up on all machines before cloning, rather than allow third party programs to configure them.

Linux systems simply require the necessary kernel modules to be available (or compiled directly into the kernel), in order to support new hardware when the machine boots. For most variants of Linux, the monolithic kernel will automatically load all necessary modules. However, if the hardware is considerably different between the original and the cloned machine, further configuration may need to be performed, or prepared in advance. This is usually simply a case of ensuring that the correct files are present under the /etc directory.

There are ways to help make images for cloning with Microsoft Windows more portable. One such example would be a product called Universal Imaging Utility from Big Bang, LLC which incorporates a large number of hardware device drivers into the sysprep routine.

There are files in some Microsoft operating systems (called BOOTSECT.*) which are copies of the Boot Partition Block (BPB) used by alternate operating systems that Microsoft Windows loader (NTLDR) can load. BOOTSECT.* files may have to be altered if partition sizes or layouts are changed during the clone.

Linux systems usually boot using either the LILO or GRUB bootloaders. These contain lists of absolute disk sectors in their MBR, which must be altered by the cloning program if the files they refer to are not in the same location of the destination disk. For example, if the original boot loader script points to the system being on a disk on channel 0 and the system being of the second partition, the target computer will need to have the same configuration.

Operating environment

A disk cloning program needs to be able to read even protected operating system files on the source disk, and must guarantee that the system is in a consistent state at the time of reading. It must also overwrite any operating system already present on the destination disk. To simplify these tasks, most disk cloning programs can run under an operating system different from the native operating system of the host computer, for example, MS-DOS or an equivalent such as PC DOS or DR-DOS, or Linux. The computer is booted from this operating system, the cloning program is loaded and copies the computer's file system. Many programs (e.g. Acronis True Image MiniTool Partition Wizard, Paragon Partition Manager or AOMEI Backupper) can clone a disk, or make an image, from within the running system, with special provision for copying open files; but an image cannot be restored onto the Windows System Drive under Windows.

A disc cloning program must have device drivers or equivalent for all devices used. The manufacturers of some devices do not provide suitable drivers, so the manufacturers of disk cloning software must write their own drivers, or include device access functionality in some other way. This applies to tape drives, CD and DVD readers and writers, and USB and FireWire drives. Cloning software contains its own TCP/IP stack for multicast transfer of data where required.

Image transfer

The simplest method of cloning a disk is to have both the source and destination disks present in the same machine, but this is often not possible. Disk cloning programs can link two computers by a parallel cable, or save and load images to an external USB drive or network drive. As disk images tend to be very large (usually a minimum of several hundred MB), performing several clones at a time puts excessive stress on a network. The solution is to use multicast technology. This allows a single image to be sent simultaneously to many machines without putting greater stress on the network than sending an image to a single machine.

Image manipulation

Although disk cloning programs are not primarily backup programs, they are sometimes used as such. A key feature of a backup program is to allow the retrieval of individual files without needing to restore the entire backup. Disk cloning programs either provide a Windows Explorer-like program to browse image files and extract individual files from them, or allow an image file to be mounted as a read-only filesystem within Windows Explorer.

Some such programs allow deletion of files from images, and addition of new files.

See also

• List of disk cloning software
• Comparison of disk cloning software
• Disk mirroring
• Disk image
• Live USB
• Recovery disc
• Security Identifier