A disk compression software utility increases the amount of information that can be stored on a hard disk drive of given size. Unlike a file compression utility which compresses only specified files - and which requires the user designate the files to be compressed - a disk compression utility works automatically without the user needing to be aware of its existence. When information needs to be stored to the hard disk, the utility will compress the information. When information needs to be read, the utility will decompress the information.
- 1 Early Disk Compression Utilities
- 2 Modern Disk Compression Utilities
- 3 Common early disk compression utilities
- 4 How early stage disk compression works
- 5 Modern disk compression utilities
- 6 How modern disk compression works
- 7 See also
- 8 References
- 9 External links
Early Disk Compression Utilities
Early disk compression utilities override the standard operating system routines. Since all software applications access the hard disk using these routines, they continue to work after disk compression has been installed.
Disk compression utilities were popular especially in the early 1990s, when microcomputer hard disks were still relatively small (20 to 80 megabytes). Hard drives were also rather expensive at the time, costing roughly 10 USD per megabyte. For the users who bought disk compression applications, the software proved to be in the short term a more economic means of acquiring more disk space as opposed to replacing their current drive with a larger one. A good disk compression utility could, on average, double the available space with negligible speed loss. Disk compression fell into disuse by the late 1990s, as advances in hard drive technology and manufacturing led to increased capacities and lower prices.
Note: While the most familiar disk compression utilities were designed to work on DOS systems, the concept was not specific to DOS. The utility DiskDoubler, for example, worked on the Apple Macintosh platform.
Modern Disk Compression Utilities
Starting in the 2010s, increased hard drive costs due to the advent of fast but expensive SSD technology, and the growth of fixed storage slate tablets with non-upgradeable storage, led to a resurgence of disk compression utilities. Windows 10 includes a new option to compress Windows binaries and program files. ZIPmagic offers three disk compression solutions.
Modern disk compression solutions do not override the native operating system routines, but leverage compression technologies built into the operating system. For example, ZIPmagic DriveSpace is based on NTFS compression, a feature of Windows for two decades, first introduced in Windows NT 3.51; ZIPmagic DoubleSpace similarly extends WIMBoot, a very recent Windows feature.
Common early disk compression utilities
The initial compression utilities were sold independently. A user had to specifically choose to install and configure the software.
- Stacker from Stac Electronics
- XtraDrive from Integrated Information Technology ( IIT)
- SuperStor Pro from AddStor
- DoubleDisk Gold from Vertisoft Systems
- DiskDoubler from Salient Software
The idea of bundling disk compression into new machines appealed to resellers and users. Resellers liked that they could claim more storage space; users liked that they did not have to configure the software. Bundled utilities included:
- DR DOS 6.0, from Digital Research, included a version of SuperStor.
- PalmDOS 1.0, from Novell (after acquiring Digital Research), included a DPMS-enabled version of SuperStor.
- MS-DOS 6.0 and MS-DOS 6.2, from Microsoft, included DoubleSpace, which was based in part on Vertisoft Systems' DoubleDisk.
- PC DOS 6.1, from IBM, included a version of SuperStor.
- MS-DOS 6.22, from Microsoft, included DriveSpace, which was DoubleSpace with a different compression algorithm.
- PC DOS 6.3, from IBM, included a version of SuperStor.
- Novell DOS 7, from Novell (after acquiring Digital Research), included a DPMS-enabled version of Stacker 3.12.
- PC DOS 7.0, from IBM, included a DPMS-enabled version of Stacker 4.02 from Stac Electronics.
- OpenDOS 7.01, from Caldera (after acquiring Novell DOS), included a DPMS-enabled version of Stacker 3.12.
- DR-DOS 7.02/7.03, from Caldera, included a DPMS-enabled version of Stacker 3.12.
- PC DOS 2000, from IBM, included a DPMS-enabled version of Stacker 4.02.
- Windows 95, from Microsoft, included an updated version of DriveSpace.
- Plus!, from Microsoft, included DriveSpace 3.
- Double Tools for DoubleSpace from Addstor, Inc. was an add-on product, enhancing the functionality of the DoubleSpace bundled with MS-DOS 6.0.
- Multimedia Stacker from Helix Software was a bundle of Stac's DPMS-enabled Stacker 4.01 combined with Helix Cloaking and utilities, utilizing Cloaking's built-in DPMS server to relocate and run in Protected mode.
While Windows XP, from Microsoft, included both a native support and a command line utility named 'compact' that compresses files on NTFS systems, that is not implemented as a separate "compressed drive" like those above.
How early stage disk compression works
Disk compression usually creates a single large file, which becomes a virtual hard drive. This is similar to how a single physical hard drive can be partitioned into multiple virtual drives. The compressed drive is accessed via a device driver.
Compressing existing drives
All drives would initially be empty. The utility to create a drive would usually offer to "compress a current drive". This meant the utility would:
- Create an empty compressed drive, stored on the existing drive.
- Transfer existing files on the old drive to the new compressed drive.
- Increase the size of the new compressed drive as necessary to accommodate more files and allow empty space when done.
- When all files were transferred, the drive letters would be swapped.
Compressing the boot drive
Note that the device driver had to be loaded to access the compressed drive. A compressed drive C: required changes to the boot process as follows:
- BIOS loads sector 0 of the first physical hard drive (partition sector)
- Partition sector loads sector 0 of the bootable partition. In this case, it's the host drive.
- Host drive sector 0 loads (in the case of MS-DOS) IO.SYS and begins Config.Sys processing
- Compression device driver is loaded. Compressed drive becomes C; host drive usually became F.
- Processing continues from compressed drive.
On systems with slower hard drives, disk compression could actually increase system performance. This was accomplished two ways:
- Once compressed, there was less data to be stored.
- Disk accesses would often be batched together for efficiency.
If the system had to frequently wait for hard drive access to complete (IO bound) converting the hard drive to compressed drives could speed up the system significantly. Compression and decompression of the data will increase the CPU utilization. If the system was already CPU bound, disk compression will decrease overall performance.
Some common drawbacks to using disk compression:
- Not all compression utilities would confirm the absence of errors in the file system before compressing a disk in place. Some errors, such as crosslinked files, could result in additional data loss during the transfer process.
- The compressed drive is only visible if the device driver is loaded and the compressed drive is mounted. A boot disk, for example, might not contain the driver.
- Users did not always realize that the large file on the host drive contained the compressed drive. While it was usually "hidden" by default, users who did find the large file curious or suspicious were able to delete it. This would normally result in data loss.
Modern disk compression utilities
- Windows 10 Disk Cleanup Utility from Microsoft (OS based)
- DoubleSpace from ZIPmagic Software (WIMBoot based)
- DriveSpace from ZIPmagic Software (NTFS compression based)
How modern disk compression works
WIMBoot creates a single compressed WIM file which is read-only and highly compressed. Pointers are extracted from the WIM file to the target file system; these pointers work in conjunction with the WoF (Windows overlay Filter) driver to serve actual file data from the compressed WIM file. Unlike early stage disk compression tools, when these files are updated, the changes are not compressed or stored back inside the WIM file; instead, any time a file is opened for write access, the WoF drive fully extracts the original compressed file to the target file system - from this point onwards, the file is uncompressed. One drawback of this approach is that space is wasted on disk because of duplicated storage for files extracted in this way: Files opened for write access under WIMBoot consume space both inside the WIM file and outside of it, uncompressed.
NTFS compression is transparent and works on a file-by-file basis. Compressing an entire disk with NTFS compression processes each file on disk individually. As files are read from or written to, data is transparently decompressed and recompressed. In the case of large files, the recompression process often continues in the background for an indeterminate amount of time, until the file system has been able to transparently compress all file data. While NTFS compression is very fast (having been designed in the early 1990s), its compression ratios are not nearly as impressive as WIMBoot.
NTFS compression has been demonstrated to significantly improve read performance on compressed systems, with similar expectations for WIMBoot. Write performance is typically not enhanced but tolerably degraded with either technology.
- In crosslinked files, two files are storing at least part of their data in the same location. At least part of one file (the "bad" file) is always lost in this instance. However, if the "bad" file is copied and then deleted, part of the "good" file is deleted as well. Microsoft ScanDisk was created, in part, to perform a better check of the file system prior to compression than the old MS-DOS CHKDSK utility.
- For example, DOS associated up to four attributes with files: System, Hidden, Read-Only, and Archivable. Files with the System or Hidden attributes are often not displayed by default. Files with the System or Read-Only attribute cannot be deleted with the "Erase" (or "Del") DOS command. Most compression utilities would mark the drive file with at least one or more of the System, Hidden, and Read-Only attributes (many would use all three). However, files marked with such attributes can be viewed and deleted by other means. In addition, the user can also remove attributes.