Linear Tape File System
||This article appears to be written like an advertisement. (March 2013)|
Linear Tape File System (LTFS) refers to both the format of data recorded on magnetic tape media and the implementation of specific software that uses this data format to provide a file system interface to data stored on magnetic tape. The Linear Tape File System format is a self-describing tape format developed by IBM to address tape archive requirements. The LTFS Format specification, which was adopted by the LTO Technology Provider Companies, defines the organization of data and metadata on tape - files stored in hierarchical directory structure. Data tapes written in the LTFS Format can be used independently of any external database or storage system allowing direct access to file content data and file metadata. This format makes it possible to implement software that presents a standard file system view of the data stored in the tape media. This file system view makes accessing files stored on the LTFS formatted media similar to accessing files stored on other forms of storage media such as disk or removable flash drives.
The Linear Tape File System format is an open description of the layout of data-structures stored on a sequential-access media. These data-structures hold the file content data and associated file metadata. Data media, such as LTO data tape, written using this format can be exchanged between systems that understand the Linear Tape File System format. Software systems that understand the format can provide users with a file system view of the media. Software systems may alternatively understand the format only to the degree that allows the system to read data from the media, or produce a tape that can be accepted by other systems that implement the Linear Tape File System format.
- 1 Announcement
- 2 History
- 3 Format specification
- 4 Implementations
- 4.1 IBM Linear Tape File System - Single Drive Edition
- 4.2 IBM Linear Tape File System - Library Edition
- 4.3 Oracle's StorageTek Linear Tape File System, Open Edition
- 4.4 Oracle's StorageTek Linear Tape File System, Library Edition
- 4.5 HP Linear Tape File System
- 4.6 Quantum Linear Tape File System
- 5 LTFS compatible products
- 6 LTFS projects
- 7 Industry recognition
- 8 References
- 9 External links
The Linear Tape File System format was announced on April 12, 2010, and presented at the National Association of Broadcasters (NAB) trade show. This announcement was made in conjunction with announcing the public availability of the IBM Long Term File System software product that uses the LTFS Format.
Magnetic tape data storage has been used for over 50 years. Prior to the introduction of the Linear Tape File System format, tapes used in open systems typically did not hold file metadata in a form that is easy to access or modify independent of the file content data. Instead, the tapes often contained files stored in a trivial sequence, without any directories or even filenames. Various external databases were used to maintain file metadata (file names, timestamps, directory hierarchy) to hold this data independent of the tape media.
For example, both backup software and hierarchical storage management (HSM) software typically maintain a database separate from the tape media. This type of software often allows indexing and accessing data spread over entire tape libraries, but these external databases are generally not designed for interoperability. Tapes in the system may or may not contain an index of their content. The index format depends on the software used to write the tape and is generally not interoperable with other systems.
In Unix-like systems, the tar format is an interoperable standard designed to allow encapsulation of multiple files for tape storage. Tar includes the files content and file-based metadata in a per-file header structure but does not maintain a central index of files. Tar allows one to create self-describing tapes, but does not provide a filesystem interface or characteristics. The tar format is not well-suited to allow modification of file metadata independent of modifying file content data.
The LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at NAB 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release. The LTFS Format Specification and filesystem implementation were released on April 12, 2010 and enjoyed the support of IBM, HP, Quantum, and the LTO Consortium.
The early versions of LTFS released in 2010 were based on the on-media format described in the "LTFS Format Specification v1.0" document. In March 2011 IBM released an updated document "LTFS Format Specification v2.0.0". The updated specification improved the text to remove ambiguity and clarify the intent of the specification. Additionally, the updated specification extends the LTFS Format to:
- support sparse files,
- support persistent file identifiers,
- define virtual extended attributes for filesystem metadata and control, and
- define minimum and recommended blocksize values for LTFS Volumes. (Primarily for compatibility across various HBA hardware implementations.)
Version 2.0.0 of the LTFS Format Specification defines rules for how the Format Specification version number may change in future and how compatibility is maintained across varying implementations of the specification. In particular, all implementations compliant with version 2.0.0 (and later versions) must correctly read media that was compliant with any prior versions of the LTFS Format Specification. All implementations compliant with version 2.0.0 (and later versions) must write media that is compliant with the "current" version of the LTFS Format Specification. Where the "current" version is the version of the LTFS Format Specification with which the implementation claims compliance. This specifically means that media modified by an implementation compliant with version 2.0.0 of the format specification will implicitly update the media to be compliant with LTFS Format Specification version 2.0.0. (This update only impacts the LTFS Index written to the media by the updated implementation.)
Implementations in compliance with version 2.0.0 (and later versions) of the LTFS Format Specification will correctly read all media generated using prior versions of the LTFS Format Specification.
Support for the LTFS Format Specification version 2.0.0 is included in the LTFS-SDE implementation of the Linear Tape File System version 1.2.0.
|Format Specification Version||Specification Published||Conforming Software|
|LTFS Format Specification v1.0||April 2010||IBM Long Term File System (LTFS) v1.0.0, v1.0.1
HP Linear Tape File System (LTFS) v1.0.0, v1.1.0
|LTFS Format Specification v2.0.0||March 2011||IBM Linear Tape File System - Single Drive Edition (LTFS-SDE) v1.2.0,
Oracle StorageTek Linear Tape File System, Open Edition v1.0.0
|LTFS Format Specification v2.1||October 2012||IBM: Linear Tape File System (LTFS) Format Specification|
SNIA Technical Work Group
In August 2012, SNIA announced that it was forming a TWG (Technical Work Group) to continue technical development of the specification. LTFS Format Specification v 2.1 is the baseline for the technical work and standards accreditation process; SNIA LTFS TWG members include HP, IBM, Oracle and Quantum.
IBM Linear Tape File System - Single Drive Edition
|Initial release||April 2010|
|Stable release||184.108.40.206 (4101) / 6 December 2013|
|Operating system||Linux, Mac OS X, Microsoft Windows|
|Platform||x64, Intel x86 - 32-bit|
|License||GNU Lesser General Public License|
The IBM Linear Tape File System - Single Drive Edition product is software developed by IBM to implement a tape-based file system using the Linear Tape File System Format. This software implements the LTFS Format and allows tapes to be formatted as an LTFS Volume. These LTFS Volumes may then be mounted using the Linear Tape File System software allowing users and applications direct access to files and directories stored on the tape, including drag-and-drop of files.
Using the Linear Tape File System, files can be created on tape and accessed similar to the process of creating and accessing files on an external hard drive or a USB flash drive. Applications, such as file browsers, image viewers and media players can directly browse and access files on tape. LTFS enables easy and simple use of tapes in desktop computers and embedded systems.
IBM released the first version of the Linear Tape File System product as open source software. This release supports the Linear Tape Open Ultrium 5 (LTO-5) drives. Every release of IBM LTFS - SDE is based on the Linear Tape File System Format Specification developed by IBM and released as an open specification.
Prior to version 1.2.0 the "IBM Linear Tape File System - Single Drive Edition" product was distributed under the product name "IBM Long Term File System".
Operating systems supported by IBM LTFS-SDE
- Red Hat Enterprise Linux (RHEL) 5.4, 5.5 with LTFS-SDE 1.2.0 for Linux (latest release: Jun 16, 2011)
- SUSE Linux Enterprise Server (SLES) 11 SP 1 with LTFS-SDE 1.2.0 for Linux (latest release: Jun 16, 2011)
- Mac OS X 10.5.6 (Leopard), 10.6.7 (Snow Leopard) with LTFS-SDE 1.2.0 for Mac OS X (latest release: Jun 16, 2011)
- Windows 7 with LTFS-SDE - Windows version 1.2.0 (latest release: Sept 2, 2011)
- Windows Server 2008 R2 with LTFS-SDE - Windows version 1.2.2 (latest release: Sept 2, 2011)
IBM Linear Tape File System - Library Edition
|Initial release||June 2011|
|Stable release||2.0.0 / 28 March 2011|
|Platform||x64, Intel x86 - 32-bit|
|License||Proprietary commercial software|
The IBM Linear Tape File System - Library Edition (LTFS-LE) product is software developed by IBM to implement a tape-based file system using the Linear Tape File System Format. This software implements the LTFS Format and allows tapes to be formatted as an LTFS Volume. These LTFS Volumes may be used in a tape library along with the LTFS-LE software. The LTFS-LE software allows the user to mount the tape library allowing users and applications direct access to all files and directories stored in the LTFS Volumes. When the library is mounted each LTFS-formatted tape cartridge in the library appears as a separate folder under the filesystem mount point. The user or application can navigate into each of these folders to access the files stored on each tape. The LTFS-LE software automatically controls the tape library robotics to load and unload the necessary LTFS Volumes to provide access to the stored files.
IBM Linear Tape File System - Library Edition implements the LTFS Format Specification version 2.0.0. LTFS Volumes produced in LTFS-LE are fully compatible with the open-source LTFS-SDE software product. LTFS-LE is available for all current IBM tape library products.
Operating systems supported by IBM LTFS-LE
- Red Hat Enterprise Linux (RHEL) 5.4, 5.5 with LTFS-LE 2.0.0 for Linux (latest release: June, 2011)
- SUSE Linux Enterprise Server (SLES) 11 SP 1 with LTFS-LE 2.0.0 for Linux (latest release: June, 2011)
Oracle's StorageTek Linear Tape File System, Open Edition
|Platform||x64, Intel x86 - 32-bit|
Oracle's free open source StorageTek Linear Tape File System (LTFS), Open Edition software enables customers to write files to tape in a self-describing format, much the same way files are written to disk and flash storage devices.
Oracle’s StorageTek LTFS software offering is the first solution to store 8.5TB (native capacity) on a single cartridge. StorageTek LTFS software supports Oracle’s midrange StorageTek LTO 5 and LTO 6 tape drives from HP and IBM as well as Oracle’s StorageTek T10000C and T10000D tape drives.
With Oracle’s LTFS software, customers have the opportunity to archive data with the StorageTek T10000C and T10000D tape drive, while still maintaining the option to exchange data on a midrange StorageTek LTO 5 or LTO 6 tape drive from HP or IBM.
Oracle's StorageTek Linear Tape File System, Library Edition
Oracle’s StorageTek LTFS-LE software offering supports the StorageTek SL8500 Modular Library System, the StorageTek SL3000 Modular Library System, and the StorageTek SL150 Modular Tape.
HP Linear Tape File System
|Stable release||1.1.0 / 29 November 2010|
|Operating system||Linux, Mac OS X|
|Platform||x64, Intel x86 - 32-bit|
|License||GNU Lesser General Public License|
The HP Linear Tape File System (HP LTFS) is a free open source software application which provides a self-describing tape format based on the LTFS specification from the LTO Technology Provider Companies.
HP LTFS provides users with the ability to access files on an LTO-5 tape using the same commands and operations as that of a disk drive. With HP LTFS, users can share content across heterogeneous systems without the need for separate application software to read or write data to tape. Because HP LTFS is an open format, users can be sure that their data is accessible without the need for any proprietary software, making it particularly suited for long-term data archive.
Operating systems supported by HP LTFS
Quantum Linear Tape File System
|Operating system||Windows, Linux, Mac OS X|
|License||GNU Lesser General Public License|
By Quantum Corporation. Windows, Linux and Mac OS X are supported. All Quantum Scalar Libraries with LTO5 / LTO6 drives are supported. The Scalar LTFS Appliance is a file system that presents a Quantum tape library as an NAS share. This appliance makes files viewable as if they resided on your local disk and allows users to drag and drop files directly to and from a tape cartridge. This appliance provides large file portability so that you can quickly and easily retrieve large files, whether they are active or archived, from your library, exchange files with other organizations (inside and outside of your company) for collaborative and sequential work-flow applications, and drag and drop files to and from tape.
LTFS compatible products
- IBM TS1140 Tape Drive
- IBM 4 TB Native Capacity Tape Drive 
- Oracle StorageTek T10000C Tape Drive
- Oracle 5 TB Native Capacity Tape Drive 
- HP StorageWorks LTO-5 Ultrium 3000
- HP LTO5 external half height tape drive, 6 Gbit/s SAS
- HP StorageWorks LTO-5 Ultrium 3280
- HP LTO5 full height tape drive, 6 Gbit/s SAS
- IBM System Storage TS2250 Tape Drive
- IBM LTO5 external stand-alone half height tape drive, 6 Gbit/s SAS interface
- IBM System Storage TS2350 Tape Drive
- IBM LTO5 external stand-alone full height tape drive, 6 Gbit/s SAS interface
- IBM LTO5 FH FC Tape Drive
- IBM LTO5 internal full height tape drive, 8 Gbit/s Fiber Channel interface
- IBM LTO5 FH SAS Tape Drive
- IBM LTO5 internal full height tape drive, 6 Gbit/s SAS interface
- IBM LTO5 HH FC Tape Drive
- IBM LTO5 internal half height tape drive, 8 Gbit/s Fiber Channel interface
- IBM LTO5 HH SAS Tape Drive
- IBM LTO5 internal half height tape drive, 6 Gbit/s SAS interface
- Quantum LTO5 HH SAS Tape Drive
- Quantum LTO5 external half height tape drive, 6 Gbit/s SAS interface
- Quantum LTO5 FH SAS Tape Drive
- Quantum LTO5 internal half height tape drive, 6 Gbit/s SAS interface
- 1 Beyond Wrangler LTO-5 Offload Stations
- The 1 Beyond Wrangler LTO-5 Offload Stations is an appliance to store broadcast contend directly on tape via LTFS.
- Cache-A’s LTO-5 based products
- Cache-A Corporation demonstrated one of the early integrations of the LTFS format on LTO Gen5 using IBM's LTFS codebase at NAB 2010. Cache-A is continuing this development with the HP codebase to provide LTFS-based archive appliances that provide access to this technology without requiring any client-side software.
- Crossroads StrongBox LTFS tape-based archive
- Crossroads announced on April 11th, 2011, to launch in a new tape-based archive product that leverages the Linear Tape File System (LTFS) technology. This solution will serve as an online archive for tier3(Tertiary storage) data, presenting itself as disk storage, but incorporating both disk for fast file storage and retrieval, and physical LTO tape for cost-effective, long-term capacity storage.
Products available: StrongBOX T1 and StrongBOX T3.
- FOR-A LTR-110HS/120HS Video Archive Recorder
- The FOR-A LTR-100HS digital video recorder device is using IBM LTFS formatted tape cartridges. It was introduced at the NAB 2010 and won the IBC 2010 Pick Hit Awards from Broadcast Engineering. Its successor is the FOR-A LTR-120HS [AVC-Intra/DVCPRO Model], which supports additional MXF formats.
ISVs (Independent Software Vendors) supporting LTFS
- BRU PE
- TOLIS Group's BRU Producer's Edition version 2.3.0 provides native LTFS (Linear Tape Filesystem Support) for OS X users.
- SGL (Software Generation Limited) a UK-based broadcast archive software solutions company, was the first Independent Software Vendor (ISV), which announced their support of LTFS with their FlashNet product.
- Masstech Group Inc.
- Masstech Group Inc., a Toronto, Canada-based developer of media asset management, archiving, workflow automation and transcoding solutions for the media and entertainment industry, announced LTFS support across the company's solution portfolio in 2012.
- PreRollPost can be used with any LTFS compliant tape drive within Macintosh OSX 10.7 and 10.8. Backup, scheduling, and retrieving are all possible with PreRollPost. No formatting, software, nor other hardware is required. A single click is required to install everything for LTFS. Backup can be used with any files ie NLE project files or entire hard drives. PreRollPost also supports backing up camera originals with an index saved on the local hard drive or LTO tape.
- QStar Archive Manager with LTFS Volume Spanning
- QStar announced on 19th July 2012, QStar Unveils Industry’s First Software-Based LTFS Volume-Spanning and File-Spanning Capability for Tape Libraries. With the use of QStar’s LTFS volume-spanning technology, all media within the tape library can be seen as one or more ever-growing network share(s). New LTFS media is automatically added to the set as the previous media becomes full. This approach eliminates the constant stopping and manual reconfiguration required by standard LTFS methodologies as a tape reaches capacity.
- XenData’s entire digital video archive product range supports LTFS, offering a number of advantages compared to other LTFS solutions.
- YoYottaID production and post workflow software for archiving and backup to LTFS tapes created by YoYotta.
- Thought Equity Motion is executing a major film digitization and preservation project for the EYE Film Institute Netherlands. The project involves scanning more than 150 million discrete DPX files and storing them on LTO Gen5 using the LTFS format. More than 1 petabyte of film will be scanned and archived over two years (2010–2012).
- IBM LTFS technology received a Pick Hit Award from Broadcast Engineering at NAB 2011.
- IBM and FOX Networks received an Engineering Emmy Award in 2011 for a project that uses LTFS to store, exchange, and archive video content.
- IBM received the 2011 Hollywood Post-Alliance (HPA) Engineering Excellence Award.
- "SNIA Announces Linear Tape File System Technical Work Group"
- Oracle StorageTek Linear Tape File System, Open Edition Webpage
- Oracle StorageTek T10000C Tape Drive Press Release
- Oracle Introduces StorageTek T10000D Tape Drive
- Oracle StorageTek LTFS, Library Edition Press Release
- "IBM System Storage TS1140 Tape Drive Model E07 delivers higher performance, reliability, and capacity". IBM. Retrieved 4 October 2012.
- Oracle StorageTek T10000C Tape Drive Webpage
- IBM TS2250 Webpage
- IBM TS2350 Webpage
- IBM Ultrium tape drives, libraries, and autoloaders
- Quantum LTO-5 Webpage
- 1 Beyond - Homepage
- Cache-A - Homepage
- IBM LFTS Linux Codebase
- HP LTFS Linux Codebase
- Crossroads - Homepage
- Crossroads - Press Release, Austin, TX, April 11, 2011 : Plans to Launch Archive Product that Ushers in a New Era for Tape
- FOR-A - Homepage
- Tolisgroup - Homepage
- TOLIS Group - Press Release, Las Vegas, NV — April 11, 2011: BRU PRODUCER'S EDITION (BRU PE) SUPPORTS LTFS
- SGL - Homepage
- SGL - Press Release, Fareham, UK - February 9, 2011: SGL announces LTFS implementation transforming digital media workflows
- Masstech Group Inc.-Homepage
- Masstech provides archive support for IBM’s groundbreaking Linear Tape File System Press Release
- Imagine Products Inc.-Homepage
- QStar - Homepage
- QStar - Press Release, Navarre, Florida:
- XenData - Products
- XenData First to Launch LTO Video Archives with Multiple Format File System Support Including LTFS and tar
- YoYotta - Homepage
- Thought Equity Motion - Homepage
- Thought Equity Motion - Press Release: Film Archive Digitization and Preservation Project for EYE Film Institute Netherlands
- NAB 2011 Pick Hit Announcement
- IBM Almaden Blog
- HPA Award Announcement
- LTFS at SNIA
- Free LTFS for Dummies Book
- LTFS Format specification document
- Archive of LTFS Format Specification documents - download site for all versions of LTFS Format Specification.
- LTO Ultrium - LTO Technology Provider Companies website
- IBM Linear Tape File System - IBM's product website
- Oracle Linear Tape File System - Oracle's product website
- HP Linear Tape File System - HP's product website
- Quantum Linear Tape File System - Quantum's product website
- IBM Linear Tape File System - Single Drive Edition software - download site for binary and open source release (Storage Systems/ Tape Systems/ Tape Device Drivers and Software/ Linear Tape File System)
- FOR-A LTR-100HS Video Archive Recorder - digital video recorder device using LTFS formatted tape cartridges
- 1 Beyond Wrangler LTO-5 Offload Stations - Offload and backup system