Jump to content

NILFS

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 81.109.109.139 (talk) at 14:12, 16 November 2016 (PDF URL pointed to a hijacked site (nilfs.org is no-longer the official site)). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

NILFS
Developer(s)Nippon Telegraph and Telephone Cyber Space Laboratories
Full nameNew Implementation of a Log-structured File System
Introducedwith Linux kernel
Structures
File allocationB-tree
Limits
Max file size8 EiB
Other
Supported
operating systems		Linux kernel, (ReadOnly for NetBSD)

NILFS (New Implementation of a Log-structured File System) is a log-structured file system implementation for the Linux kernel. It is being developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and released under the terms of the GNU General Public License (GPL).

Important features

NILFS supports continuous snapshotting. In addition to versioning capability of the entire file system, users can even restore files mistakenly overwritten or destroyed just a few seconds ago. Since NILFS can keep consistency like conventional LFS, it achieves quick recovery after system crashes.

NILFS creates a number of checkpoints every few seconds or per synchronous write basis (unless there is no change). Users can select significant versions among continuously created checkpoints, and can change them into snapshots which will be preserved until they are changed back to checkpoints.

There is no limit on the number of snapshots until the volume gets full. Each snapshot is mountable as a read-only file system. It is mountable concurrently with a writable mount and other snapshots, and this feature is convenient to make consistent backups during use.

Snapshot administration is easy and quickly performable. NILFS will make snapshotting or versioning of the POSIX filesystem much familiar to you. The possible use of NILFS includes, versioning, tamper detection, SOX compliance logging, and so forth. It can serve as an alternative filesystem for Linux desktop environment, or as a basis of advanced storage appliances.

The current major version of NILFS is version 2, which is referred to as NILFS2. NILFS2 realized online garbage collection that reclaims disk space with keeping multiple snapshots.

Other NILFS features include:

  • B-tree based file and inode management.
  • Immediate recovery after system crash.
  • 64-bit data structures; support many files, large files and disks.
  • 64-bit on-disk timestamps which are free of the year 2038 problem.
  • Loadable kernel module; no recompilation of the kernel is required.

Current Status

Supported Features

  • Basic POSIX file system features
  • Snapshots
    • Automatically and continuously taken, you can retrieve past state of NILFS file system.
    • No limit on the number of snapshots until the volume gets full
    • Mountable as read-only file systems
    • Mountable concurrently with the writable mount (convenient to make consistent backups during use)
    • Quick listing
    • Easy administration
  • Background Garbage Collection
    • Can maintain multiple snapshots
    • Selectable GC Policy, which is given by a userland daemon.
  • Quick crash recovery on-mount
  • Read-ahead for meta data files as well as data files
  • Block sizes smaller than page size (e.g. 1KB or 2KB)
  • Redundant super block
  • Online resizing (since Linux-3.x and nilfs-utils 2.1)
  • Related utilities (by contribution of Jiro SEKIBA)
    • grub2
    • util-linux (blkid, libblkid, uuid mount)
    • udisks, palimpsest
    • Filesystem label (nilfs-tune)

Additional features

  • Fast write and recovery times
  • Minimal damage to file data and system consistency on hardware failure
    • 32-bit checksums (CRC32) on data and metadata for integrity assurance (per block group, in segment summary) [1]
    • Correctly ordered data and meta-data writes
    • Redundant superblock
  • Internal data is processed in 64-bit wide word size
  • Can create and store huge files (8 EiB)

OS Compatibility

Distribution Description
Debian NILFS is available in Debian 5.0 and later (lenny, squeeze, wheezy, and sid).
Ubuntu NILFS is available in Ubuntu 9.10 and later.
CentOS x86 and x86_64 for CentOS 5/6 are available on [1].
Fedora Building and installing nilfs-utils-2.1 is required in order to use NILFS in Fedora 14~16. For Fedora 15 and later, user also needs to build NILFS2 kernel module with explained steps.
openSUSE i586 and x86_64 for openSUSE and SUSE Linux Enterprise are available on openSUSE Build Service.
Gentoo NILFS is available in Gentoo Linux. A boot-cd with NILFS is available on PrRescue
Arch Linux NILFS is available in Arch Linux.
Linux Mint NILFS is available in Linux Mint.
NixOS NILFS is available in NixOS.
  • NILFS was merged into the Linux kernel 2.6.30.[2]
  • On distributions where NILFS is available out-of-the-box, the user needs to download the nilfs-utils package, following the instructions from [2].

A separate, BSD licensed implementation, currently with read-only support, is included in NetBSD.[3]

Description

Using a copy-on-write technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, an approach that is designed to reduce seek times, as well as minimize the kind of data loss that occurs after a crash with conventional file systems. For example, data loss occurs on ext3 file systems when the system crashes during a write operation. When the system reboots, the journal notes that the write did not complete, and any partial data writes are lost.

Some file systems, like UFS-derived file systems used by the Solaris operating system and BSDs, provide a snapshot feature that prevents[citation needed] such data loss, but the snapshot configuration can be lengthy on large file systems. NILFS, in contrast, can "continuously and automatically [save] instantaneous states of the file system without interrupting service", according to NTT Labs.[4]

The "instantaneous states" that NILFS continuously saves can actually be mounted, read-only, at the same time that the actual file system is mounted read-write — a capability useful for data recovery after hardware failures and other system crashes. The "lscp" (list checkpoint) command of an interactive NILFS "inspect" utility is first used to find the checkpoint's address, in this case "2048": 

# inspect /dev/sda2
...
nilfs> listcp
   1     6 Tue Jul 12 14:55:57 2005 MajorCP|LogiBegin|LogiEnd
2048  2352 Tue Jul 12 14:55:58 2005 MajorCP|LogiEnd
...
nilfs> quit

The checkpoint address is then used to mount the checkpoint: 

# mount -t nilfs -r -o cp=2048 /dev/sda2 /nilfs-cp
# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             70332412   8044540  62283776  12% /nilfs
/dev/sda2             70332412   8044540  62283776  12% /nilfs-cp

Relative Performance

In the January 2015 presentation SD cards and filesystems for embedded systems at Linux.conf.au, it was stated:[5]

If you've got a workload that's latency sensitive, you might want to use NILFS. If you've got one that's throughput sensitive, you might want to use F2FS.

— Peter Chubb

See also

References

  1. ^ the Nilfs version 1: overview
  2. ^ Linux Kernel 2.6.30 Is Out, Includes the NILFS2 Filesystem
  3. ^ NiLFS(2) source commit
  4. ^ "An article about NILFS". Retrieved 2008-07-28.
  5. ^ Peter Chubb. "SD cards and filesystems for embedded systems". Linux.conf.au.
Retrieved from "https://en.wikipedia.org/w/index.php?title=NILFS&oldid=749851264"