NTFS reparse point

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage hierarchical storage system. They also can act as hard links, but aren't limited to point to files on the same volume: they can point to directories on any local volume. The feature is inherited to ReFS.[1]

The open source NTFS-3G driver implements built-in support for the link-type reparse points, namely soft links and junctions. A plugin filter system is available to handle additional types of reparse points, allowing for chunk-deduplicated files, system-compressed files, and OneDrive files to be read.[2]

Structure[edit]

A reparse point has the following general structure, in C struct form:

struct REPARSE_BUFFER {
  uint32_t  ReparseTag;
  uint32_t  ReparseDataLength;
  uint16_t  Reserved;
  uint8_t   DataBuffer[]; // flexible array member
}

The reparse tag is unique to each type of reparse points. It defines which reparse point handler the filesystem should look for.[3]

Types[edit]

Volume mount points[edit]

Volume mount points are similar to Unix mount points, where the root of another file system is attached to a directory. In NTFS, this allows additional file systems to be mounted without requiring a separate drive letter (such as C: or D:) for each.

Once a volume has been mounted on top of an existing directory of another volume, the contents previously listed in that directory become invisible and are replaced by the content of the root directory of the mounted volume. The mounted volume could still have its own drive letter assigned separately. The file system does not allow volumes to be mutually mounted on each other. Volume mount points can be made to be either persistent (remounted automatically after system reboot) or not persistent (must be manually remounted after reboot).[citation needed]

Mounted volumes may use other file systems than just NTFS, possibly with their own security settings and remapping of access rights according to the remote file system policy.

All mount points use a "dos device" reference of the form \??\DeviceName\.[4][3]

Directory junctions[edit]

Directory junctions are defined using the exact same mechanism (and reparse tag) as mount points are. The only difference is that they point to a sub-directory of another volume that usually already has a drive letter. This function is conceptually similar to symbolic links to directories in Unix, except that the target in NTFS must always be another directory (typical Unix file systems allow the target of a symbolic link to be any type of file).[3]

For instance, the directory C:\exampledir with a directory junction attribute that contains a link to D:\linkeddir will automatically refer to the directory D:\linkeddir when it is accessed by a user-mode application.[5]

Directory junctions (which can be created with the command MKLINK /J junctionName targetDirectory and removed with RMDIR junctionName from a console prompt) are persistent, and resolved on the server side as they share the same security realm of the local system or domain on which the parent volume is mounted and the same security settings for its contents as the content of the target directory; however the junction itself may have distinct security settings. Unlinking a directory junction does not delete files in the target directory.

Some directory junctions are installed by default on Windows Vista, for compatibility with previous versions of Windows, such as Documents and Settings in the root directory of the system drive, which links to the Users physical directory in the root directory of the same volume. However they are hidden by default, and their security settings are set up so that the Windows Explorer will refuse to open them from within the Shell or in most applications, except for the local built-in SYSTEM user or the local Administrators group (both user accounts are used by system software installers). This additional security restriction has probably been made to avoid users of finding apparent duplicate files in the joined directories and deleting them by error, because the semantics of directory junctions is not the same as hardlinks; the reference counting is not used on the target contents and not even on the referenced container itself.[citation needed]

Directory junctions are soft links (they will persist even if the target directory is removed), working as a limited form of symbolic links (with an additional restriction on the location of the target), but it is an optimized version allowing faster processing of the reparse point with which they are implemented, with less overhead than the newer NTFS symbolic links, and can be resolved on the server side (when they are found in remote shared directories).[citation needed]

Symbolic links[edit]

Symbolic links (or soft links) were introduced in Windows Vista.[6] Symbolic links are resolved on the client side. So when a symbolic link is shared, the target is subject to the access restrictions on the client, and not the server.[citation needed]

Symbolic links can be created either to files (created with MKLINK symLink targetFilename) or to directories (created with MKLINK /D symLinkD targetDirectory), but (unlike Unix symbolic links) the semantic of the link must be provided with the created link. The target however need not exist or be available when the symbolic link is created: when the symbolic link will be accessed and the target will be checked for availability, NTFS will also check if it has the correct type (file or directory); it will return a not-found error if the existing target has the wrong type.[citation needed]

They can also reference shared directories on remote hosts or files and subdirectories within shared directories: their target is not mounted immediately at boot, but only temporarily on demand while opening them with the OpenFile() or CreateFile() API. Their definition is persistent on the NTFS volume where they are created (all types of symbolic links can be removed as if they were files, using DEL symLink from a command line prompt or batch).[citation needed]

Distributed Link Tracking (DLT)[edit]

Distributed link tracking allows applications to track files, shell shortcuts or OLE links even if they were renamed or moved to another volume within the same machine, domain or workgroup.[7] Tracking is implemented as a system service, which uses the object identifier (OID) index stored in a metafile.[8] When the application requests a track to a file or directory, the tracking service creates the OID entry, which points to the file, and file rename, copy or move operation to a NTFS v3 volume also copies the object ID. This allows the tracking service to eventually find the target file.

Data deduplication[edit]

When there are several directories that have different but similar files, some of these files may have identical content. Single instance storage, found in Windows Server 2000 through Windows Storage Server 2008, allows identical files to be merged to one file and create references to that merged file. SIS consists of a file system filter that manages copies, modification and merges to files; and a user space service (or groveler) that searches for files that are identical and need merging. SIS was mainly designed for remote installation servers as these may have multiple installation images that contain many identical files; SIS allows these to be consolidated but, unlike for example hard links, each file remains distinct; changes to one copy of a file will leave others unaltered. This is similar to copy-on-write, which is a technique by which memory copying is not really done until one copy is modified.[9]

Since Windows Server 2012, there is a new chunk-based data deduplication mechanism (tag 0x80000013) that allows files with similar content to be deduplicated as long as they have stretches of identical data.[2] This mechanism is more powerful than SIS.[10] Since Windows Server 2019, the feature is fully supported on ReFS.[11]

Hierarchical Storage Management (HSM)[edit]

Hierarchical Storage Management is a means of transferring files that are not used for some period of time to less expensive storage media. When the file is next accessed, the reparse point on that file determines that it is needed and retrieves it from storage.[citation needed]

Native Structured Storage (NSS)[edit]

NSS was an ActiveX document storage technology that has since been discontinued by Microsoft.[citation needed] It allowed ActiveX Documents to be stored in the same multi-stream format that ActiveX uses internally. An NSS file system filter was loaded and used to process the multiple streams transparently to the application, and when the file was transferred to a non-NTFS formatted disk volume it would also transfer the multiple streams into a single stream.[12]

Unix domain socket (socket)[edit]

With windows 10 build 17063 (For stable release version 1803), Microsoft introduced Unix domain sockets to windows, these are realized by using the afunix.sys kernel driver and a new reparse point in the filesystem. Unix domain sockets are common on BSD and Linux systems since ages and can be seen as the standard for inter process communication on these systems. Therefore their introduction to windows will allow simplified adoption of code and cross platform portability.[13]

System compression[edit]

Windows 10 introduces a few read-only compression algorithms for NTFS, taken from Windows Imaging Format. They are XPRESS4K/8K/16K and LZX. Both are based on LZ77 with Huffman entropy coding, which LZNT1 lacked. They are mainly used for new CompactOS feature, which compresses the entire system partition using one of these algorithms.[14] They can also be manually turned on per file with the /exe flag of the compact command.

Internally, the file is recorded as a reparse point with tag 0x80000017 to record the fact that the file has been specially compressed, and the actual data is stored in an alternate data stream named "WofCompressedData" (for Windows Overlay Filesystem). The new design is meant purely for read-only access, so any writes to compressed files result in fully decompressing the file on Windows.[15][16][2]

OneDrive[edit]

OneDrive tags the files and directories it has downloaded to the local storage as a reparse point with tag 0x9000001a. The actual data is stored normally.[2]

Known risks[edit]

The Stuxnet as part of its series of Win32 exploits uses NTFS junction points as part of its overall mode of operation.[citation needed]

See also[edit]

References[edit]

  1. ^ "Microsoft Windows Vista Client Configuration Study Guide" Wiley Publishing, Inc. 2007 p.285
  2. ^ a b c d André, Jean-Pierre (March 1, 2019). "NTFS-3G: Junction Points, Symbolic Links and Reparse Points". jp-andre.pagesperso-orange.fr.
  3. ^ a b c "NTFS Links, Directory Junctions, and Windows Shortcuts". www.flexhex.com.
  4. ^ "winapi - Does the "SubstituteName" string in the PathBuffer of a REPARSE_DATA_BUFFER structure always start with the prefix "\??\", and if so, why?". Stack Overflow. Retrieved 4 October 2019.
  5. ^ Mark Russinovich. "Inside Win2K NTFS, Part 1". Microsoft Developer Network. Retrieved 2008-04-18.
  6. ^ "Symbolic Links (Windows)". MSDN.
  7. ^ http://msdn.microsoft.com/en-us/library/windows/desktop/aa363997.aspx
  8. ^ https://technet.microsoft.com/en-us/library/cc736811(WS.10).aspx
  9. ^ "Single Instance Storage in Windows 2000" (PDF). Microsoft Research and Balder Technology Group.
  10. ^ FileCAB-Team (10 April 2019). "Introduction to Data Deduplication in Windows Server 2012". Microsoft Tech Community.
  11. ^ "Data Deduplication interoperability". docs.microsoft.com.
  12. ^ Saville, John (date unknown). What is Native Structured Storage? Windows IT Pro. Retrieved from "Archived copy". Archived from the original on 2007-09-27. Retrieved 2007-12-03.CS1 maint: archived copy as title (link).
  13. ^ "AF_UNIX comes to Windows". Windows Command Line Tools For Developers. Retrieved 2018-07-03.
  14. ^ "Compact OS, single-instancing, and image optimization". Microsoft. Retrieved 1 October 2019.
  15. ^ "Re: [ntfs-3g-devel] Experimental support for Windows 10 "System Compressed" files". sourceforge.net. Retrieved 1 October 2019.
  16. ^ Biggers, Eric (29 April 2019). "ntfs-3g-system-compression: NTFS-3G plugin for reading "system compressed" files". GitHub. Retrieved 1 October 2019.

External links[edit]