Filename
A filename (also written as two words, file name) is a name used to uniquely identify a computer file stored in a file system. Different file systems impose different restrictions on filename lengths and the allowed characters within filenames.
A filename may include one or more of these components:
- host (or node or server) – network device that contains the file
- device (or drive) – hardware device or drive
- directory (or path) – directory tree (e.g., /usr/bin, \TEMP, [USR.LIB.SRC], etc.)
- file – base name of the file
- type (format or extension) – indicates the content type of the file (e.g., .txt, .exe, .COM, etc.)
- version – revision or generation number of the file
The components required to identify a file varies across operating systems, as does the syntax and format for a valid filename.
History [edit]
| This section requires expansion. (July 2012) |
Around 1962, the Compatible Time-Sharing System introduced the concept of a file (i.e., non-paper file).
Around this same time appeared the dot (period or full-stop) as a filename extension separator, and the limit to three letter extensions might have come from RAD50 16-bit limits.[1]
Traditionally, filenames allowed only alphanumeric characters, but as time progressed, the number of characters allowed increased. This led to incompatibility problems when moving files from one file system to another.[2]
Around 1995, VFAT, a variant of the FAT filesystem with an extended directory format, was introduced in Windows 95 and Windows NT 3.5. It allowed mixed-case Unicode long filenames (LFNs), in addition to classic "8.3" names.
In 1985, RFC 959 officially defined a pathname to be the character string which must be entered into a file system by a user in order to identify a file.[3]
OS X 10.3 marked Apple's adoption of Unicode 3.2 character decomposition, superseding the Unicode 2.1 decomposition used previously. This change caused problems for developers writing software for OS X.[4]
Unicode migration [edit]
| This section requires expansion. (July 2012) |
One issue was migration to Unicode. For this purpose, several software companies provided software for migrating filenames to the new Unicode encoding.
- Microsoft provided migration transparent for the user throughout the vfat technology
- Apple provided "File Name Encoding Repair Utility v1.0".[5]
- The Linux community provided “convmv”.[6]
References: absolute vs relative [edit]
An absolute reference includes all directory levels. In some systems, a filename reference that does not include the complete directory path it defaults to the current working directory. This is a relative reference. One advantage of using a relative reference in program configuration files or scripts is that different instances of the script or program can use different files.
This makes an absolute or relative path composed of a sequence of filenames.
Number of names per file [edit]
Unix-like file systems allow a file to have more than one name; in traditional Unix-style file systems, the names are hard links to the file's inode or equivalent. Windows supports hard links on NTFS file systems, and provides the command fsutil in Windows XP, and mklink in later versions, for creating them.[7][8] Hard links are different from Windows shortcuts, Mac OS aliases, or symbolic links. The introduction of LFNs with VFAT allowed filename aliases "for example, longfile~1.??? was a filename alias of "long file name.???" as a way to conform to 8.3 limitations for older programs.
This property was used by the move command algorithm which first creates a second filename and then only removes the first filename.
Other filesystems, by design, provide only one filename per file, which guarantees that alteration of one filename's file do not alter the other filename's file.
Length restrictions [edit]
Some filesystems restrict the length of filenames to 11 (e.g., MS-DOS), 14 (e.g., early Unix), 31, 44 (e.g., IBM S/370),[9] or 255 (e.g., early Berkeley Unix) characters or bytes.
Filename extensions [edit]
Many file systems, including FAT, NTFS, and VMS systems, allow a filename extension that consists of one or more characters following the last period in the filename, dividing the filename into two parts: a base name or stem and an extension or suffix used by some applications to indicate the file type. Multiple output files created by an application use the same basename and various extensions. For example a compiler might use the extension FOR for the source input file, OBJ for the object output and LST for the listing. Although there are some common extensions they are arbitrary and a different application might use REL and RPT. On filesystems that do not segregate the extension, files will often have a longer extension such as html.
Encoding interoperability [edit]
There is no general encoding standard for filenames.
Because file names have to been exchanged between software environments (think network file transfer, file system storage, backup and file synchronization software, configuration management, data compression and archiving, etc.) it is very important not to lose file name information between applications. This led to wide adoption of Unicode as a standard for encoding file names, although legacy software might be non-Unicode-aware.
Encoding indication interoperability [edit]
| This section may need to be rewritten entirely to comply with Wikipedia's quality standards. (September 2012) |
Traditionally, filenames allowed any character in their filenames as long as they were file system safe.[2] Although this permitted the use of any encoding, and thus allowed the representation of any local text on any local system, it caused many interoperability issues.
A filename could be stored using different byte strings in distinct systems within a single country: for example if one used Japanese Shift JIS encoding and another Japanese EUC encoding. Conversion was not possible as most systems did not expose a description of the encoding used for a filename as part of the extended file information. This forced costly filename encoding guessing with each file access.[2]
A solution was to adopt Unicode as the encoding for filenames.
In the Mac OS however, encoding of the filename was stored with the filename attributes.[2]
Unicode interoperability [edit]
The Unicode standard solves the encoding determination issue.
Nonetheless, some limited interoperability issues remain, such as normalization (equivalence), or the Unicode version in use. For instance, UDF is limited to Unicode 2.0; Mac OS applies NFD Unicode normalization and is not case-sensitive. Filename maximum length is not standard and might depend on the code unit size. Although it is a serious issue, in most cases this is a limited one.[2]
On Linux, this means the filename is not enough to open a file: additionally, the exact byte representation of the filename on the storage device is needed. This can be solved at the application level, with some tricky normalization calls.[10]
The issue of Unicode equivalence is known as "normalized-name collision". A solution is the Non-normalizing Unicode Composition Awareness used in the Subversion and Apache technical communities.[11] This solution does not normalize paths in the repository. Paths are only normalized for the purpose of comparisons. Nonetheless, some communities have patented this strategy, forbidding its use by other communities.[clarification needed]
Perspectives [edit]
To limit interoperability issues, some ideas described by Sun are to:
- use one Unicode encoding (such as UTF-8)
- do transparent code conversions on filenames
- store no normalized filenames
- check for canonical equivalence among filenames, to avoid two canonically equivalent filenames in the same directory.[2]
Those considerations create a limitation not allowing a switch to a future encoding different from UTF-8.
Uniqueness [edit]
Within a single directory, filenames must be unique. Since the filename syntax also applies for directories, it is not possible to create a file and directory entries with the same name in a single directory. Multiple files in different directories may have the same name.
Uniqueness approach may differ both on the case sensitivity and on the Unicode normalization form such as NFC, NFD. This means two separate files might be created with the same text filename and a different byte implementation of the filename, such as L"\x00C0.txt" (UTF-16, NFC) (Latin capital A with grave) and L"\x0041\x0300.txt" (UTF-16, NFD) (Latin capital A, grave combining).[12]
Letter case preservation [edit]
Some filesystems, such as FAT, store filenames as upper-case regardless of the letter case used to create them. For example, a file created with the name "MyName.Txt" or "myname.txt" would be stored with the filename "MYNAME.TXT". Any variation of upper and lower case can be used to refer to the same file. These kinds of file systems are called case-insensitive and are not case-preserving. Some filesystems prohibit the use of lower case letters in filenames altogether.
Some file systems store filenames in the form that they were originally created; these are referred to as case-retentive or case-preserving. Such a file system can be case-sensitive or case-insensitive. If case-sensitive, then "MyName.Txt" and "myname.txt" may refer to two different files in the same directory, and each file must be referenced by the exact capitalisation by which it is named. On a case-insensitive, case-preserving file system, on the other hand, only one of "MyName.Txt", "myname.txt" and "Myname.TXT" can be the name of a file in a given directory at a given time, and a file with one of these names can be referenced by any capitalisation of the name.
From its original inception, Unix and its derivative systems were case-preserving. However, not all Unix-like file systems are case-sensitive; by default, HFS+ in Mac OS X is case-insensitive, and SMB servers usually provide case-insensitive behavior (even when the underlying file system is case-sensitive, e.g. Samba on most Unix-like systems), and SMB client file systems provide case-insensitive behavior. File system case sensitivity is a considerable challenge for software such as Samba and Wine, which must interoperate efficiently with both systems that treat uppercase and lowercase files as different and with systems that treat them the same.[13]
Reserved characters and words [edit]
All file systems did not always provide the same character set for composing a filename. Before Unicode became a defacto standard, filesystem were mostly working on the basis of a character set depending on the locale, while some new systems permit a filename to be composed of about every character of the Unicode repertoire, with some specific limitations.
Depending on the system, limitation can be provided by the filesystem, by the operating system, by the application, or by the user need of interoperability with other systems.
The basic character set might be limited to a subset of unicode characters, such as a 256 values 8 bit encoding, be based on Unicode character set, or include special bytes sequences which do not form any valid unicode character.
Many file system utilities prohibit control characters from appearing in filenames. In Unix-like file systems the null character, as that is the end-of-string indicator[14] and the path separator / are prohibited.
When they allow any Unicode character, some file system utilities and naming conventions prohibit some particular characters from appearing in filenames:[15]
| Character | Name | Reason |
|---|---|---|
/ |
slash | used as a path name component separator in Unix-like, Windows, and Amiga systems. (The MS-DOS command.com shell would consume it as a switch character, but Windows itself always accepts it as a separator.[16][vague]) |
\ |
backslash | Also used as a path name component separator in MS-DOS, OS/2 and Windows (where there are few differences between slash and backslash); allowed in Unix filenames, see Note 1 |
? |
question mark | used as a wildcard in Unix, Windows and AmigaOS; marks a single character. Allowed in Unix filenames, see Note 1 |
% |
percent | used as a wildcard in RT-11; marks a single character. |
* |
asterisk or star |
used as a wildcard in Unix, MS-DOS, RT-11, VMS and Windows. Marks any sequence of characters (Unix, Windows, later versions of MS-DOS) or any sequence of characters in either the basename or extension (thus "*.*" in early versions of MS-DOS means "all files". Allowed in Unix filenames, see note 1 |
: |
colon | used to determine the mount point / drive on Windows; used to determine the virtual device or physical device such as a drive on AmigaOS, RT-11 and VMS; used as a pathname separator in classic Mac OS. Doubled after a name on VMS, indicates the DECnet nodename (equivalent to a NetBIOS (Windows networking) hostname preceded by "\\".). Colon is also used in Windows to separate an alternative data stream from the main file. |
| |
vertical bar or pipe |
designates software pipelining in Unix and Windows; allowed in Unix filenames, see Note 1 |
" |
quote | used to mark beginning and end of filenames containing spaces in Windows, see Note 1 |
< |
less than | used to redirect input, allowed in Unix filenames, see Note 1 |
> |
greater than | used to redirect output, allowed in Unix filenames, see Note 1 |
. |
period or dot |
allowed but the last occurrence will be interpreted to be the extension separator in VMS, MS-DOS and Windows. In other OSes, usually considered as part of the filename, and more than one period (full stop) may be allowed. In Unix, a leading period means the file or folder is normally hidden. |
| space |
allowed (apart MS-DOS) but the space is also used as a parameter separator in command line applications. This can be solved by quoting, but typing quotes around the name every time is inconvenient. |
Note 1: While they are allowed in Unix file and folder names, most Unix shells require certain characters such as spaces, <, >, |, \, and sometimes :, (, ), &, ;, #, as well as wildcards such as ? and *, to be quoted or escaped:
five\ and\ six\<seven(example of escaping)
'five and six<seven'or"five and six<seven"(examples of quoting)
In Windows utilities the space and the period are not allowed as the final character of a filename.[17] The period is allowed as the first character, but certain Windows applications, such as Windows Explorer, forbid creating or renaming such files (despite this convention being used in Unix-like systems to describe hidden files and directories). Among workarounds are using different explorer applications or saving a file with the desired filename from within an application .[18]
Some file systems on a given operating system (especially file systems originally implemented on other operating systems), and particular applications on that operating system, may apply further restrictions and interpretations. See comparison of file systems for more details on restrictions.
In Unix-like systems, MS-DOS, and Windows, the filenames "." and ".." have special meanings (current and parent directory respectively).
In addition, in Windows and DOS utilities, some words might also be reserved and can not be used as filenames.[18] For example, DOS device files:[19]
CON, PRN, AUX, CLOCK$, NUL COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9 LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, and LPT9.
Systems that have these restrictions cause incompatibilities with some other filesystems. For example, Windows will fail to handle, or raise error reports for, these legal UNIX filenames: aux.c, q"uote"s.txt, or NUL.txt.
NTFS filenames that are used internally include:
$Mft, $MftMirr, $LogFile, $Volume, $AttrDef, $Bitmap, $Boot, $BadClus, $Secure, $Upcase, $Extend, $Quota, $ObjId and $Reparse
Usage [edit]
While system offer a wide choice of characters and bytes to compose filenames, users tend to self-censor.
Nobody should want to use a bytes sequence which does not match any kind of character as this would result in a non displayable filename.
In the same way, control characters are useless in a filename.
Additionally, to ensure a filename will be usable in most systems, and workaround system deficiencies, most users self-censor to use only the ASCII subset of characters, for file exchange.
Then the filename is generally composed of a sequence of few words.
Comparison of filename limitations [edit]
| System | Case sensitive |
Case preserving |
Allowed character set | Reserved characters | Reserved words | Maximum length | Comments |
|---|---|---|---|---|---|---|---|
| MS-DOS FAT | No | No | any 8-bit set | x00-x1F SPACE DEL " * / : < > ? \ | | Device names including: AUX COM1 … COM9 CON LPT1 … LPT9 NUL PRN | 11 | Maximum 8 character base name limit and 3 character extension; see 8.3 filename |
| Windows VFAT | No | Yes | Unicode, using UTF-16 encoding | |\?*<":>+[]/ control characters | 255 | ||
| NTFS | Optional | Yes | Unicode, using UTF-16 encoding | / null (i.e., 0x00) | Only in root directory: $AttrDef $BadClus $Bitmap $Boot $LogFile $MFT $MFTMirr pagefile.sys $Secure $UpCase $Volume $Extend $Extend\$ObjId $Extend\$Quota $Extend\$Reparse ($Extend is a directory) | 255 | forbids the use of characters in range 1-31 (0x01-0x1F) and characters " * : < > ? \ / | unless the name is flagged as being in the Posix namespace. NTFS allows each path component (directory or filename) to be 255 characters long[dubious ].
Windows forbids the use of the MS-DOS device names AUX, CLOCK$, COM1, …, COM9, CON, LPT1, …, LPT9, NUL and PRN, as well as these names with any extension (for example, AUX.txt), except when using Long UNC paths (ex. \\.\C:\nul.txt or \\?\D:\aux\con). (CLOCK$ may be used if an extension is provided.) The Win32 API strips trailing space and period (full-stop) characters from filenames, except when UNC paths are used. These restrictions only apply to Windows; in Linux distributions that support NTFS, filenames are written using NTFS's Posix namespace, which allows any Unicode character except / and NUL. |
| OS/2 HPFS | No | Yes | any 8-bit set | |\?*<":>/ | 254 | ||
| Mac OS HFS | No | Yes | any 8-bit set | : | 255 | old versions of Finder are limited to 31 characters | |
| Mac OS HFS+ | Optional | Yes | Unicode, using UTF-16 encoding | : on disk, in classic Mac OS, and at the Carbon layer in Mac OS X; / at the Unix layer in Mac OS X | 255 | Mac OS 8.1 - Mac OS X | |
| most UNIX file systems | Yes | Yes | any 8-bit set | / null | 255 | a leading . indicates that ls and file managers will not show the file by default |
|
| zOS | ? | ? | dab per Talk:AT&T#Incoming links to AT&T | other than $ # @ - x'C0' | 44 | first character must be alphabetic or national ($, #, @)
"Qualified" contains |
|
| early UNIX (AT&T Corporation) | Yes | Yes | any 8-bit set | / | 14 | a leading . indicates a "hidden" file | |
| POSIX "Fully portable filenames"[21] | Yes | Yes | A–Z a–z 0–9 . _ - | / null | Filenames to avoid include: a.out, core, .profile, .history, .cshrc | 14 | hyphen must not be first character |
| ISO 9660 | No | ? | A–Z 0–9 _ . | "close to 180"(Level 2) or 200(Level 3) | Used on CDs; 8 directory levels max (for Level 1, not level 2,3) | ||
| AmigaOS | No | Yes | any 8-bit set | : / " | 107 | dos.library | |
| Amiga OFS | No | Yes | any 8-bit set | : / " | 30 | Original File System 1985 | |
| Amiga FFS | No | Yes | any 8-bit set | : / " | 30 | Fast File System 1988 | |
| Amiga PFS | No | Yes | any 8-bit set | : / " | 255 | Professional File System 1993 | |
| Amiga SFS | No | Yes | any 8-bit set | : / " | 32,000 | Smart File System 1998 | |
| Amiga FFS2 | No | Yes | any 8-bit set | : / " | 107 | Fast File System 2 2002 | |
| BeOS BFS | Yes | Yes | Unicode, using UTF-8 encoding | / | 255 | ||
| DEC PDP-11 RT-11 | No | No | RADIX-50 | 6 + 3 | Flat filesystem with no subdirs. A full "file specification" includes device, filename and extension (file type) in the format: dev:filnam.ext. | ||
| DEC VAX VMS | No | From v7.2 |
A–Z 0–9 $ - _ | 32 per component; earlier 9 per component; latterly, 255 for a filename and 32 for an extension. | a full "file specification" includes nodename, diskname, directory/ies, filename, extension and version in the format: OURNODE::MYDISK:[THISDIR.THATDIR]FILENAME.EXTENSION;2 Directories can only go 8 levels deep. | ||
| Commodore DOS | Yes | Yes | any 8-bit set | :, = | $ | 16 | length depends on the drive, usually 16 |
See also [edit]
- File system
- Fully qualified file name
- Long filename
- Path (computing)
- Slug (web publishing)
- Symbolic link
- Uniform Resource Identifier (URI)
- Uniform Resource Locator (URL)
References [edit]
- ^ http://randalljhoward.com/2008/12/31/who-invented-filenames-dot/
- ^ a b c d e f File Systems, Unicode, and Normalization San Francisco CA MArch 2006 David Robinso,, Ienup Sung, Nicolas Williams http://developers.sun.com/global/products_platforms/solaris/reference/presentations/IUC29-FileSystems.pdf
- ^ RFC 959 IETF.org RFC 959, File Transfer Protocol (FTP)
- ^ "Re: git on MacOSX and files with decomposed utf-8 file names". KernelTrap. 2010-05-07. Retrieved 2010-07-05.
- ^ http://support.apple.com/downloads/File_Name_Encoding_Repair_Utility
- ^ http://www.j3e.de/linux/convmv/man/
- ^ "Fsutil command description page".
- ^ "NTFS Hard Links, Directory Junctions, and Windows Shortcuts". Flex hex. Inv Softworks. Retrieved 12 March 2011.
- ^ "DSN".
- ^ http://nedbatchelder.com/blog/201106/filenames_with_accents.html
- ^ http://wiki.apache.org/subversion/NonNormalizingUnicodeCompositionAwareness
- ^ http://www.gamedev.net/topic/628734-cross-platform-filepath-naming-conventions/
- ^ "CaseInsensitiveFilenames - The Official Wine Wiki". Wiki.winehq.org. 2009-11-08. Retrieved 2010-08-20.
- ^ "The Open Group Base Specifications Issue 6".
- ^ http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
- ^ "When did Windows start accepting forward slash as a path separator? - Python answers". Thescripts.com. July 18, 2005. Retrieved 2010-08-20.
- ^ Windows Naming Conventions (see last bulleted item)
- ^ a b Naming a file msdn.microsoft.com (MSDN), filename restrictions on Windows
- ^ MS-DOS Device Driver Names Cannot be Used as File Names., Microsoft
- ^ http://publib.boulder.ibm.com/infocenter/zos/v1r11/index.jsp?topic=/com.ibm.zos.r11.ieab600/iea2b690211.htm
- ^ Lewine, Donald. POSIX Programmer's Guide: Writing Portable UNIX Programs 1991 O'Reilly & Associates, Inc. Sebastopol, CA pp63-64
External links [edit]
- Data Formats/ Filename at the Open Directory Project
- File Extension Library
- FILExt
- MSDN view
- 2009 POSIX portable filename character set
- Standard ECMA-208, December 1994, System-Independent Data Format [1]