Linux kernel interfaces
This article is written like a personal reflection or opinion essay that states a Wikipedia editor's personal feelings about a topic. (July 2014) (Learn how and when to remove this template message)
The Linux kernel provides several interfaces to user-space applications that are used for different purposes and that have different properties by design. There are two types of application programming interface (API) in the Linux kernel that are not to be confused: the "kernel–user space" API and the "kernel internal" API.
The Linux API is the kernel–user space API, which allows programs in user space to access system resources and services of the Linux kernel. It is composed out of the System Call Interface of the Linux kernel and the subroutines in the GNU C Library (glibc). The focus of the development of the Linux API has been to provide the usable features of the specifications defined in POSIX in a way which is reasonably compatible, robust and performant, and to provide additional useful features not defined in POSIX, just as the kernel–user space APIs of other systems implementing the POSIX API also provide additional features not defined in POSIX.
The Linux API, by choice, has been kept stable over the decades and never breaks; this stability guarantees the portability of source code. At the same time, Linux kernel developers have historically been conservative and meticulous about introducing new system calls.
Much available free and open-source software is written for the POSIX API. Since so much more development flows into the Linux kernel as compared to the other POSIX-compliant combinations of kernel and C standard library, the Linux kernel and its API have been augmented with additional features. As far as these additional features provide a technical advantage, programming for the Linux API is preferred over the POSIX-API. Well-known current examples are udev, systemd and Weston. People such as Lennart Poettering openly advocate to prefer the Linux API over the POSIX API, where this offers advantages.
At FOSDEM 2016, Michael Kerrisk explained some of the perceived issues with the Linux kernel's user-space API, describing that it contains multiple design errors by being non-extensible, unmaintainable, overly complex, of limited purpose, in violation of standards, and inconsistent. Most of those mistakes cannot be fixed because doing so would break the ABI that the kernel presents to the user space.
System Call Interface of the Linux kernel
System Call Interface is the denomination for the entirety of all implemented and available system calls in a kernel. Various subsystems, such as e.g. the DRM define their own system calls and the entirety is called System Call Interface.
The C standard library
The GNU C Library is a wrapper around the system calls of the Linux kernel; the combination of the Linux kernel System Call Interface and glibc is what builds the Linux API.
Additions to POSIX
As in other Unix-like systems, additional capabilities of the Linux kernel exist that are not part of POSIX:
- cgroups subsystem, the system calls it introduces and libcgroup
- The system calls of the Direct Rendering Manager, especially the driver-private ioctls for the command submission are not part of the POSIX specifications.
- Advanced Linux Sound Architecture could set system calls, which are not part of the POSIX specifications
- The system calls
futex(fast userspace mutex),
inotifyhave been exclusive to the Linux kernel so far.
- The system call
getrandomwas introduced in version 3.17 of the Linux kernel mainline
memfdwas proposed by the kdbus developers
memfd_createwas merged into the Linux kernel mainline in kernel version 3.17
readaheadinitiates a file "read-ahead" into page cache
DRM has been paramount for the development and implementations of well-defined and performant free and open-source graphics device drivers without which no rendering acceleration would be available at all, or even worse, only the 2D drivers would be available in the X.Org Server. DRM was developed for Linux, and since has been ported to other operating systems as well.
- libdrm (for Direct Rendering Manager)
- libnl (The libnl suite is a collection of libraries providing APIs to netlink protocol based Linux kernel interfaces.)
- libevdev (for evdev)
- libasound (Advanced Linux Sound Architecture)
This article needs attention from an expert in Free Software, Software or Computing. The specific problem is: This section mostly ignores the kernel-userspace ABI (that is very real and important thing) and jumps into userspace-to-userspace APIs.(February 2018)
The term Linux ABI refers to a kernel–user space ABI. The Application binary interface refers to the compiled binaries, in machine code. Any such ABI is therefore bound to the instruction set. Defining a useful ABI and keeping it stable is less the responsibility of the Linux kernel developers or of the developers of the GNU C Library, and more the task for Linux distributions and Independent software vendor (ISVs) who wish to sell and provide support for their proprietary software as binaries only for such a single Linux ABI, as opposed to supporting multiple Linux ABIs.
In the Wikipedia a category is maintained for articles on Category:Proprietary software for Linux.
It should be able to compile the software with different compilers against the definitions specified in the ABI and achieve full binary compatibility. Compilers that are free and open-source software are e.g. GNU Compiler Collection, LLVM/Clang.
End-users are in fact not all interested in the Linux API (or the Windows API), but in the ABIs.
There are a lot of kernel-internal APIs for all the subsystems to interface with one another. These are being kept fairly stable, but there is no guarantee for stability. In case new research or insights make a change seem favorable, an API is changed, all necessary rewrite and testing have to be done by the author.
The Linux kernel is a monolithic kernel, hence device drivers are kernel components. To ease the burden of companies maintaining their (proprietary) device drivers out-of-tree, stable APIs for the device drivers have been repeatedly requested. The Linux kernel developers have repeatedly denied guaranteeing stable in-kernel APIs for device drivers. Guaranteeing such would have faltered the development of the Linux kernel in the past and would still in the future and, due to the nature of free and open-source software, are not necessary. Ergo, by choice, the Linux kernel has no stable in-kernel API.
Since there are no stable in-kernel APIs, there cannot be stable in-kernel ABIs.
For several use cases the Linux API is considered too low-level and higher abstraction APIs are used. Such of course still need to work on top of the low-level Linux APIs. Examples:
- implementation of the OpenGL and Vulkan specifications in proprietary Linux graphics drivers and the free and open-source implementation in Mesa
- implementation of the OpenAL specification
- Simple DirectMedia Layer: abstraction API for input/sound/etc. available for many operating systems
- Simple and Fast Multimedia Library: like above
- The Linux Programming Interface by Michael Kerrisk
- Semaphore (programming)
- system call – is a function to facilitate programs to request services from the kernel
- netlink – socket family used for IPC between kernel and user space processes, designed as the successor of ioctl; Netlink was added by Alan Cox during Linux kernel 1.3 development as a character driver interface to provide multiple kernel and user-space bidirectional communications links. Then, Alexey Kuznetsov extended it during Linux kernel 2.1 development to provide a flexible and extensible messaging interface to the new advanced routing infrastructure. Since then, Netlink sockets have become one of the main interfaces that kernel subsystems provide to user-space applications in Linux. Modern WNIC drivers use it to communicate with user-space.
- Windows API – article on various API available on Microsoft Windows operating systems
- Wine – a compatibility layer between Linux and programs written for Microsoft Windows
- libhybris – compatibility layer between Linux and programs written for Android
- "ControlGroupInterface". freedesktop.org.
- "libevdev". freedesktop.org.
- Alessandro Rubini (2006-11-02). "Kernel System Calls". linux.it. Retrieved 2014-11-11.
- Linus Torvalds (2012-12-23). "Re: [Regression w/ patch] Media commit causes user space to misbahave (was: Re: Linux 3.8-rc1)". Linux kernel mailing list. Retrieved 2014-08-26.
If a change results in user programs breaking, it's a bug in the kernel. We never EVER blame the user programs.
- "Choosing between portability and innovation". LWN.net. 2011-03-02.
- "Interview: Lennart Poettering - Lennart Poettering will give a talk about "Systemd: beyond init" at FOSDEM 2011". fosdem.org. 2011. Retrieved 2014-06-16.
In fact, the way I see things the Linux API has been taking the role of the POSIX API and Linux is the focal point of all Free Software development. Due to that I can only recommend developers to try to hack with only Linux in mind and experience the freedom and the opportunities this offers you. So, get yourself a copy of The Linux Programming Interface, ignore everything it says about POSIX compatibility and hack away your amazing Linux software. It's quite relieving!
- Michael Kerrisk (2016-01-31). "How to design a Linux kernel API". Retrieved 2016-02-04.
- "System Call Organization".
- "Making a universal list of syscalls?". LKML. 2014-02-27.
- "Flags as a system call API design pattern". LWN.net. 2014-02-12.
- "On vsyscalls and the vDSO". LWN.net. 2011-06-08.
- "[PATCH, RFC] random: introduce getrandom(2) system call". LKML. 2014-07-17.
- "memfd.c". Archived from the original on 2014-04-22.
- "NetBSD 7.0 Will Finally Have DRM/KMS Drivers". Phoronix. 2014-03-19.
- "The Linux Kernel Driver Interface".
- "Analysis of ABI changes in the Linux kernel". Andrey Ponomarenko's ABI laboratory. 2016-03-15.
- The API of Linux kernel 2.6.20 – sadly no current version available
- The Linux Programming Interface
- Interactive Linux kernel map with main API functions and structures
- Linux Device Drivers by Jonathan Corbet, Greg Kroah-Hartman and Alessandro Rubini, 3rd edition
- Linux Kernel Linked List Explained
- TLPI: API changes, Linux API changes since The Linux Programming Interface was released in 2010