Self-hosting (compilers)

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

In computer programming, self-hosting is the use of a program as part of the toolchain or operating system that produces new versions of that same program—for example, a compiler that can compile its own source code. Self-hosting software is commonplace on personal computers and larger systems. Other programs that are typically self-hosting include kernels, assemblers, command-line interpreters and revision control software.

Operating Systems[edit]

An operating system is self-hosted when the toolchain to build the operating system runs on that OS. For example, Windows can be built on a computer running Windows.[citation needed]

However, operating systems don't start out as self-hosted. When developing for a new computer or operating system, you need a system to run the development software, but you need development software to write the operating system. This is called a bootstrapping problem or, more generically, a Chicken or the egg dilemma.

The solution to this problem is the cross Compiler (or cross assembler when working with assembly language). A cross compiler allows you to use one computer to develop software for a different machine or operating system, making it possible to create an operating system for a machine for which one does not yet exist. Once written, software can be deployed to the target system using the usual means: an EPROM, floppy diskette, flash memory (such as a USB thumb drive), or JTAG device. This is similar to the method used to write software for gaming consoles or for handheld devices, like cellular phones or tablets, which will never host their own development tools.

Once the software is mature enough to compile its own code, the cross-development dependency ends. At this point, an operating system is said to be self-hosted.

Compilers[edit]

Software development using compiler or interpreters can also be self hosted when the compiler is capable of compiling itself.[1]

Since self-hosted compilers suffer from the same bootstrap problems as operating systems, a compiler for a new programming language needs to be written in an existing language. So the developer may use something like assembly language, C/C++, or even a scripting language like Python or Lua to build the first version of the compiler. Once the language is mature enough, development of the compiler can shift to the compiler's native language, allowing the compiler to build itself.

History[edit]

The first self-hosting compiler (excluding assemblers) was written for Lisp by Hart and Levin at MIT in 1962. They wrote a Lisp compiler in Lisp, testing it inside an existing Lisp Interpreter. Once they had improved the compiler to the point where it could compile its own source code, it was self-hosting.[2]

The compiler as it exists on the standard compiler tape is a machine language program that was obtained by having the S-expression definition of the compiler work on itself through the interpreter.

— AI Memo 39[2]

This technique is usually only practicable when an interpreter already exists for the very same language that is to be compiled; though possible, it is extremely uncommon to humanly compile a compiler with itself.[3] The concept borrows directly from and is an example of the broader notion of running a program on itself as input, used also in various proofs in theoretical computer science, such as the proof that the halting problem is undecidable.

Examples[edit]

Ken Thompson started development on Unix in 1968 by writing and compiling programs on the GE-635 and carrying them over to the PDP-7 for testing. After the initial Unix kernel, a command interpreter, an editor, an assembler, and a few utilities were completed, the Unix operating system was self-hosting - programs could be written and tested on the PDP-7 itself.[4]

Douglas McIlroy wrote TMG (a compiler-compiler) in TMG on a piece of paper and "decided to give his piece of paper to his piece of paper," doing the computation himself, thus compiling a TMG compiler into assembly, which he typed up and assembled on Ken Thompson's PDP-7.[3]

Development of the GNU system relies largely on GCC (the GNU C Compiler) and GNU Emacs (a popular editor), making possible the self contained, maintained and sustained development of free software for the GNU Project.

Many programming languages have self-hosted implementations: compilers that are both in and for the same language. In some of these cases, the initial implementation was developed using bootstrapping, i.e. using another high-level language, assembler, or even machine language.

List of languages having self-hosting compilers[edit]

The following programming languages have self-hosting compilers:[citation needed]

See also[edit]

References[edit]

  1. ^ Heaton, Robert. "What is a self-hosting compiler?". robertheaton.com/.
  2. ^ a b Tim Hart and Mike Levin. "AI Memo 39-The new compiler" (PDF). Archived from the original (PDF) on 2020-12-13. Retrieved 2008-05-23.
  3. ^ a b Ken Thompson. "VCF East 2019 -- Brian Kernighan interviews Ken Thompson". YouTube. Retrieved 2019-10-28.
  4. ^ Dennis M. Ritchie. "The Development of the C Language". 1993.
  5. ^ BASICO compiler bootstrapping example
  6. ^ ClojureScript Next
  7. ^ "flat assembler". Retrieved 7 January 2022. The flat assembler is self-hosting and the complete source code is included.
  8. ^ "Haskell Communities and Activities Report".
  9. ^ https://www.pyret.org Archived 2018-04-10 at the Wayback Machine
  10. ^ "Implement TCL in TCL". Archived from the original on 2017-06-04. Retrieved 2017-09-19.
  11. ^ "Google Code Archive - Long-term storage for Google Code Project Hosting". Archived from the original on 2014-12-28. Retrieved 2015-05-27.