x86 memory segmentation

x86 memory segmentation refers to the implementation of memory segmentation on the x86 architecture. Memory is divided into portions that may be addressed by a single index register without changing a 16-bit segment selector. In real mode or V86 mode, a segment is always 64 kilobytes in size (using 16-bit offsets). In protected mode, a segment can have variable length. Segments can overlap.

Real mode

In real mode, the 16-bit segment selector is interpreted as the most significant 16 bits of a linear 20-bit address, called a segment address, of which the remaining four least significant bits are all zeros. The segment address is always added with a 16-bit offset to yield a linear address. For instance, the segmented address 06EFh:1234h has a segment selector of 06EFh, representing a segment address of 06EF0h, to which we add the offset, yielding the linear address 06EF0h + 1234h = 08124h (cf. hexadecimal).

Because of the way the segment address and offset are added, a single linear address can be mapped to 4096 distinct segment:offset pairs. For example, the linear address 08124h can have the segmented addresses 06EFh:1234h, 0812h:0004h, 0000h:8124h, etc. This could be confusing to programmers accustomed to unique addressing schemes, but it can also be used to advantage, for example when addressing multiply-nested data structures. While real mode segments are always 64 KiB long, the practical effect is only that no segment can be longer than 64 KiB, rather than that every segment must be 64 KiB long. Because there is no protection or privilege limitation in real mode, even if a segment could be defined to be smaller than 64 KiB, it would still be entirely up to the programs to coordinate and keep within the bounds of their segments, as any program can always access any memory (since it can arbitrarily set segment selectors to change segment addresses, with absolutely no supervision). Therefore, real mode can just as well be imagined as having a variable length for each segment, in the range 1 to 65536 bytes, that is just not enforced by the CPU.

(Note that the leading zeros of the linear address, segmented addresses, and the segment and offset fields, which are usually neglected, were shown here for clarity).

The effective 20-bit address space of real mode limits the addressable memory to 2²⁰ bytes, or 1,048,576 bytes. This derived directly from the hardware design of the Intel 8086 (and, subsequently, the closely related 8088), which had exactly 20 address pins. (Both were packaged in 40-pin DIP packages; even with only 20 address lines, the address and data buses were multiplexed to fit all the address and data lines within the limited pin count.)

In 16-bit real mode, enabling applications to make use of multiple memory segments (in order to access more memory than available in any one 64K-segment) is quite complex, but was viewed as a necessary evil for all but the smallest tools (which could do with less memory). The root of the problem is that no appropriate address-arithmetic instructions suitable for flat addressing of the entire memory range are available.^{[citation needed]} Flat addressing is possible by applying multiple instructions, which however leads to slower programs.

Protected mode

80286 protected mode

The 80286's protected mode extends the processor's address space to 2²⁴ bytes (16 megabytes), but not by adjusting the shift value. Instead, the 16-bit segment registers now contain an index into a table of segment descriptors containing 24-bit base addresses to which the offset is added. To support old software, the processor starts up in "real mode", a mode in which it uses the segmented addressing model of the 8086. There is a small difference though: the resulting physical address is no longer truncated to 20 bits, so real mode pointers (but not 8086 pointers) can now refer to addresses between 100000_h and 10FFEF_h. This roughly 64-kilobyte region of memory was known as the High Memory Area (HMA), and later versions of MS-DOS could use it to increase the available "conventional" memory (i.e. within the first MB). With the addition of the HMA, the total address space is approximately 1.06MB. Though the 80286 does not truncate real-mode addresses to 20 bits, a system containing an 80286 can do so with hardware external to the processor, by gating off the 21st address line, the A20 line. The IBM PC AT provided the hardware to do this (for full backward compatibility with software for the original IBM PC and PC-XT models), and so all subsequent "AT-class" PC clones did as well.

The protected mode segmentation system, present in the 80286 and later x86 CPUs, can be used to enforce separation of unprivileged processes, but most 32-bit operating systems uses the paging mechanism introduced with the 80386 for this purpose instead. Such systems set all segment registers to point to a segment descriptor with offset=0 and limit=2³², giving an application full access to a 32-bit flat virtual address space through any segment register. By this method, normal application code does not have to deal with segment registers at all. This was possible because the 80386 widened the general purpose registers (i.e. the offset registers) to 32 bits. Naturally, the base addresses in the descriptors were also widened to 32 bits.

Detailed Segmentation Unit Workflow

A logical address consists of a 16-bit segment selector (supplying 13+1 address bits) and a 16-bit offset. The segment selector must be located in one of the segment registers. That selector consists of a 2-bit Requested Privilege Level (RPL), a 1-bit Table Indicator (TI), and a 13-bit index.

The processor accesses the 64-bit segment descriptor structure in the Global Descriptor Table if TI is 0 or in the Local Descriptor Table if TI is 1. It then performs the privilege check:

DPL < max (CPL,RPL)

where CPL is the current privilege level (found in the lower 2 bits of the CS register), RPL is the requested privilege level from the segment selector, and DPL is the descriptor privilege level of the segment (found in the descriptor). All privilege levels are integers in the range 0..3, where the lowest number corresponds to the highest privilege.

If the inequality is true, the processor generates a general protection fault (GP). Otherwise, address translation continues. The processor then takes the 32-bit or 16-bit offset and compares it against the segment limit specified in the segment descriptor. If it is larger, a GP fault is generated. Otherwise, the processor adds the 24-bit segment base, specified in descriptor, to the offset, creating a linear physical address.

The privilege check is done only when the segment register is loaded, because segment descriptors are cached in hidden parts of the segment registers.^{[citation needed]} (Is this true on the 80286, or only on the 80386 and above?)

80386 protected mode

In the 386 and later, protected mode retains the segmentation mechanism of 80286 protected mode, but a paging unit has been added as a second layer of address translation between the segmentation unit and the physical bus. Also, importantly, address offsets are 32-bit (instead of 16-bit), and the segment base in each segment descriptor is also 32-bit (instead of 24-bit). The general operation of the segmentation unit is othewise unchanged. The paging unit may be enabled or disabled; if disabled, operation is the same as on the 80286. If the paging unit is enabled, addresses in a segment are now virtual addresses, rather than physical addresses as they were on the 80286. That is, the address that is derived by adding the segment starting address from a segment descriptor to an offset is a virtual (or logical) address; the segment starting address is a virtual address, and the offset is also in the virtual address space. Programs issue logical (46-bit^[1]) addresses which go through the segmentation unit to be checked and translated into linear 32-bit addresses, before being sent to the enabled paging unit which ultimately translates them into physical addresses (which are also 32-bit on the 386, but can be larger on newer processors which support Physical Address Extension).

The 80386 also introduced two new general-purpose data segment registers, FS and GS, to the original set of four segment registers (CS, DS, ES, and SS).

Later developments

The x86-64 architecture does not use segmentation in long mode (64-bit mode). Four of the segment registers: CS, SS, DS, and ES are forced to 0, and the limit to 2⁶⁴. However, in x86 versions of Microsoft Windows, the FS segment points to a small data structure, different for each thread, which contains information about exception handling, thread-local variables, and other per-thread state. The x86-64 architecture supports this technique by allowing a nonzero base address for FS & GS.

Practices

Logical addresses can be explicitly specified in x86 assembler language, e.g. (AT&T syntax):

movl $42, %fs:(%eax) ; Equivalent to M[fs:eax]<-42) in RTL

However, segment registers are usually used implicitly.

All CPU instructions are implicitly fetched from the code segment specified by the segment selector held in the CS register.

Most memory references come from the data segment specified by the segment selector held in the DS register. These may also come from the extra segment specified by the segment selector held in the ES register, if a segment-override prefix precedes the instruction that makes the memory reference. Most, but not all, instructions that use DS by default will accept an ES override prefix.

Processor stack references, either implicitly (e.g. push and pop instructions) or explicitly (memory accesses using the ESP or (E)BP registers) use the stack segment specified by the segment selector held in the SS register.

String instructions (e.g. stos, movs) also use the extra segment specified by the segment selector held in the ES register.

Segmentation cannot be turned off on x86-32 processors (this is true for 64-bit mode as well, but beyond the scope of discussion), so many 32-bit operating systems simulate a flat memory model by setting all segments' bases to 0 in order to make segmentation neutral to programs. For instance, the Linux kernel sets up only 4 general purpose segments:

* __KERNEL_CS (Kernel code segment, base=0, limit=4GB, DPL=0) 
* __KERNEL_DS (Kernel data segment, base=0, limit=4GB, DPL=0) 
* __USER_CS   (User code segment,   base=0, limit=4GB, DPL=3) 
* __USER_DS   (User data segment,   base=0, limit=4GB, DPL=3)

Since the base is set to 0 in all cases and the limit 4 GiB, the segmentation unit does not affect the addresses the program issues before they arrive at the paging unit. (This, of course, refers to 80386 and later processors, as the earlier x86 processors do not have a paging unit.)

Current Linux also uses GS to point to thread-local storage.

Segments can be defined to be either code, data, or system segments. Additional permission bits are present to make segments read only, read/write, execute, etc.

Note that, in protected mode, code may always modify all segment registers except CS (the code segment). This is because the current privilege level (CPL) of the processor is stored in the lower 2 bits of the CS register. The only way to raise the processor privilege level (and reload CS) is through the lcall (far call) and int (interrupt) instructions. Similarly, the only way to lower the privilege level (and reload CS) is through lret (far return) and iret (interrupt return) instructions. In real mode, code may also modify the CS register by making a far jump (or using an undocumented POP CS instruction on the 8086 or 8088)^[2]). Of course, in real mode, there are no proviledge levels; all programs have absolute unchecked access to all of memory and all CPU instructions.

For more information about segmentation, see the IA-32 manuals freely available on the AMD or Intel websites.

External links

^ The 46 bits are 14 bits from a 16-bit segment register (the other two bits are the priviledge level) plus a 32-bit offset.
^ POP CS must be used with extreme care and has limited usefulness, because it immediately changes the effective address that will be computed from the instruction pointer to fetch the next instruction. Generally, a far jump is much more useful. The existence of POP CS is probably an accident, as it follows a pattern of PUSH and POP instruction opcodes for the four segment registers on the 8086 and 8088.

[1] The 46 bits are 14 bits from a 16-bit segment register (the other two bits are the priviledge level) plus a 32-bit offset.

[2] POP CS must be used with extreme care and has limited usefulness, because it immediately changes the effective address that will be computed from the instruction pointer to fetch the next instruction. Generally, a far jump is much more useful. The existence of POP CS is probably an accident, as it follows a pattern of PUSH and POP instruction opcodes for the four segment registers on the 8086 and 8088.

[1]

[2]