Data Execution Prevention

File:Data Execution Prevention controls.png

DEP controls in Windows Vista

File:Data Execution Prevention blocked program.png

DEP causing Windows XP to end a program

Data Execution Prevention (DEP) is a feature included in modern Microsoft Windows and Linux operating systems that is intended to prevent an application or service from executing code from a non-executable memory region. This helps prevent certain exploits that store code via a buffer overflow, for example. DEP runs in two modes: hardware-enforced DEP for CPUs that can mark memory pages as nonexecutable, and software-enforced DEP with a limited prevention for CPUs that do not have hardware support. Software-enforced DEP does not protect from execution of code in data pages, but instead from another type of attack (SEH overwrite).

DEP was introduced in Windows XP Service Pack 2 and is included in Windows XP Tablet PC Edition 2005, Windows Server 2003 Service Pack 1 and Windows Vista. Later versions of the operating systems support this feature as well. NX bit support has been available under Linux since kernel release 2.6.8 in August 2004.

Hardware protection

Hardware-enforced DEP enables the NX bit on compatible CPUs, through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable. This helps prevent buffer overflow attacks from succeeding.

In some instances, Data Execution Prevention can have the unintended consequence of preventing legitimate software from executing. In these cases, the affected software needs to be flagged as being allowed to execute code in those parts of memory, but this itself leads to a possible attack if the application isn't rigorous in validating data that is passed into a region of memory that is marked as being executable.

If the x86 processor supports this feature in hardware, then the NX features are turned on automatically in Windows by default. If the feature is not supported by the x86 processor, then no protection is given. Outside of the x86 architecture, a version of NX also exists for Intel's IA-64 architecture that is supported by Windows.

Software protection

Software DEP, while unrelated to the NX bit, is what Microsoft calls their enforcement of "Safe Structured Exception Handling". Software DEP/SafeSEH simply checks when an exception is thrown to make sure that the exception is registered in a function table for the application, and requires the program to be built with it. However, even though it creates an impression that software DEP is related to the prevention of executing code in data pages, it is a separate form of protection.

Limitations

Unlike similar protection schemes available on other operating systems, DEP provides no address space layout randomization (a feature now available in Windows Vista), which may allow return-to-libc attacks that could feasibly be used to disable DEP during an attack.

The possibility has now been demonstrated against Windows Hardware-enforced DEP by skape in the following 'Uninformed' article [1] which relies on a return-to-libc style attack. This technique relies on directly pointing the EIP register to the known service-pack-dependent location which applies the OptIn/OptOut mechanism. It is reliant on the boot-time option of OptOut/OptIn being available. If all pages are strictly enforced, then this attack will not succeed. The PaX documentation further elaborates on why ASLR is necessary.

Software conflicts

DEP is occasionally the cause of software problems, usually with older software.

It has exposed bugs in the Virtuozzo virtualization software that prevent certain programs from being virtualized correctly. Users have also experienced problems using Tcl/Tk for Windows distribution from ActiveState when using the Expect extension to spawn Telnet sessions. DEP kills the Telnet session child process, giving an error: "child process terminated abnomally".

In most cases, these problems may be solved by disabling the DEP features. DEP can be turned off on a per-application basis, or turned off entirely, retaining compatibility for older programs.

See this external link by Microsoft for a detailed description of DEP.

Software configuration

DEP configuration for the system is controlled through switches in the Boot.ini file. DEP can be configured by using the System dialog box in Control Panel.

The Boot.ini file settings are as follows: /noexecute= policy_level Note policy_level is defined as AlwaysOn, AlwaysOff, OptIn, or OptOut.

OptIn: This setting is the default configuration for Windows XP. On systems with processors that can implement hardware-enforced DEP, DEP is enabled by default for limited system binaries and programs that "opt-in." With this option, only Windows system binaries are covered by DEP by default.

OptOut: This setting is the default configuration for Windows 2003. DEP is enabled by default for all processes. A list of specific programs that should not have DEP applied can be entered using the System dialog box in Control Panel. Network administrators can use the Application Compatibility Toolkit to "opt-out" one or more programs from DEP protection. System compatibility fixes, or shims, for DEP do take effect. Also note that Windows silently disables DEP for certain executables, such as those packaged with ASPack.

AlwaysOn: This setting provides full DEP coverage for the whole system. All processes always run with DEP applied. The exceptions list to exempt specific programs from DEP protection is not available. System compatibility fixes for DEP do not take effect. Programs that have been opted-out by using the Application Compatibility Toolkit run with DEP applied.

AlwaysOff: This setting does not provide any DEP coverage for any part of the system, regardless of hardware DEP support.

See also

• Executable space protection
• Buffer overflow
• Heap overflow
• Stack-smashing protection
• NX bit

External links

• MSKB 875352 — A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003
• NXTEST A free tool that can test executing code from a data segment, the process heap and the stack to see if DEP is enabled and working.
• Guide to hardware-based DEP protection
• The "Understanding Data Execution Prevention" help topic incorrectly states the default setting for DEP in Windows Server 2003 Service Pack 1