Data Execution Prevention
||It has been suggested that this article be merged into Executable space protection. (Discuss) Proposed since March 2015.|
Data Execution Prevention (DEP) is a security feature included in modern operating systems. It marks areas of memory as either "executable" or "nonexecutable", and allows only data in an "executable" area to be run by programs, services, device drivers, etc. It is known to be available in Linux, OS X, Microsoft Windows, iOS and Android operating systems.
DEP protects against some program errors, and helps prevent certain malicious exploits, especially attacks that store executable instructions in a data area via a buffer overflow. It does not protect against attacks that do not rely on execution of instructions in the data area. Other security features such as address space layout randomization, structured exception handler overwrite protection (SEHOP) and Mandatory Integrity Control, can be used in conjunction with DEP.
DEP runs in two modes: hardware-enforced DEP for CPUs that can mark memory pages as nonexecutable, and software-enforced DEP with limited protection for CPUs that do not have hardware support. Software-enforced DEP does not protect against execution of code in data pages, but counters SEH overwrite, another type of attack.
Mode of enforcement
Hardware-enforced DEP enables the NX bit on compatible CPUs, through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable. This helps prevent buffer overflow attacks from succeeding. In Windows, from version Vista, whether DEP is enabled or disabled for a particular process can be viewed on the Processes tab in the Windows Task Manager.
Some legitimate programs erroneously but harmlessly execute instructions in a data area; enabling Data Execution Prevention can have the unintended consequence of causing such software to malfunction. This needs to be remediated by the software developer; until then the program can be flagged as being allowed to execute code in data memory, although this is vulnerable to a possible targeted attack.
If an x86 processor supports DEP, and the system's BIOS supports it and it has been enabled, either by the manufacturer or the user, then the NX features are turned on in Windows, on a limited "OptIn" basis. (PAE must also be enabled in 32-bit Windows.) This setting provides protection only for a limited set of Windows system and binary files. To achieve full protection, the user must choose either "OptOut" ("turn on DEP for all program services except those I select"), covering all programs and processes not specifically exempted, or "AlwaysOn", covering all. These are configurable through the System Properties interface. If DEP is not supported by the particular x86 processor used, no protection is given. Intel's IA-64 architecture also has a form of NX, supported by Windows.
One kind of software DEP (without the use of the NX bit) is Microsoft's "Safe Structured Exception Handling" (SafeSEH). For properly compiled applications, SafeSEH checks that, when an exception is raised during program execution, the exception's handler is one defined by the application as it was originally compiled. The effect of this protection is that an attacker is not able to add his own exception handler which he has stored in a data page through unchecked program input. 
Where code is written and executed at runtime—a JIT compiler is a prominent example—the compiler can potentially be used to produce exploit code (e.g. using JIT Spray) that has been flagged for execution and therefore would not be trapped by DEP.
DEP occasionally encounters software problems, usually with older software that was not compiled and tested to conform to its restrictions. Users have experienced problems using various command-line commands that are a part of Microsoft's Services for Unix, which is included as part of Vista as well as Windows Server 2003 R2.
These problems may be prevented by disabling DEP, but this increases vulnerability of the system to malware. DEP can be turned off on a per-application basis, or turned off entirely for all non-essential Windows programs and services. Microsoft recommends that DEP not be globally disabled where an application malfunctions due to incompatibility with DEP. Instead, the supplier of the offending software should be contacted for an updated version that does not violate DEP; until the problem is corrected DEP may be disabled on an exception basis for the offending application only.
DEP is applied to an entire process, so even an application compatible with DEP may need to have it disabled if a non-DEP-compliant extension is added that runs in the same process space. For example, DEP-related problems can occasionally occur with DEP-compliant core operating system components such as Windows Explorer, Internet Explorer and Windows Installer as they support in-process third party extensions or plugins that may not be DEP-compliant.
- "IOS Hackers Handbook".
- Engler, Tobias (22 February 2011). "Schwere der Lücken ist besorgniserregend". Mac & i (in German). Heinz Heise. Retrieved 22 February 2011.
- http://pax.grsecurity.net/docs/aslr.txt elaborates
- "Changelog for kernel 2.6.8". Retrieved 2012-02-28.
- "A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003". Microsoft. 2006-09-26. Retrieved 2008-07-11.
- "Microsoft Knowledge Base kb912923". 2007-10-07. Retrieved 2009-03-14.
- "Microsoft Knowledge Base kb875352". 2006-09-26. Retrieved 2009-03-14.
- "System-wide configuration of DEP". 2006-09-26. Retrieved 2009-03-14.
- Johnson, Peter. "Yasm User Manual, win32: Safe Structured Exception Handling". Tortall Networks: Open Source and Free Software. Retrieved 27 September 2015.
- Writing JIT-Spray Shellcode for fun and profit, Alexey Sintsov, (pdf) "interesting way to bypass DEP and ASLR in browsers (not only) and Just-In-Time compilers was presented."
- Marc Liron. "Adding Software Exceptions In Data Execution Prevention (DEP)". Windows XP Update. Retrieved June 8, 2006.
- "Data Execution Prevention: frequently asked questions". Microsoft. Retrieved 2008-09-15.
- "Windows Installer custom action code must be compatible with Data Execution Prevention in Windows Vista". Microsoft. 2007-03-15. Retrieved 2008-07-10.
- A description of DEP and how to configure it
- Guide to hardware-based DEP protection
- The "Understanding Data Execution Prevention" help topic incorrectly states the default setting for DEP in Windows Server 2003 Service Pack 1
- Easy steps to define DEP exceptions in Vista and XP