Trace vector decoder

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

Trace vector decoder is a system that uses a microprocessor's trace mode to decode encrypted code just-in-time before it is executed and possibly re-encrypt it after the execution. It can be used to enforce copy protections for some computer systems.

Trace Vector in Motorola 68000[edit]

As an example, Motorola 68000 has a trace mode where a trace exception vector is executed before each instruction in the main program. The processor automatically changes execution to the trace exception vector before executing any instruction from the main program. The trace exception vector decodes the instruction that will be executed after the exception vector. The next time trace exception happens the old decoded location may possibly be re-encrypted.

Following code snippet is an example of a program initializing a trace exception routine.

	MOVEM.L	Stack,D0-D7/A0-A6      ; Initialize registers
Stack	MOVE.L	#$4E730000,-(SP)       ; Start loading trace exception
	MOVE.L	#$00000010,-(SP)       ; vector into stack
	MOVE.L	#$0004DDB9,-(SP)
	MOVE.L	#$BD96BDAE,-(SP)
	MOVE.L	#$B386B586,-(SP)
	MOVE.L	#$D046D246,-(SP)
	MOVE.L	#$0246A71F,-(SP)
	MOVE.L	#$00023C17,-(SP)
	MOVE.W	#$2C6F,-(SP)
	MOVE.L	SP,($24).W             ; Set trace exception vector
	ORI.W	#$A71F,SR              ; Enter trace mode
        NOP
        ; Trace vector happens now for the first time.
        ; Code after this line is encrypted.

A disassembly of the trace exception vector that is loaded on the stack:

TraceCode:
	MOVE.L	(2,SP),A6              ; Load return address from
                                       ; supervisor stack.
	MOVE.W	(SP),D6                ; Load condition codes of the main
                                       ; program.
	AND.W	#$A71F,D6
	ADD.W	D6,D0
	ADD.W	D6,D1
	EOR.L	D1,D6
	EOR.L	D2,D6
	EOR.L	D6,(A6)                ; Decrypt 8 bytes ahead in main
	EOR.L	D6,(4,A6)
	RTE                            ; Return from exception

Note that registers altered in the trace vector affect the main program that is being traced. Usually registers are pushed onto stack in any exception vector, because altering them would break the main program. However, purpose of this vector is to obfuscate the code against reverse engineering.

It should also be noted that condition code register (CCR) affects the decryption process. For example, an arithmetic operation in the main program having the 0 number as a result, will cause zero flag bit to be set in CCR. This will cause the value in (SP) to be changed in the trace vector. This is also done to obfuscate against reverse engineering.

Rob Northen copylock has a trace vector decoder that was used on the Amiga and Atari ST platforms.