Karl Sack studied law in Heidelberg where he joined a Burschenschaft (Burschenschaft Vineta) and after a time in legal practice became a judge in Hesse. He married Wilhelmine Weber and had two sons. In 1934, Sack joined the newly established Reichskriegsgericht (Reich Military Court) where he quickly rose to a senior position. He was able to delay proceedings against Army Commander-in-Chief Werner von Fritsch who had been falsely accused of homosexuality by the Gestapo in an attempt to discredit him for his opposition to Hitler's attempts to subjugate the German armed forces. In the fall of 1942, Karl Sack became Judge Advocate General of the Army.
During World War II, Sack maintained contacts within the resistance circles in the military, including Admiral Wilhelm Canaris, Major General Hans Oster and Hans von Dohnanyi, as well as with others within the Abwehr (German military intelligence). He was part of the attempt to assassinate Hitler on July 20, 1944 and after that failed attempt he was arrested on August 9, 1944. In the very last days of the war, he was brought before an SS drumhead court-martial presided over by Otto Thorbeck. He was sentenced to death and hanged 2 days later. Sack had been slated for the role of Justice Minister within a planned post-coup civilian government.
In 1984, Sack's role as a member of the resistance was remembered with a bronze plaque placed in the former Reichskriegsgericht in Berlin-Charlottenburg. There was some opposition to this honour as Sack favoured a far-reaching interpretation of what constituted desertion, which must have led to more than a few death sentences.
In Bosenheim, a suburb of Bad Kreuznach, a street has been named in his honour.