Tan Zhenlin was born in You County, Hunan. He joined the Chinese Communist Party in 1926, and participated in the Chinese Civil War since the Jinggang Mountains rebellion in the early days of the war. By 1949 he rose to the first deputy political commissar of the Third Field Army of the PLA.
After the founding of the People's Republic in 1949, Tan became the Party Secretary and Governor of Zhejiang. He served as the Third Secretary of the Communist Party's East China Bureau, then Governor of Jiangsu. In December 1954 Tan became the party's deputy Secretary-General (not to be confused with General Secretary), and later Vice-Premier. During the Great Leap Forward, Tan became a Secretary of the Secretariat, in charge of agriculture. He supported Mao's policies during the Great Leap, which he regretted later in life.
Tan became a central figure opposed to the Central Cultural Revolution Group (CCRG) in the early stages of the Cultural Revolution. Tan, along with other Communist revolutionaries such as Chen Yi and Nie Rongzhen, openly attacked the CCRG and their insistence on expanding the Cultural Revolution. Tan bluntly told the assembled leaders that the Cultural Revolution was an attempt to purge the Communist old guard. Tan also reputedly said, "I made three mistakes in my life. I should not have lived to this day. I should have never joined the revolution with Mao. I should have never joined the Communist Party." He was then thoroughly denounced by Lin Biao as part of the "February Countercurrent" and subsequently purged. However, after Lin Biao's own demise in 1971, Mao attempted to reach out to his old revolutionary colleagues, and restored Tan's political standing. In 1975 Tan was named a Vice-Chairman of the National People's Congress.