In 1984 he was elected to the Knesset, representing the Likud party. He chaired the subcommittee for Soviet Jewry. He retained his seat in the 1988 election and chaired the subcommittee for the defense budget during the 12th Knesset's term. Re-elected in 1992, 1996, 1999 and 2003, he served as chair of the foreign affairs and defense committee from 1996 until 1999, when he became chairman of the State Control Committee.
Landau lost his seat in the 2006 election; he was fourteenth on the party's list but Likud won only 12 seats.
In 2008 he announced that he was joining Yisrael Beiteinu. He was placed second on the party's list for the 2009 elections, and was returned to the Knesset as the party won 15 seats. After Yisrael Beiteinu joined the Likud-led coalition, Landau was appointed Minister of National Infrastructure.
On June 5, 2011 Landau spoke at the inauguration of Ketura Sun, Israel's first commercial solar field built by Arava Power Company, located at Kibbutz Ketura.
On the 12th of October 2011 Landau was one of only three cabinet minsters to vote against the proposed swap of 1027 Palestinian prisoners for Captured Israeli soldier Gilad Shalit. He called the deal a 'great victory for terrorism'