Between 1993 and 2003 he served as chairman of the local religious council in Beersheba. Since 2001 he has been the director general of Shas. He was first elected to the Knesset in 2003. After retaining his seat in the 2006 elections, Margi served as group chairperson of the party's faction at the Knesset. He also served briefly as chairman of the House Committee.
He retained his seat again in the 2009 elections, having been placed sixth on the Shas list, and was appointed Minister of Religious Services in the Netanyahu government.
Margi lives in Sde Tzvi. He is married and a father of two.
During Pope Benedict XVI's visit to Israel, he wrote a letter to the pontiff demanding that he clearly condemn the purveyors of Holocaust denial and antisemitism.