Gazit was born in 1926 in Istanbul, Turkey, to a family of UkrainianJews. He served as the head of the assessment department in IDF intelligence before the Six Day War, but then took leave to study for a Master degree in history. Gazit's studies were interrupted in the summer of 1967, when he was appointed by Defense Minister Moshe Dayan to be in charge of a committee tasked with running the political, security, and later economic affairs in the newly captured territories. This group would later be renamed "The Unit for the Coordination of Operations in the Territories".
After seven years, Gazit was promoted to head of the Military Intelligence Directorate, a position he held from 1974 to 1978.