The band's first album, 1704, was produced By Ian Ball of the British band Gomez.
Gelbison's second album, See the World, was released in 2004, and also featured Ian Ball on production duties, along with Ben Frost and Gelbison. A number of tracks, including "All the Rage" and "Holy", received significant rotation on Triple J. The album also featured a number well-known artists, such as Ben Lee, Old Man River, Ian Ball, and Luke Steele.
Gelbison severed ties with record label EMI in 2006 and worked on their third album towards the end of 2006 with J Walker of Machine Translations as producer. After recording with J Walker, the band decided to head back into the studio and record a new range of songs with producer Tony Buchen. Core members Edo and Nadav Kahn decided to conclude Gelbison and release these new songs under their new project Kahn Brothers, with their debut album Love Melts Fear released in April 2007 through Shakti/Interia Records.