Bailly replaced Jan Moons as Genk's first-choice goalkeeper at the beginning of the 2006–07 season; he played 90 minutes in every single Belgian League match for the club that season. He kept 14 clean sheets and received no bookings. His good performance contributed to Genk's surprising second-place finish. He was transferred to Borussia Mönchengladbach during the winter of season 2008–09 and immediately adapted to the Bundesliga. He was elected best player of gameday 20 in the league. During the 2011–12 season he was initially loaned out to Swiss side Neuchâtel Xamax, for whom he never played a single match. In the second part of the season, he was loaned back to Genk. After the season he returned to Mönchengladbach where his contract was not renewed, allowing him to sign as a free agent player for Oud-Heverlee Leuven. At OHL, he was rejoined with coach Ronny Van Geneugden, who was his coach during his period as a youth player at Genk. After a series of good performances, Bailly was rewarded on 2 December with a one year extension, giving him a contract until the end of the 2013–14 season.