Ben Khalfallah played in the French second and third divisions until the age of 25, when he got his first taste of Ligue 1 football with SM Caen. His new team, however, was relegated at the end of the season after a decent first season in Ligue 1 for the midfielder.
In the summer of 2009, he signed for Valenciennes FC for three years. He started the season very well, starting five games and coming on as a substitute in another five, scoring 3 beautiful goals and providing 5 assists.
In August 2010, he joined Bordeaux for a fee of €5 million.