He first tasted Italian chocolate at a local fair and spent four years in Turin (Italy) learning the art of chocolate making. When he returned to Switzerland, he set up the first Swiss chocolate factory in Corsier, near Vevey in 1819. In 1825, he opened a second factory, which he later sold to his son Julian and son-in-law Daniel Peter.
His great innovation was the development of a smooth chocolate that could be formed into bars. This was a worldwide sensation.
In 1875, Daniel Peter had the idea of combining the chocolate with his neighbor Henri Nestlé's condensed milk to make milk chocolate. The Caillers and Peter eventually merged with the operation of Charles-Amédée Kohler (the inventor of hazelnut chocolate) to form the firm of Peter, Cailler, Kohler. In 1904, the "branches" were invented. These are sticks of chocolate. In 1937, the chocolate bar called Rayon was invented. The company was later purchased by the manufacturing giant Nestlé, in 1929.