Summers was born at Charlton Mackrell, near Ilchester, in Somerset, son of George Summers, builder and mason, and elder brother of Joseph Summers. George Summers had roaming ways which caused his family to be frequently in financial difficulties; his mother, however, was a woman of good character. Summers went to work from eight years of age and while working as a mason began to show ability in carving fancy stone work. This led to his being employed as an assistant in setting up a monumental figure at Weston-super-Mare which had been modelled by Henry Weekes, R.A. He saved money from his wages and at the age of 19 went to London and obtained work at Weekes's studio. Summers subsequently worked under L. Watson, another sculptor of the period, and studied at the Royal Academy schools. In 1851 he won the silver medal for the best model from life and the gold medal for the set subject, Mercy interceding for the Vanquished.
Summers, a hard worker, fell into ill health, and in 1852 sailed for Australia joining one of his brothers who had previously settled there. He tried his fortunes at gold-digging but seeing an advertisement for modellers for the newly built Parliament House in Melbourne, obtained a position and modelled the figures on the ceiling of the council chamber. The exhibiting of some busts at the intercolonial exhibition held in 1854 led to his getting commissions, and he opened a studio in Collins Street. Summers was a founder of the Victorian Society of Fine Arts in October 1856. Summers tutored William Stanford in sculpture design.
In 1864 it was decided to erect a memorial to the explorers Burke and Wills. Summers obtained the commission, and not only modelled the figures but built a furnace and himself cast them in bronze. The colossal figure of Burke was cast in one operation, an amazing feat when it is considered that there were no skilled workmen for this type of work in Australia. On the completion of this group he sailed for England on the True Briton in May 1867, and after obtaining various commissions went to Rome and opened a studio. There he did a large amount of work and was able to employ many assistants. In 1876 Sir William J. Clarke employed him to do four large statues in marble of Queen Victoria, Prince Albert, and the Prince and Princess of Wales for presentation to the Melbourne art gallery. These were completed in 1878. Soon afterwards Summers while on his way to England was taken seriously ill, and died after an operation for acute goitre at Paris on 30 November 1878.
Summers was a regular exhibitor at Royal Academy exhibitions; more than 40 of his works were shown between 1849 and 1876. He was a competent sculptor in a dull and uninspiring period of English art, and comparatively little of his work has lasting qualities. His 'Burke and Wills group' at Melbourne is a sound and dignified piece of work, his frieze of putti on the old Bank of New South Wales building, now in the University of Melbourne grounds, is charming, and the recumbent figure of Lady Macleay at Godstone, Surrey, is also meritorious. Personally Summers was modest, and his willingness to see ability in the work of other artists was a good influence in the early time of art in Victoria. Several examples of his work together with his portrait of Margaret Thomas are in the historical collection at the National Gallery of Victoria, Melbourne. Summers is also represented in the Adelaide gallery and at the Mitchell Library, Sydney. Summers married when a young man; his son, Charles Francis Summers, who survived him, was also a sculptor.