He studied at the architectural school of the Royal Institute of Technology in Stockholm, and at the Royal Swedish Academy of Arts. Celsing has been the assistant of the widely known Swedish architect Sigurd Lewerentz. According to Adam Caruso, it was Celsing that helped Lewerentz to win the design competition for the Church of St. Mark (Markuskyrkan). This building and the later St. Petri Church (Olaus Petri kyrka) are now known as a starting point of Brutalist architecture.  
He later became professor of architecture at the Royal Institute of Technology. After working for some time in Beirut, he became head of the architectural office of AB Stockholms Spårvägar, the Stockholm tram and local railway authority, and built a number of suburban metro stations. He also designed several churches: in Härlanda (Gothenburg), Almtuna (Uppsala), and Vällingby, a much-publicized modernist suburb of Stockholm.
Celsing often worked in a brutalist style with large exposed grey concrete surfaces, but occasionally combined this with large glass panes exposing the structure of the building from the outside, and interior details in wood. The best known examples of this are the Kulturhuset at Sergels torg in central Stockholm (1966–1971), the Filmhuset, the house of the Swedish Film Institute on Gärdet in Stockholm (1968–1970), and his addition to Carolina Rediviva, the main building of the Uppsala University Library (1953–1962). In Uppsala, Celsing also built a new wing for the Stockholms nation building (begun in 1961), and the current Ekonomikum building at Uppsala University (completed 1976). 
Kulturhuset (begun 1966, completed 1971) is from most angles dominated by its concrete structure, with the adjacent theatre building having a façade of stainless steel, but from the front by its glass façade and the thin lines of the concrete floors, giving the impression of a number of shelves open towards the open place outside, Sergels torg. It is located in a part of Stockholm where the old structure of the city, the old architecture and even the topography was almost entirely replaced in the 1950s, 1960s and early 1970s with modernist architecture and new functions, mainly finance and large-scale retail businesses. The reaction against this development has been very strong, and many planned building and traffic projects had to be stopped from the 1970s and later as a result of the public opinion against what largely was, and continues to be, regarded as a significant loss of irreplaceable cultural values (see for instance the film documentary Staden i mitt hjärta from 1992 by Anders Wahlgren).
Kulturhuset was intended as a "cultural oasis" in this new city of commerce, with an open library in the bottom floor, a large theatre, and space for exhibitions in the rest of the building; the original intention was for the Moderna Museet (the Stockholm Museum of Modern Art) to occupy large parts of the building, but the museum dropped out of the project in 1969. As the flagship building of the city redevelopment, Kulturhuset has, perhaps unfairly, become the target of much of the anger against the whole city redevelopment project. Already in 1970, one critic, Claes Brunius, noted in Expressen that "Stockholm is building the largest vacuum in the country". The public opinion against the project may have been strengthened by the fact that the Kulturhuset and the interconnected Stockholm City Theatre were used as a temporary parliamentary building for several years, but as such the structure won the prestigious Kasper Salin Prize (Kasper Salin-priset) in 1972.
One of his last buildings is the Bank of Sweden building at Brunkebergstorg in Stockholm, the design of which is dominated by the use of cubes, squares, spheres and circles, all intended to give the impression of stability. It makes use of thick slabs of roughly hewn dark granite in the façade, thus borrowing from the renaissance tradition of using rustication to give the impression of solidity and particularly from the 19th century neo-renaissance practice of using rustication in bank buildings for this purpose. The Bank of Sweden had since 1907 been located to a semi-circular building on Helgeandsholmen, but moved to Celsing's building in 1976. (The older building was combined with the adjacent Parliament House as a new parliamentary building, where the parliament returned after its years in exile in Celsing's Kulturhuset.)
In 1948, he married Birgitta Dyrssen (1922–2004). Their son, Johan Celsing, has become noted as an architect of public buildings since the 1990s, and one of his buildings won the Kasper Salin Prize in 1999.
- Härlanda Church, Gothenburg
- St Tomas Church, Vällingby
- Boliden Church, Boliden
- Nacksta Church, Sundsvall
- Kulturhuset, Stockholm
- Bank of Sweden Building, Stockholm
- Villa Klockberga, private property in Lovön, outside Stockholm
- Villa Friis, private property in Lovön, outside Stockholm.
- Filmhuset in Gärdet, Stockholm
- Olaus Petri Church, Stockholm
- Almtuna Church in Uppsala
- Extension of university library Carolina Rediviva, Uppsala
- Ekonomikum, Uppsala
- "The architect Peter Celsin". Riksbank. Retrieved April 1, 2019.
- Harnesk, Paul, ed. (1962). Vem är vem? 1, Stor-Stockholm [Who is who? 1, Greater Stockholm] (in Swedish) (2nd ed.). Stockholm: Vem är vem. p. 250.
- Harnesk, Paul, ed. (1964). Vem är vem?. 2, Svealand utom Stor-Stockholm [Who is Who?. 2, Svealand excluding Greater Stockholm] (in Swedish) (2nd ed.). Stockholm: Bokförlaget Vem är vem. p. 145.
- "Brutalism". Designing Buildings. Retrieved April 1, 2019.
- Patrick A. George. "Sigurd Lewerentz (1885-1975) Church of St. Mark, Bjoerkhagen, Sweden (1956-60)". upenn.edu. Retrieved April 1, 2019.
- "Stockholms Nation". Stockholms Nation. Archived from the original on 2014-04-21. Retrieved April 1, 2019.
- "Kulturhuset Stadsteatern". kulturhusetstadsteatern. Retrieved April 1, 2019.
- "Tidigare Kasper Salin-pristagare (1962–2004)". arkitekt.se. Retrieved April 1, 2019.
- "Guided tour of the building". Riksbank. Retrieved April 1, 2019.
- "Johan Celsing". celsing.se. Retrieved April 1, 2019.
- Peter Celsing at archINFORM
- The Bank of Sweden building, with images and a brief biography of the architect
- History of Kulturhuset, by Beate Sydhoff, former director of Kulturhuset (in Swedish)
- Filmhuset, main page and Filmhuset, history
- History of the Stockholm Nation buildings, Uppsala (in Swedish)
- Ekonomikum[permanent dead link], formerly known as Humanistiskt-samhällsvetenskapligt centrum, Uppsala University (in Swedish)