Rowlands was educated at St Mary's College, Crosby and St Edmund Hall, Oxford before starting his career with the international engineering business IMI plc. He left the private sector to join the Department of Trade and Industry in 1974. At the DTI he was appointed private secretary to the Minister of State for Industry.
He moved to the then Ministry of Transport in 1983 where he has held a number of posts with responsibilities for finance, aviation, shipping, ports and railways. Before being appointed Permanent Secretary in May 2003, he was the director general for railways, aviation, logistics, maritime and security.
In this role he was central to the creation of Network Rail after Railtrack was put into administration and the delivery of public-private partnerships for the London Underground and the National Air Traffic Services. Rowlands received a knighthood in the Birthday Honours List 2006. He retired at 60. 
Since leaving the civil service his career moves were subject to a number of controversies, including government blocks on his attempts to join the Boards of British Airports Authority and Bechtel. At the time of his death his roles in the private sector included chairmanships of Gatwick Airport and Angel Trains. He was also the chairman of High Speed 2 Ltd, .
- "Water UK pays tribute to chairman". Utility Week. Retrieved 2014-05-16.
- "Sir David Rowlands - Ex-mandarin is on track for rail revolution". Daily Mail. 2009-10-19. Retrieved 2010-03-18.
- "David Rowlands". New Civil Engineer. 2003-05-29. Retrieved 2010-03-18.
- "Sir David Rowlands dies". transportxtra.com. 30 May 2014. Retrieved 17 December 2017.