He was born in 1926, the son of George Willoughby of Tinahely, County Wicklow, and his wife, Mina (née Rothwell). His first clerical post was Curate of Drumglass, near Dungannon from 1950 to 1953. It was at that time he first met his future wife, Valerie. His next post was Curate of St Catherine’s, Dublin 1953-55, followed by Curate of Christ Church, Bray 1955-59, then Rector of Delgany 1959-69, and Rector of Glenageary 1969-80. It was while at Glenageary that he also became Treasurer of St Patrick's Cathedral, Dublin 1977-80 and Archdeacon of Dublin 1978-80. On 28 March 1980 he was elected Bishop of Cashel and Ossory and was consecrated on 25 April 1980. A position which he kept until his retirement in 1997. He died on 6 February 2006 and was buried near the main door of St. Canice's Cathedral, Kilkenny.
- Pioneering figure in ecumenical relations. The Irish Times, Saturday, 2 February 2006.
- The Rt Revd Noel Vincent Willoughby. The Church Times. Retrieved on 8 January 2008.
- The Rt Revd Noel Willoughby, former Bishop of Cashel and Ossory dies. Retrieved on 8 January 2009.
- Fryde, E. B.; Greenway, D. E.; Porter, S.; Roy, I. (1986). Handbook of British Chronology (Third Edition, revised ed.). Cambridge: Cambridge University Press. p. 382. ISBN 0-521-56350-X.