A Democrat, Smith was elected to the North Carolina House of Representatives from the Greensboro area in 1970 but before the completion of his term, he was appointed to fill a vacancy in the North Carolina Senate caused by the resignation of Skipper Bowles. He was elected to the state Senate in 1972, 1974 and 1976. Smith did not run for re-election in 1978 in order to make an unsuccessful run for the United States Senate.
As an attorney, Smith was noted for taking up unpopular causes, such as in his defense of Junius Scales and in his lawsuit over the ban on Communists speaking on University of North Carolina campuses. Smith later taught constitutional law at the University of North Carolina School of Law. In 2006, the North Carolina Bar Association established an award named for Smith to honor "a person who has demonstrated extraordinary commitment to the ideals embodied in the Constitution of the United States and the Constitution of North Carolina."