Levine was born in Rockingham, North Carolina on June 8, 1937. He is the son of a local department store owner. As a boy he did work at his father's store. His father died in 1949 leaving Levine's older brother in charge of the store with the younger Levine being second in command. In 1951 Leon's brother was drafted into the Korean War, which left him in charge of the business at age 15.
After his years in charge of the family business, Levine purchased a bedspread factory in Wingate, North Carolina, and attended Wingate College, now Wingate University. After learning that his product didn't sell very well he found a market in Puerto Rico and successfully sold his factory's product. In 1959 he visited a store in Kentucky that sold nothing for over a dollar. He liked the concept and in November opened up a similar store in the 1500 block of Central Avenue in Charlotte that sold items for no more than 2 dollars. He named it Family Dollar for the affordability of its products and the demographic that it was aimed towards.
Leon, along with his wife Sandra Levine (née Poliakoff), an Anderson, SC native, have been involved in numerous charitable and philanthropic causes. Sandra Levine has served on the Board of Commissioners of the Carolinas Healthcare System. They have three grown children along with 10 grandchildren. Through their community achievements they having the pleasure of having many things in the state bear their last name. He is also a member of the North Carolina Business Hall of Fame
He and his wife currently reside in Charlotte, North Carolina.
- Queens University of Charlotte: Queens 
- UNCC: UNCC 
- Central Piedmont Community College: Levine Campus 
- Levine Children's Hospital 
- Levine Jewish Community Center  and Shalom Park 
- Levine Museum of the New South 
- Levine Science Research Center at Duke University
- Levine School of Health Sciences at Wingate University