After graduating from Harvard University in 1965, he went to work for Newsweek as a journalist. He remained there for six months and was then hired by The New Yorker. He subsequently worked as a film critic at Esquire magazine from 1969 until 1972.
He met Carly Simon in 1968 when they were both working as counselors at a summer camp in the Berkshires. The two became close friends. Most of Simon's albums include one or two songs co-written with Brackman; typically, Simon writes the music and Brackman writes the lyrics. Among the dozens of songs they have written together are the top ten hits, "That's the Way I've Always Heard It Should Be" (1971) and "Haven't Got Time for the Pain" (1974), both of which were sung by Simon.
The lyrics to the Broadway musical King of Hearts were also written by Brackman, and so, too, were the screenplays for The King of Marvin Gardens (1972) and Times Square (1980). He has also collaborated musically with James Taylor, Steve Winwood, Dr. John, Fred Astaire, Michel Polnareff and Dionne Warwick. He was the executive producer for the acclaimed Terrence Malick film, Days of Heaven (1978).
He married the late Mindy Jostyn, and co-authored the lyrics on her CDs.