"N.Y. State of Mind" ranks #74 on About.com's Top 100 Rap Songs.
Rolling Stone magazine placed the song on its list of "The 50 Greatest Hip-Hop Songs of All Time."
Marc L. Hill of PopMatters describes "N.Y. State of Mind" as a standout track on Illmatic claiming that it "provides as clear a depiction of ghetto life as a Gordon Parks photograph or a Langston Hughes poem." The song is also one of a few rap songs to be featured in the Norton Anthology of African American Literature. It is featured on Nas' 2007 greatest hits album as the only non-single song in the album, and on the 1999 compilation Best of D&D Studios, Vol. 1. Steve 'Flash' Juon of RapReviews.com states:
"[Illmatic] was to be an album steeped in the rich traditions of hip-hop history, mixed with the most advanced verbal styles and fat beats that could be put on wax. And if it couldn't be set off any more right already, the DJ Premier produced "N.Y. State of Mind" was designed to knock you right off your feet. Primo's knack for finding the illest piano loops and matching them to pounding beats was perfected in this track, and paired with a Rakim sample on the chorus that provided the mental link for an analogy most rap heads had already made by now: Nas was the NEW Rakim on the block.