"All Too Well" was the first song Swift wrote for album Red. While collaborating with different writers on Red, Swift wrote the song along with Liz Rose. The song was inspired by Swift's ex-boyfriend Jake Gyllenhaal, according to some speculation.
Swift has a said that the song was "the hardest to write on the album", saying "'All Too Well' [was difficult to write emotionally] because it took me a really long time to filter through everything I wanted to put in the song without it being a 10 minute song, which you can’t put on an album. .