It's a cute quote, this one. Specially because you can interpret it in different ways.
In my opinion, "poem", here, refers to a person, I mean, "the first person you've ever loved", in the sense that you spent your lifetime comparing everyone that you loved to the person you loved first. That can be bad, but it also shows how intensily you loved that person, to the point that you can't forget about the person. It's interesting how some people just get so attached to their first love. And, for me, that is really cute.