The Mishnah says in Pirkei Avot (5:19): "Any love that is dependent on a specific reason, when the reason disappears, so does the love. A love that is not dependent on a specific reason, that love will never cease". Often feelings of love are only based on certain qualities of the other person. Obviously, it is these qualities, such as beauty, a good sense of humor, or kindness, that creates the initial attraction. But, says the Mishnah, if this is the sole basis for the love, the instant these qualities cease to exist, so will the love.
I'm almost certain that is the source of the quotation that the rabbi gave. Perhaps you are able to study the text and its significance to life. Pirkei Avot ("Ethics of the Fathers") is one of the classical texts of Jewish morality.
I hope this is helpful. I wish you much happiness and success and please feel free to write anytime.