I've heard a number of reasons for this custom. The most obvious being that it is a sign that the person visited the grave, and stones are generally readily accessible in a cemetery. Another reason has to do with the Hebrew of the word stone. Even. This word can be viewed as a combination of the words Av and Ben, thereby showing continuity.
A very early reference to this custom is found in a commentary to the Shulchan Aruch, written by Rav Yehuda Ashkenazi (early 1700s) called the B'er Heitev. He quotes the Maharash, who explains that the custom of placing stones or tufts of grass on the grave is for the honor of the deceased person by marking the fact that his grave has been visited.
Once, when I was touring the Mount of Olives cemetery, my Yerushalmitour guide told me the following story, a story that purports to explain this custom:
Sometime during the Turkish occupation of Israel, on a Shabbat, an Arab was murdered in Jerusalem. Quickly, the rumor spread that he was killed by a Jew, and an immediate expulsion order was declared. The Jews of Jerusalem had to pick themselves up and leave or be killed. A noted kabbalist (mystic) came upon the scene of the crime, which was crowded with Arab onlookers. Even though it was Shabbat, the kabbalist wrote one of G-d's names on a piece of paper and placed it upon the body of the dead man. The dead man rose and pointed to one of the Arabs standing in the crowd who became violently afraid and admitted that he had done the killing. The expulsion order was rescinded.
Shortly afterwards the kabbalist, who was an elderly man, approached the chevra kadisha (burial society) and asked that his tombstone be pelted with stones after his death because he had written during Shabbat. He understood that due to the danger to life he had been permitted to desecrate the Shabbat, but he felt that some form of repentance was in order nevertheless. Stoning his grave would symbolize the stoning penalty meted out to Shabbat desecraters. At first the chevra kadisha refused because of the implied dishonor the stoning would represent to so righteous a Jew, but the kabbalist persisted. Finally, they agreed to place stones on his grave, but only if they would institute the custom that all graves would have stones placed on them in the future. If stones were place on everyone's grave, it would not be a dishonor to the kabbalist. From then on, stones were placed on the graves of all Jews buried in Jerusalem, and from Jerusalem the custom spread, and today Jews all over the world place stones on tombstones when visiting a grave.
This may not be the actual source of the custom, but it's an interesting story.
