What you saw is a widespread and fascinating custom called “Tashlich”. Tashlich is a Hebrew word that means “cast off”. During this brief prayer service we symbolically “cast off” our sins into the body of water. Some communities have the custom to throw crumbs into the water and shake empty their pockets there to symbolize the "casting off" of sins.
It is a custom that has roots in Jewish history and is based on a Midrashic story about Abraham our ancestor. The Tashlich ceremony is usually performed on the afternoon of the first day of Rosh Hashana. However, when the first day of Rosh Hashana is on Shabbat it is postponed until the second day of Rosh Hashana due to the laws of Shabbat. The prayers that are recited are found in the traditional Rosh Hashana prayerbook, called the Machzor.
The traditional service for tashlich is recited individually and includes the prayer "Who is like unto you, O God...and You will cast all their sins into the depths of the sea", and Biblical verses including Isaiah 11:9 ("They will not injure nor destroy in all My holy mountain, for the earth shall be as full of the knowledge of the Lord as the waters cover the sea") and Psalms 118:5-9, 121 and 130, as well as personal prayers. Although it is a service that may be performed by the individual and does not require a minyan or quorum, most communities walk together in the afternoon of Rosh Hashana for tashlich. Tashlich has remained a solemn tradition throughout history.
