The shoe symbolizes the physical body. Just as the shoe encases the lowest part of the body and allows it to ambulate in the world, so too the body encases the lowest level of the soul and allows it to ambulate and relate to the physical world.
Therefore, whenever God wants a person to relate on a totally spiritual level, ignoring the body, He commands him to remove his shoes. This was true when God spoke to Moses and to Joshua. It was true for the kohanim priests in the Temples in Jerusalem. And it is true for every Jew on Yom Kippur.
We ignore the physical aspects of our existence for one day a year, and to symbolize this we remove our leather shoes. Leather specifically, because it encompassed a living creature and hence symbolizes the body in a much more graphic way than other materials.
