Gaza, part of the biblical "land of the Philistines" was originally promised to the Jewish nation who had been freed from Egypt and wandered in the desert for 40 years. It was intended to be part of the region owned by Judah. However, Gaza was one of the regions that Joshua did not conquer. The verse prophetically states "they will be a snare and an obstical to you, a lash in your sides and thorns in your eyes."
Gaza was the place where Samson toppled the idolatrous Temple of Dagon on his enemies, giving his life in the process. Gaza was first conquered for Israel by King David, and subsequently fell to the Assyrians, the Egyptians, the Babylonians, Persians and Greeks. But in the days of King David and throughout the first Temple period it was part of Israel. In Jesus' day, it was part of the Hasmonean Kingdom that ended with Herod the Great. For nearly 2000 years it was conquered by numerous powers.
By the way, in Hebrew the word is pronounced "Aza", without a G.