The Talmud explains that in order to know when the night has begun (which has tremendous significance as the Jewish day begins at night) there needs to be a simple visual method of measuring at which point it is definitely night-time. Three big stars can be seen too easily in the sky as it darkens and it may not yet be night. Three small stars are harder to spot and they mean that it is past the moment that it becomes night. Three medium stars, however, are small enough to indicate that the night has just fallen and that is why they are used.
