The Torah tells us that two goats had to be taken and set aside for the Yom Kippur service (ויקרא ט"ז:ה'). According to the Mishnah in Yoma (דף ס"ב.), these two goats were supposed to be of the same appearance, the same height, and the same value. The Abarbanel says that these two goats of Yom Kippur symbolize the struggle between Yaakov and Eisav. Like the goats, Yaakov and Eisav were once similar, but later in life, they moved in opposite directions, always struggling against one another. Similarly, the descendants of Yaakov and of Eisav continue to struggle against each other to this very day. The goats of Yom Kippur symbolically represent this struggle, and can be understood as the focal point of a prayer to Hashem that this bitter struggle should come to an end. One of the goats is pushed off a cliff called Azazel, which symbolizes the destruction of Eisav and of the evil which he represents; that goat is removed from the presence of Hashem. The other goat goes to Hashem, thus symbolizing that a good person has the capacity to get close to Hashem, as Yaakov did. The key lesson, though, is that people can come from the exact same background and still end up at very different destinations.
The Kli Yakar says that the two goats are not representing the struggle between Yaakov and Eisav, but are rather representing the two goats that Yaakov brought to his father, Yitzchak, when he got the Berachos which were originally intended for Eisav. From this Beracha, Jewish history began, because once Yaakov received this Beracha, he was able to become the father of the Jewish people. The Kli Yakar therefore says that just as the two goats brought by Yaakov brought about a great result for Yaakov and the Jewish people, so too should the two goats brought by the Kohein Gadol on Yom Kippur bring about a great result for the Kohein Gadol and for the Jewish people, changing history for the better.