Matthew 2:1-11: "1 Now after Jesus was born in Bethlehem of Judea in the days of Herod the king, behold, wise men from the east came to Jerusalem, 2 saying, 'Where is he who has been born king of the Jews? For we saw his star when it rose and have come to worship him.' 3 When Herod the king heard this, he was troubled, and all Jerusalem with him; 4 and assembling all the chief priests and scribes of the people, he inquired of them where the Christ was to be born. 5 They told him, 'In Bethlehem of Judea, for so it is written by the prophet: 6 "And you, O Bethlehem, in the land of Judah, are by no means least among the rulers of Judah; for from you shall come a ruler who will shepherd my people Israel."'
"7 Then Herod summoned the wise men secretly and ascertained from them what time the star had appeared. 8 And he sent them to Bethlehem, saying, 'Go and search diligently for the child, and when you have found him, bring me word, that I too may come and worship him.' 9 After listening to the king, they went on their way. And behold, the star that they had seen when it rose went before them until it came to rest over the place where the child was. 10 When they saw the star, they rejoiced exceedingly with great joy. 11 And going into the house they saw the child with Mary his mother, and they fell down and worshiped him. Then, opening their treasures, they offered him gifts, gold and frankincense and myrrh."
Isaiah 60:19: "The sun shall be no more your light be day, nor for brightness shall the moon give you light; but the LORD will be your everlasting light, and your God will be your glory."
Revelation 21:23: "And the city has no need of sun or moon to shine on it, for the glory of God gives it light, and its lamp is the Lamb."
+++
The text is public domain:
As with gladness men of old
Did the guiding star behold;
As with joy they hailed its light,
Leading onward, beaming bright;
So, most gracious Lord, may we
Evermore be led by Thee.
As with joyful steps they sped,
Savior, to Thy lowly bed,
There to bend the knee before
Thee, whom heav'n and earth adore;
So may we with willing feet
Ever seek Thy mercy seat.
As they offered gifts more rare
At Thy cradle, rude and bare,
So may we with holy joy,
Pure and free from sin's alloy,
All our costliest treasures bring,
Christ, to Thee, our heav'nly King.
Holy Jesus, ev'ry day
Keep us in the narrow way;
And when earthly things are past,
Bring our ransomed souls at last
Where they need no star to guide,
Where no clouds Thy glory hide.
In the heav'nly country bright
Need they no created light;
Thou its light, its joy, its crown,
Thou its sun which goes not down;
There forever may we sing
Alleluias to our King.
+++
The Biblical sources for this hymn are straight forward. The text from Matthew appears at the beginning of each the first three verses. First, the wise men's actions are described, and then each verse goes on to say, "So may we" follow their example.
The texts from Isaiah 60 and Revelation 21 contain the same idea of God as our light rather than the sun or moon. In the hymn, this is primarily in the fifth verse, but it's also hinted at in the fourth verse (where this old light is connected to the star that led the wise men).