Psalm 96:1: "Oh sing to the LORD a new song; sing to the LORD, all the earth!"
Psalm 149:1-3: "1 Praise the LORD! Sing to the LORD a new song, his praise in the assembly of the godly! 2 Let Israel be glad in his Maker; let the children of Zion rejoice in their King! 3 Let them praise his name with dancing, making melody to him with tambourine and lyre!"
Isaiah 42:10-12: "10 Sing to the LORD a new song, his praise from the end of the earth, you who go down to the sea, and all that fills it, the coastlands and their inhabitants. 11 Let the desert and its cities lift up their voice, the villages that Kedar inhabits; let the habitants of Sela sing for joy, let them shout from the top of the mountains. 12 Let them give glory to the LORD, and declare his praise in the coastlands."
Exodus 15:1-2: "1 Then Moses and the people of Israel sang this song to the LORD, saying, 'I will sing to the LORD, for he has triumphed gloriously; the horse and his rider he has thrown into the sea. 2 The LORD is my strength and my song, and he has become my salvation; this is my God, and I will praise him, my father's God, and I will exalt him.'"
+++
All of the cited passages mention singing to the Lord, and this is present throughout the hymn in the recurring line "Sing to the Lord a new song!" (taken verbatim from any of the first three citations). The passage from Exodus 15 may also be alluded to in the first line of the refrain: "He has done marvelous things."
Psalm 148 seems to be an important source, too. Similar to the hymn, it's a rhetorical catalogue of various entities who are encouraged to "Praise the LORD!" The strongest resemblance is between the hymn's title line ("Earth and all stars!") and Psalm 148:3: "Praise him, sun and moon, praise him, all you shining stars!"
