Psalm 137:1-6: "1 By the waters of Babylon, there we sat down and wept, when we remembered Zion. 2 On the willows there we hung up our lyres. 3 For there our captors required of us songs, and our tormentors, mirth, saying, 'Sing us one of the songs of Zion!' 4 How shall we sing the LORD's song in a foreign land? 5 If I forget you, O Jerusalem, let my right hand forget its skill! 6 Let my tongue stick to the roof of my mouth, if I do not remember you, if I do not set Jerusalem above my highest joy!"
Revelation 19:1-8: "1 After this I heard what seemed to be the loud voice of a great multitude in heaven, crying out, 'Hallelujah! Salvation and glory and power belong to our God, 2 for his judgements are true and just; for he has judged the great prostitute who corrupted the earth with her immortality, and has avenged on her the blood of his servants.'
"3 Once more they cried out, 'Hallelujah! The smoke from her goes up forever and ever.' 4 And the twenty-four elders and the four living creatures fell down and worshiped God who was seated on the throne, saying, 'Amen. Hallelujah!' 5 And from the throne came a voice saying, 'Praise our God, all you his servants, you who fear him, small and great.' 6 Then I heard what seemed to be the voice of a great multitude, like the roar of many waters and like the sound of mighty peals of thunder, crying out, 'Hallelujah! For the Lord our God the Almighty reigns. 7 Let us rejoice and exult and give him the glory, for the marriage of the Lamb has come, and his Bride has made herself ready; 8 it was granted her to clothe herself with fine linen, bright and pure' - for the fine linen is the righteous deeds of the saints."
+++
The text is public domain:
Alleluia, song of gladness,
Voice of joy that cannot die;
Alleluia is the anthem
Ever raised by choirs on high;
In the house of God abiding
Thus they sing eternally.
Alleluia, thou resoundest,
True Jerusalem and free;
Alleluia, joyful mother,
All thy children sing with thee,
But by Babylon's sad waters
Mourning exiles now are we.
Alleluia cannot always
Be our song while here below;
Alleluia, our transgressions
Make us for a while forgo;
For the solemn time is coming
When our tears for sin must flow.
Therefore in our hymns we pray Thee,
Grant us, blessed Trinity,
At the last to keep Thine Easter
With Thy faithful saints on high;
There to Thee forever singing
Alleluia joyfully.
+++
The Biblical sources for this one are easily distinguished. The main source for the hymn is the text from Revelation, specifically the parts about singing "Hallelujah," which appear in phrases like "Alleluia is the anthem / Ever raised by choirs on high" and "There to Thee forever singing / Alleluia joyfully." The last two lines of the second verse ("But by Babylon's sad waters / Mourning exiles now are we") are from Psalm 137, primarily verse 1, but the "exiles" part comes from "in a foreign land" in verse 4.