Psalm 115:1: "Not to us, O LORD, not to us, but to your name give glory, for the sake of your steadfast love and your faithfulness!"
Ephesians 3:21: "to him be glory in the church and in Christ Jesus throughout all generations, forever and ever. Amen."
Philippians 3:8-9: "8 Indeed, I count everything as loss because of the surpassing worth of knowing Christ Jesus my Lord. For his sake I have suffered the loss of all things and count them as rubbish, in order that I may gain Christ 9 and be found in him, not having a righteousness of my own that comes from the law, but that which comes through faith in Christ, the righteousness from God that depends on faith -"
Hebrews 11:13-16: "13 These all died in faith, not having received the things promised, but having seen them and greeted them from afar, and having acknowledged that they were strangers and exiles on the earth. 14 For people who speak this make it clear that they are seeking a homeland. 15 If they had been thinking of that land from which they had gone out, they would have had opportunity to return. 16 But as it is, they desire a better country, that is, a heavenly one. Therefore God is not ashamed to be called their God, for he has prepared for them a city."
+++
The verses from Psalm 115 and Ephesians 3 appear primarily in the hymn's first verse ("Not unto us, not unto us be glory, Lord; / Not unto us but to Your name be praise"), but they're also at the beginning of the last verse.
The verses from Philippians 3 seem more to inform the hymn than be directly quoted or even paraphrased in it. The hymn expresses "The surpassing worth of knowing Christ Jesus my Lord" in its second and third verses.
Hebrews 11 shows up in a few places in the hymn's third verse, which uses words like "shepherded" and "led" to describe God's direction of this journey of "strangers and exiles on the earth." It appears most clearly, however, in the final lines of the fourth verse: "Abide with us till trav'ling days are over and done, / And pilgrim feet lead us home, Lord, to You."