Psalm 31:1-3: "1 In you, O LORD, do I take refuge; let me never be put to shame; in your righteousness deliver me! 2 Incline your ear to me; rescue me speedily! Be a rock of refuge for me, a strong fortress to save me!
"3 For you are my rock and my fortress; and for your name's sake you lead me and guide me"
Ephesians 6:10-17: "10 Finally, be strong in the Lord and in the strength of his might. 11 Put on the whole armor of God, that you may be able to stand against the schemes of the devil. 12 For we do not wrestle against flesh and blood, but against the rulers, against the authorities, against the cosmic powers over this present darkness, against the spiritual forces of evil in the heavenly places. 13 Therefore take up the whole armor of God, that you may be able to withstand in the evil day, and having done all, to stand firm. 14 Stand therefore, having fastened on the belt of truth, and having put on the breastplate of righteousness, 15 and, as shoes for your feet, having put on the readiness given by the gospel of peace. 16 In all circumstances take up the shield of faith, with which you can extinguish all the flaming darts of the evil one; 17 and take the helmet of salvation, and the sword of the Spirit, which is the word of God"
Psalm 55:22: "Cast your burden on the LORD, and he will sustain you; he will never permit the righteous to be moved."
Isaiah 46:4: "even to your old age I am he, and to gray hairs I will carry you. I have made, and I will bear; I will carry and will save."
+++
The text is public domain:
Triune God, be Thou our stay;As the hymnal notes, "the hymn may be sung once" with "Triune God, be Thou our stay" as the first line or "it may be sung three times," each time with a different first line: "God the Father, be our stay;" "Jesus Christ, be Thou our stay;" and "Holy Spirit, be our stay;"
O let us perish never!
Cleanse us from our sins, we pray,
And grant us life forever.
Keep us from the evil one;
Uphold our faith most holy,
And let us trust Thee solely
With humble hearts and lowly.
Let us put God's armor on,
With all true Christians running
Our heav'nly race and shunning
The devil's wiles and cunning.
Amen, amen! This be done;
So sing we, "Alleluia!" [Lent: O Lord, have mercy on us.]
+++
Psalm 31:1-3, Psalm 55:22, and Isaiah 46:4 all provide reasons to trust in God, which is central to the hymn ("And let us trust Thee solely"), but they seem to be incorporated more generally than specifically.
The passage from Ephesians 6 appears in the line "Let us put God's armor on."
"With all true Christians running / Our heav'nly race" seems to come from Hebrews 12:1: "Therefore, since we are surrounded by so great a cloud of witnesses, let us also lay aside every weight, and sin which clings so closely, and let us run with endurance the race that is set before us."