Psalm 119:5: "Oh that my ways may be steadfast in keeping your statutes!"
Psalm 119:33: "Teach me, O LORD, the way of your statutes; and I will keep it to the end."
Psalm 119:133: "Keep steady my steps according to your promise, and let no iniquity get dominion over me."
Psalm 119:176: "I have gone astray like a lost sheep; seek your servant, for I do not forget your commandments."
There are only ten chapters in Ezra. I think the citation should be Ezekiel 34:11-12: "11 'For thus says the Lord GOD: Behold, I, I myself will search for my sheep and will seek them out. 12 As a shepherd seeks out his flock when he is among his sheep that have been scattered, so will I seek out my sheep, and I will rescue them from all places where they have been scattered on a day of clouds and thick darkness.'"
Matthew 18:12-14: "12 'What do you think? If a man has a hundred sheep, and one of them has gone astray, does he not leave the ninety-nine on the mountains and go in search of the one that went astray? 13 And if he finds it, truly, I say to you, he rejoices over it more than over the ninety-nine that never went astray. 14 So it is not the will of my Father who is in heaven that one of these little ones should perish.'"
+++
The text is public domain:
1 Oh, that the Lord would guide my waysTo keep His statutes still!Oh, that my God would grant me graceTo know and do His will!2 Order my footsteps by Thy WordAnd make my heart sincere;Let sin have no dominion, Lord,But keep my conscience clear.3 Assist my soul, too apt to stray,A stricter watch to keep;And should I e'er forget Thy way,Restore Thy wand'ring sheep.4 Make me to walk in Thy commands -'Tis a delightful road -Nor let my head or heart or handsOffend against my God.
+++
Verses 5 and 33 from Psalm 119 seem to be combined in the hymn's first verse, although the specific phrasing bears more resemblance to verse 5, and Psalm 119:133 is paraphrased in the hymn's second and fourth verses (more closely in the second than the fourth).
The other three cited texts (Psalm 119:176, Ezekiel 34:11-12, and Matthew 18:12-14) appear in the third verse, particularly in the line "Restore Thy wand'ring sheep."