Isaiah 53:3-6: "3 He was despised and rejected by men; a man of sorrows, and acquainted with grief; and as one from whom men hide their faces he was despised, and we esteemed him not. 4 Surely he has borne our griefs and carried our sorrows; yet we esteemed him stricken, smitten by God, and afflicted. 5 But he was wounded for our transgressions; he was crushed for our iniquities; upon him was the chastisement that brought us peace, and with his stripes we are healed. 6 All we like sheep have gone astray; we have turned - every one - to his own way; and the LORD has laid on him the iniquity of us all."
Acts 4:11-12: "11 This Jesus is the stone that was rejected by you, the builders, which has become the cornerstone. 12 And there is salvation in no one else, for there is no other name under heaven given among men by which we must be saved."
Romans 4:25: "[Jesus our Lord], who was delivered up for our trespasses and raised for our justification."
+++
The text is public domain:
Stricken, smitten, and afflicted,
See Him dying on the tree!
'Tis the Christ, by man rejected;
Yes, my soul, 'tis He, 'tis He!
'Tis the long-expected Prophet,
David's Son, yet David's Lord;
Proofs I see sufficient of it:
'Tis the true and faithful Word.
Tell me, ye who hear Him groaning,
Was there ever grief like His?
Friends through fear His cause disowning,
Foes insulting His distress;
Many hands were raised to wound Him,
None would intervene to save;
But the deepest stroke that pierced Him
Was the stroke that justice gave.
Ye who think of sin but lightly
Nor suppose the evil great
Here may view its nature rightly,
Here its guilt may estimate.
Mark the sacrifice appointed,
See who bears the awful load;
'Tis the Word, the Lord's anointed,
Son of Man and Son of God.
Here we have a firm foundation,
Here the refuge of the lost:
Christ, the Rock of our salvation,
Is the name of which we boast;
Lamb of God, for sinners wounded,
Sacrifice to cancel guilt!
None shall ever be confounded
Who on Him their hope have built.
+++
The text from Isaiah 53 is the main source for the hymn (the first line is straight from verse 4), but the verse from Romans is similar enough that it's difficult to determine whether some parts of the hymn (like "the deepest stroke that pierced Him / Was the stroke that justice gave" and "Mark the sacrifice appointed, / See who bears the awful load") are from Romans or from Isaiah.
The verses from Acts 4 appear in the last verse, specifically in phrases like "a firm foundation," "Christ, the Rock of our salvation," and "on Him their hope have built."
"David's Son, yet David's Lord" in the first verse is from the Gospels. I first found it in Mark 12:35-37: "35 And as Jesus taught in the temple, he said, 'How can the scribes say that the Christ is the son of David? 36 David himself, in the Holy Spirit, declared, "The Lord said to my Lord, Sit at my right hand, until I put your enemies under your feet." 37 David himself calls him Lord. So how is he his son?' And the great throng heard him gladly." There are parallel accounts in Matthew 22:41-46 and Luke 20:41-44.