1 Corinthians 11:23-29: "23 For I received from the Lord what I also delivered to you, that the Lord Jesus on the night when he was betrayed took bread, 24 and when he had given thanks, he broke it, and said, 'This is my body which is for you. Do this in remembrance of me.' 25 In the same way also he took the cup, after supper, saying, 'This cup is the new covenant in my blood. Do this, as often as you drink it, in remembrance of me.' 26 For as often as you eat this bread and drink the cup, you proclaim the Lord's death until he comes.
"27 Whoever, therefore, eats the bread or drinks the cup of the Lord in an unworthy manner will be guilty concerning the body and blood of the Lord. 28 Let a person examine himself, then, and so eat of the bread and drink of the cup. 29 For anyone who eats and drinks without discerning the body eats and drinks judgement on himself."
Psalm 34:8: "Oh, taste and see that the LORD is good! Blessed is the man who takes refuge in him!"
1 Peter 2:2-3: "2 Like newborn infants, long for the pure spiritual milk, that by it you may grow up into salvation - 3 if indeed you have tasted that the Lord is good."
+++
The passage from 1 Corinthians 11 is the main source and appears throughout the hymn. The other two passages are at the end of each verse; the last lines all begin with "O taste and see...."
Some lines in the third verse ("My God, my God, / Why have You not forsaken me?") are clearly patterned on the first part of Psalm 22:1: "My God, my God, why have you forsaken me?"