'In sin my mother conceived me'

Rashi reads the verse as the psalmist asking how he could avoid sin, since the main part of his creation came through coitus, a source of many iniquities. Alternatively, he was formed from a male and a female, both full of iniquity. Rashi notes that many midrashim do not fit the psalm’s context.

Rashi (R. Shlomo Yitzhaki, 1040-1105); English: The Judaica Press complete Tanach with Rashi, trans. A. J. Rosenberg, Rashi on Psalms (Judaica Press translation, Sefaria) — Rashi on Psalm 51:7 (Hebrew numbering; 'Behold, with iniquity I was formed' = English 51:5)

Augustine says that being conceived in iniquity means iniquity is drawn from Adam. The bond of death, with iniquity itself, is ingrained, so no one is born without bringing punishment and the desert of punishment.

Augustine of Hippo (354-430), Expositions on the Book of Psalms (Oxford 'Library of the Fathers' translation, condensed by A. Cleveland Coxe), in Nicene and Post-Nicene Fathers, ser. 1, vol. 8 (New York: Christian Literature Publishing Co., 1888) — Exposition of Psalm 51, §10 (on verse 5)

The Council of Trent states that Adam’s prevarication injured not himself alone but his posterity also. The holiness and justice he lost, he lost for us too; being defiled by disobedience, he transmitted not only death and bodily pains but sin also, the death of the soul, to the whole human race.

The Council of Trent (Session V, 17 June 1546), Decree concerning Original Sin, Session V (ed. and trans. J. Waterworth, London: Dolman, 1848) — Session V, Canon 2 (on the transmission of Adam's sin)

The Synod of Jerusalem teaches that the first man created by God fell in Paradise by disregarding the divine commandment and yielding to the serpent’s deceitful counsel. As a result, hereditary sin flowed to his posterity, so that no one born after the flesh is without this burden or its fruits in this present world.

The Synod of Jerusalem (1672), Confession of Dositheus, The Acts and Decrees of the Synod of Jerusalem (trans. J. N. W. B. Robertson, London: Baker, 1899) — Confession of Dositheus, Decree VI (on the fall and its inheritance)

Calvin reads Psalm 51:5 as a striking testimony to original sin entailed by Adam upon the whole human family. Against the Pelagians’ claim that sin descended from Adam only by imitation, he says the Bible clearly asserts we are born in sin and that it exists within us as a disease fixed in our nature.

John Calvin (1509-1564), Commentary on the Book of Psalms, vol. 2 (trans. James Anderson, Calvin Translation Society, 1846) — Commentary on Psalm 51:5 ('Behold, I was born in iniquity')

Wesley says this is not the only sin he has reason to bewail; it is a filthy stream leading him to a corrupt fountain. Reviewing his heart, he finds this heinous crime was the proper fruit of his vile nature, which ever was and still is ready to commit ten thousand sins as occasion offers.

John Wesley (1703-1791), Explanatory Notes upon the Old Testament (1765; CCEL's 'Wesley's Notes on the Bible') — Note on Psalm 51:5

Albert Barnes holds that the general meaning of Romans 5:12 is not difficult. Its difficulties come from attaching it to a philosophical theory about human sin and from trying to prove it teaches what it was never designed to teach.

Albert Barnes (1798-1870), Notes on the New Testament, Explanatory and Practical (enlarged ed. by Robert Frew; CCEL carries the Baker reprint) — Note on Romans 5:12 (introduction to verses 12-21)