First of all, you have to understand the nuances of translation. Hebrew has no adjective “sinful”. The verse says that David was “formed in sin” and his mother “conceived him in iniquity”. We explain that to mean that the inclination toward evil is implanted into a person from his very formation, and also potentially that there was some sin involved on David’s mother’s part when she conceived him. In the context of the Psalm, he is begging God for mercy in light of his having fallen prey to the inborn, evil inclination. If he were born sinful (with guilt), why would that be relevant to being forgiven for the sin of Bathsheba? It’s a non sequitur.
If you use the Christian translation you just cited, it kind of follows, because it goes on to discuss “you desired faithfulness even in the womb,” advancing the Christian narrative that faith saves from original sin. But the problem is, that’s not what the verse says. It says: הן-אמת חפצת בטוחות - Behold, You have desired truth in the “toohot”. Toohot is a feminine plural noun. It cannot mean womb, as there is only one womb. The fact that it actually means “kidneys,” (there are two!) which were considered to be a seat of wisdom in the body, is proven by the verse: Job 38:36 “Who placed widsom in the toohot, and who granted understanding to the heart?”
King James gives the following translation: “Who hath put wisdom in the inward parts? or who hath given understanding to the heart?”
Translators have difficulty with this word and just write “inward parts” since it’s only written in a couple of places in the Bible, but the Talmud asserts that it means kidneys.
The 1985 Jewish Publication Society declines to translate it as kidneys and instead uses it as a poetic term for something hidden: “Indeed You desire truth about that which is hidden; teach me wisdom about secret things.”