The Quran was written in a certain socio-cultural environment. In that environment and era, slavery was not only common but quite the norm. See Slavery article in Wikipedia for history.
In Arabia, before Islam, non-criminal sexual relationships between sexes comprised of three forms:
- Between husband and wife
- With slaves
- Out of wedlock and outside of slavery
When the Quran was being written, it "initially" rooted out sexual relationships that were not between spouses or with slaves. That's where one finds verses such as those quoted in the question:
Qur'an (23:5-6) - "...who abstain from sex, except with those
joined to them in the marriage bond, or (the captives) whom their
right hands possess..." This verse permits the slave-owner to have sex
with his slaves. See also Qur'an (70:29-30).
At the same time, the Quran exhorted, either as expiation of sins or as an act of piety, for Muslims to start freeing slaves. Thus, for example, act of piety:
"And what could make thee conceive what it is, that steep uphill road?
[It is] the freeing of a slave" (The Quran 90:12-13)
and expiation of sin:
"Hence, as for those who would separate themselves from their wives by
saying, "Thou art as unlawful to me as my mother," and thereafter
would go back on what they have said, [their atonement] shall be the
freeing of a slave before the couple may touch one
another again: this you are [hereby] exhorted to do - for God is fully
aware of all that you do." (The Quran 58:3)
Keep in mind that these are all Meccan surahs, i.e. an Islamic state had not yet been established.
We find that in the Medinan surah Al-Nur, the Quran commands that if slaves were to demand their freedom, then it be granted:
AND [you ought to] marry the single from among you as well as such of
your male and female slaves as are fit [for marriage]. If they
are poor, [let this not deter you;] God will
grant them sufficiency out of His bounty – for God is infinite [in His
mercy], all-knowing. (The Quran 24:32)
The clause "if they are poor" indicates that one of the issues in freeing all the slaves in one go was also related to their being able to support themselves, since they were dependents of their owners. Thus, we may concluse that the Quran was gradually ending the slavery, and once the slavery ended, the question of having sexual relationships with slaves does not remain.
The fact that the Quran commands in the verse above that the slave ought to be free may also be taken as an indication that slavery is not tolerable in Islam anymore. Thus Abdullah Yusuf Ali writes in his commentary (note #2992) for this verse:
Where slavery was legal, what is now called the "white slave traffic"
was carried on by wicked people like 'Abd Allah ibn Ubayy, the
Hypocrite leader at Madinah. This is absolutely condemned. While
modern nations have abolished ordinary slavery, the "White Slave
Traffic" is still a big social problem in individual States. Here it
is absolutely condemned. No more despicable trade can be imagined.
I can likewise provide opinions of other commentators but many are in non-English languages (usually Arabic and some in Urdu).