Contrary to what some may believe, it is NOT permissible for a husband to rape his wife, although the term "rape" might not be used due to the presumption of mutual consent in marriage. But we start with the very first aya of Surat al-Nisa' (Women) 4:1:
O mankind, fear your Lord, who created you from a single soul and
created from it its mate, and dispersed from both of them countless
men and women. And fear Allah, through whom you demand your mutual
rights, and (reverence) the wombs that bore you: Indeed Allah is ever,
over you, an Observer.
Rape or forced sex harms the wife both physically and emotionally. This is contrary to how Allah wants us to conduct our lives. Fear of Allah in how one conducts one's relationships, especially in this case those of marriage (but also other types), is paramount. Applying this to sexual violence, does this seem like a God-fearing act? "Reverence the wombs" actually refers to upholding high standards in maintaining family relations generally, but at the same time, the womb specifically relates to women, and certainly rape disrespects the woman in whose trust the man has entered in marriage.
In 4:19,
O you who have believed, it is not lawful for you to inherit women by
compulsion. And do not make difficulties for them in order to take
[back] part of what you gave them unless they commit a clear
immorality. And live with them in kindness. For if you dislike them -
perhaps you dislike a thing and Allah makes therein much good.
To have sex by force is not kindness. The prophet also recommended to treat people in general with kindness and consideration, even family members who are not Muslim. So how could it be ok to harm one's wife?
In 4:21, referring to the husband taking unlawfully his wife's dower:
And how can you take it when you have become intimate with each other,
and they have taken from you a solemn covenant?
The "solemn covenant" here is in Arabic "meethaq ghaleeth" and refers to the marriage covenant itself, in which both parties pledge to treat each other with honor, respect, kindness, and consideration. The same expression is also used in reference to the covenant between Allah and bani Israel in the Quran, and their breaking of that covenant was the reason for the severe punishment of wandering in the wilderness. Such a covenant is binding in the strongest possible way, and in fact one could translate "ghaleeth" as "sacred" in the sense of "inviolable". In 2:187, the relationship of husband and wife is described as being each other's "garment." That is expressive of the ultimate trust. There is no relationship of trust that permits one of the two to cause such deliberate harm to the other, especially a part of the relationship wherein trust is given in the most vulnerable way. For the husband to violate his wife in this way would violate the sacred bond between them. She should in turn try to fulfill her part and not unreasonably deny him intimacy. And in the matter of unfaithfulness both parties are equally bound to comply, although only the husband has the right to marry multiple wives. But even that is with the caveat that he will be capable of complete fairness in dealing with them.
Should the situation of the marriage get worse between them they should seek an intermediary or some kind of legal or other mediation rather than resorting to cruelty.
From the scholar Taqi al-Din al-Subki:
At the time when it becomes obligatory for a husband to provide
financial support, clothing, (and other such provisions) for his wife,
he should exert himself in doing so, and not be negligent in this duty
such that his wife would have to file a complaint of his negligence
with the judge [haakim], and in so doing spend from her own
expenditures. Similarly, a wife should be responsive to her husband’s
request for intimacy, such that he would not need to bring a complaint
(against her) to the judge, and in so doing spend from his own
expenditures.
Here even if the wife refused intimacy (and we presume this to be on multiple occasions where it became an issue), the resolution was via a judge, just as would be a complaint by the wife against her husband. The resolution is not by force or violence, and rape is an act of violence. So no, sexual violence is not permissible in Islam by a husband against his wife.