TL;DR: The Qur'an instructs us to invite people to Islam on the condition that we do it "in the best way" (Qur'an 16:125), which tafsir interpret to include to avoid arguing (in the sense of debating); ahadith also state that the Prophet discouraged arguing. We're only responsible for conveying the message of Islam clearly (Qur'an 16:82). The Qur'an instead encourages coexisting with non-Muslims (who don't fight Islam).
Practically, actions speak louder than words. In my opinion, doing good deeds (as instructed throughout the Qur'an) is far more persuasive (and useful) than debating.
We are instructed to invite people to Islam:
Invite to the way of your Lord with wisdom and good instruction, and argue with them in a way that is best. Indeed, your Lord is most knowing of who has strayed from His way, and He is most knowing of who is [rightly] guided. -- Qur'an 16:125
which we need to do "in a way that is best":
"Best manner" implies that one should have a sweet tongue, show noble character and give reasonable and appealing arguments, and refrain from indulging in polemics, argumentation and controversies. -- Maududi Tafsir
(See also Qur'an 29:46 concerning People of the Scripture.)
After that, if someone rejects Islam, then that's not our responsibility:
But if they turn away, [O Muhammad] - then only upon you is [responsibility for] clear notification. -- Qur'an 16:82
(And also "And no bearer of burdens will bear the burden of another. ..." -- Qur'an 35:18.)
The Prophet discouraged arguing:
...I guarantee a house in Jannah for one who gives up arguing, even if he is in the right... -- Riyad as-Salihin (sunnah.com), Grade: Sahih according to An-Nawawi (source: Daily Hadith)
Other hadith against arguing available listed by Abu Amina Elias, Arguing about the religion in Islam, 2015, where arguing is said to cause the heart to harden and breeds hatred.
We don't decide who Allah guides:
Indeed, [O Muhammad], you do not guide whom you like, but Allah guides whom He wills. And He is most knowing of the [rightly] guided. -- Qur'an 28:56
and also Qur'an 2:256, Qur'an 10:41-43, Qur'an 18:29. From here we have two points:
even the Prophet was incapable of persuading all people that Islam is true (even after they witness miracles), so we need to have realistic expectations, and
who's guided and who's not guided is up to Allah, not us.
We should coexist with non-Muslims:
Say, "O disbelievers, I do not worship what you worship. Nor are you worshippers of what I worship. Nor will I be a worshipper of what you worship. Nor will you be worshippers of what I worship. For you is your religion, and for me is my religion." -- Qur'an 109
It seems reasonable to interpret this as an instruction to seek peaceful coexistence with non-Muslims, and to accept a difference of opinion. Tafsir (e.g. Sayyid Abul Ala Maududi) point out this includes those committing shirk, the worst possible sin.
Likewise we have:
Allah does not forbid you from those who do not fight you because of religion and do not expel you from your homes - from being righteous toward them and acting justly toward them. Indeed, Allah loves those who act justly. -- Qur'an 60:8
Practical points based on my experiences (what was effective at converting me to Islam):
Debating can backfire. E.g. On the way to buy milk one day, some Christians stop me and talk about how I was building my [intellectual] house on sand. It left a lasting impression: "gee, Christians are arrogant and rude".
Actions are more persuasive than words. Doing good deeds is far more likely to invoke an "I want to be like that" response than words.
Debates, by and large, are won by people outwitting their opponent. Maybe I can't refute an argument off the top of my head, but that doesn't mean it's right. This hadith attests to that:
I am only a human being, and opponents come to me (to settle their problems); maybe someone amongst you can present his case more eloquently than the other, whereby I may consider him true and give a verdict in his favor. So, If I give the right of a Muslim to another by mistake, then it is really a portion of (Hell) Fire, he has the option to take or give up (before the Day of Resurrection). -- Sahih al-Bukhari 2458 (sunnah.com)
Some non-Muslims (particularly ex-Muslims) have greater knowledge about Islam than most Muslims. We might lose the debate---and then what?
As a former atheist, I prepared myself for debates with Christians. During this time, I "won" one argument against a Christian once (she seemed almost in tears afterwards). I experienced momentary satisfaction, followed by a lifetime of regret about hurting a fellow human being. (Sometimes, even when you win, you lose.) This is a possible consequence of engaging in debates.