Briefly speaking, let’s illustrate it by presenting a related hadith (tradition). It is said Prophet Muhammad (peace be upon him and his family) said (its approximate translation):
Whenever curse gets out of the mouth of the person who curses, it
sees, if it find a way towards the mentioned thing, it goes towards
that, otherwise it will return to the individual who has cursed.
As much as I research, I didn’t find if a non-Muslim …, but it can be inferred from the hadith (and other hadiths) that the curse won’t reach/annoy the person who doesn’t deserve of that curse. It means: No curse would be effective against you if you have not done any bad thing to be deserved that.
Let’s present another related hadith from Imam Muhammad al-Baqir (a.s.) as the fifth Imam of Shia Islam, who said:
Refrain from cursing Mo’men (believer), that it will come back to you
And so forth.