Is relationship and sex between a muslim woman and non muslim man permitted in Islam?
Short answer: NO!
and many other verses regarding this. So sex is only allowed under marriage.
Basically Muslim are not permitted to marry a polytheist nor an adulterer, otherwise he/she will be considered as the same.
God made this world to test us. The real life is the next one. Being a test, you are given a free will to choose whatever you want. Whatever you choose, will have consequences in this world and the next.
As for sexual relation before marriage, please consider that fact that human being is one the weakest creature at his birth. He need the love of mother and care of father for a long time from his birth. This can only be provided in a family. Thus the institution of family is one of the basic needs for the survival of human race. Allah recognizes this fundamental need and thus given us instruction to uphold the sanctity of this institution. One of it that sexual relations are only allowed within marriage.
As for marrying a non-muslim, please consider that within a family, the man has the upper hand. it is observable across cultural and ethnic boundaries. Thus you will be putting yourself as well as your children in a difficult position, if he tries to force you out of it. You can say that he is not like that but, believe me, things CHANGE after marriage.
If he choose to embrace Islam, before marriage, then, hopefully, you (and your children) would be in a better position with regard to religion.