I know in Islamic state blasphemy, mockery etc is punishable by death.The state will carry out the execution. But what about people doing these things in non Islamic state counties? If a Muslim living in non Islamic countries and executed the person who committed blasphemy or mockery against our religion and prophet. What happens to him. Did he commit a sin or not or its good deed?
Fatwa clearly states that
Only a Muslim ruler or his deputy is entitled to execute the Hudud
If a Muslim ruler is not available, Hudud must not be executed.
Hadid on actions of the Prophet ﷺ does not apply as he was the ruler. He did not permit execution made by anybody on his own decision. A normal muslim is not allowed to kill a person who blasphemes - and even less is it an obligation to do so.
You were asking whether it is a sin:
Transgression of a rule is usually called a sin, so it is a sin.
Penalty may also be applied in non muslim countries. It is also under stately control.
Blasphemy is also under punishment under Jewish and Christian law. Jewish law provides stoning on blasphemy however it has to be executed by the whole community (and is not applied in Israel); Christian law does not prescribe a particular punishment on earth (however not forgiven by God) but in most Christian states a punishment is possible. The rest will be left to God.