From the view of a secretely following muslim , if he find about mistakes and feel depressed as he was not following religion well and he decided to leave ISLAM (AUZUBILLAH) or decide to commit suicide . What is the adequate solution/ruling for it?
It really doesn't make a big difference whether you follow Islam secretly or not when it comes to act of worship. Unless you really are afraid of being harmed or even killed by non-Muslims if caught.
When your heart is clearly following Islam and inside of you, you reject kufr or shirk then that should be acceptable even if you were in situation where shirk is involved, but Allah only knows what is in your heart. As Allah () says:
Whoever disbelieves in Allah after his belief... except for one who is forced [to renounce his religion] while his heart is secure in faith. But those who [willingly] open their breasts to disbelief, upon them is wrath from Allah , and for them is a great punishment; (16:106)
One shouldn't get desperate if in the beginning of his life as a Muslim he/she may commit some or even many mistakes, but rather learn from them and try his/her best to avoid them in future.
The point is one shouldn't commit knowingly a mistake or sin without a clear necessity. The verse above speaks about people who where "forced" to commit shirk. But be aware that it is always best to return to Allah by repentance and keep remembering Allah (dhikr), to avoid the chance of dying while performing an act of shirk and to avoid a fall-back into shirk or kufr.
And killing oneself would be a big mistake!
The Prophet (ﷺ) said, "Whoever purposely throws himself from a mountain and kills himself, will be in the (Hell) Fire falling down into it and abiding therein perpetually forever; and whoever drinks poison and kills himself with it, he will be carrying his poison in his hand and drinking it in the (Hell) Fire wherein he will abide eternally forever; and whoever kills himself with an iron weapon, will be carrying that weapon in his hand and stabbing his 'Abdomen with it in the (Hell) Fire wherein he will abide eternally forever." (Sahih al-Bukhari)
"Whoever swears by a religion other than Islam (i.e. if somebody swears by saying that he is a non-Muslim e.g., a Jew or a Christian, etc.) in case he is telling a lie, he is really so if his oath is false, and a person is not bound to fulfill a vow about a thing which he does not possess. And if somebody commits suicide with anything in this world, he will be tortured with that very thing on the Day of Resurrection; And if somebody curses a believer, then his sin will be as if he murdered him; And whoever accuses a believer of Kufr (disbelief), then it is as if he killed him." (Sahih al-Bukhari)
On the other hand dunya was made easy for the kafir and hard for the Muslim (see What does the hadith about Paradise surrounded with hardships and Jahannam surrounded by desires mean?) (Simply speaking a Muslim must fight and retain his desires and dunya is full of them a kafir can easily let them go), but the Muslim might have a bonus:
The Prophet (ﷺ) said, "No fatigue, nor disease, nor sorrow, nor sadness, nor hurt, nor distress befalls a Muslim, even if it were the prick he receives from a thorn, but that Allah expiates some of his sins for that." (Sahih al-Bukhari)
And Allah said:
... And do not kill yourselves [or one another]. Indeed, Allah is to you ever Merciful. (4:29)
By becoming a Muslim or the conversion one would have left a dark past and made a good step into light, so one shouldn't make his/her future even darker!
The whole situation of a secrete follower of Islam might be somewhat unique, so one shouldn't make it harder then it is. We may tell you what is right and what is wrong, but a secret follower of Islam should use his mind too while he should try to avoid what is wrong and try to follow what is right... he should also figure out what actually he is able to fulfil and if necessary one should also consider trying to find a good work around (whithout committing major sins or shirk etc.).
Note that scholars say if a Muslim commits apostasy and by any means fight the Muslims he should be killed (by them), else if he doesn't try to make any influence on the Muslims in his environment the matter is discussed, but it looks more like live and let live.