Usually before I finish my prayer by doing salam (and after allahumma salee ala muhammad) I say this dua:
Rabbana Atina Fid Dunya Hasanatan Wa Fil Akhirati Hasanatan Waqina Azabannar
رَبَّنَا آتِنَا فِي الدُّنْيَا حَسَنَةً وَفِي الآخِرَةِ حَسَنَةً وَقِنَا عَذَابَ النَّارِ
After that I make salam and then my prayer is ended.
Is this dua meant to be said this way in prayer?
In this video, we see that he says the dua: https://www.youtube.com/watch?v=QC28bKBlb5I&t=198s
There are some things that are recommended to say after you finish your Tashahhud and Salawat to the Prophet. Some of the duas are:
اللَّهُمَّ إِنِّي أَعُوذُ بِكَ مِنْ عَذَابِ جَهَنَّمَ وَمِنْ عَذَابِ الْقَبْرِ وَمِنْ فِتْنَةِ الْمَحْيَا وَالْمَمَاتِ وَمِنْ شَرِّ فِتْنَةِ الْمَسِيحِ الدَّجَّالِ
Meaning: O Allaah, I seek refuge with You from the punishment of Hell and the punishment of the grave, and from the trials of life and death and from the worst of the turmoil of the Dajjaal.
اللَّهُمَّ إِنِّي أَعُوذُ بِكَ مِنْ عَذَابِ الْقَبْرِ وَأَعُوذُ بِكَ مِنْ فِتْنَةِ الْمَسِيحِ الدَّجَّالِ وَأَعُوذُ بِكَ مِنْ فِتْنَةِ الْمَحْيَا وَفِتْنَةِ الْمَمَاتِ اللَّهُمَّ إِنِّي أَعُوذُ بِكَ مِنَ الْمَأْثَمِ وَالْمَغْرَمِ
Meaning: O Allaah, I seek refuge with You from the punishment of the grave and I seek refuge with You from the turmoil of the Dajjaal and I seek refuge with You from the trials of life and the trials of death. O Allaah, I seek refuge with You from sin and heavy debt.
اللَّهُمَّ إِنِّي ظَلَمْتُ نَفْسِي ظُلْمًا كَثِيرًا وَلا يَغْفِرُ الذُّنُوبَ إِلا أَنْتَ فَاغْفِرْ لِي مَغْفِرَةً مِنْ عِنْدِكَ وَارْحَمْنِي إِنَّك أَنْتَ الْغَفُورُ الرَّحِيمُ
Meaning: O Allaah, indeed I have wronged myself greatly, and there is none who forgives sins besides You. So grant me forgiveness from You and have mercy on me, for You are the All-Forgiving, Most Merciful
The first two duas have overlap, so you can probably combine them. It is also allowed to pray for yourself anything you wish after the Tashahhud and Salawat. So, there is nothing wrong in you making the dua you mentioned after seeking refuge with the first two duas.
For more information you could read this by IslamQA.
A simple answer is: No this du'a is not (exclusively) meant (or more correctly reported) to be said within the prayer at any specific act of this worship.
It is among the du'a's the prophet used to recite the most as reported by some sahabah (especially Anas ibn Malik):
Anas (RAA) narrated, “The Messenger of Allah (ﷺ) used to supplicate frequently: “O our Lord, grants us the best in this life and the best in the next life, and protects us from the punishment of the Fire.” Agreed upon.
(See in ibn Hajar's Bulugh al-Maram and also other sources of this du'a in sunnah.com)
Reports that actually make a link between this du'a and a specific worship rather speak of hajj or 'umrah or worshiping inside or clsoe to the Ka'abah.
As this is a du'a that the prophet used to recite it is certainly not wrong to recite it during the prayer or afterwards, but it would be better to recite it while prostrating.
