Like any other contentious issue between the sects, this is not an easy question. Each opinion has sources for their preferred way.
But like Muhammad Naveed Ashiq mentioned Shias rely on traditions from descendants of Rasul Allah (as) for their beliefs and practices. These start from Ali ibn Abi Talib, a close companion and family member, going down to Jaffar as-Sadiq who was a reputed scholar in his own time.
Sunnis on the other hand rely on traditions narrated from companions and some of his wives (mostly Aisha), largely to the exclusion of the family of the Prophet from Banu Hashim, like Ali, Fatima, Hassan and Hussain and those companions that were very close to Banu Hashim like Salmaan and Abu Dhar. Only one Hashimi companion features prominently in Sunni hadiths and that would be Ibn Abbas who enjoyed good relations with the other companions.
So ultimately it is on you whether you rely on traditions that have been mostly transmitted by the ruling companions after the Prophet or those related by members of Banu Hashim who were mostly marginalized after the death of the Prophet (pbuh). And this doesn't influence just the way you pray but a whole lot of other contentious issues between the sects. It all goes down to which hadith sources inform your beliefs. And these sources are sharply divided on many key questions, whether it is belief, law or history!