The full version of the hadith you mentioned is the following:
لقَدْ نَفَعَنِي اللَّهُ بكَلِمَةٍ سَمِعْتُها مِن رَسولِ اللَّهِ صلَّى
اللهُ عليه وسلَّمَ أيَّامَ الجَمَلِ، بَعْدَما كِدْتُ أنْ ألْحَقَ
بأَصْحابِ الجَمَلِ فَأُقاتِلَ معهُمْ؛ قالَ: لَمَّا بَلَغَ رَسولَ
اللَّهِ صلَّى اللهُ عليه وسلَّمَ أنَّ أهْلَ فارِسَ قدْ مَلَّكُوا عليهم
بنْتَ كِسْرَى، قالَ: لَنْ يُفْلِحَ قَوْمٌ ولَّوْا أمْرَهُمُ امْرَأَةً.
Abu Bakrah relates: 'During the times of [the battle of] al-Jamal,
Allah benefited me with words I heard from the Messenger of Allah ﷺ:
when the Prophet ﷺ heard the news that the people of Persia had made
the daughter of Khosrau their queen (ruler), he said: 'Never will a
people succeed who give their leadership to a woman.' (Al-Bukhari
The form of the hadith: Unlike many other hadiths that explicitly command or prohibit certain actions, this hadith is worded as an observation rather than a direct instruction. Some argue that it is a statement that serves as a warning or discouragement against placing women in positions of leadership without necessarily prohibiting it. Others have viewed it as an example of الخبر المراد به الإنشاء that is language that is phrased as an statement, but is in reality an instruction; hence, arguing that the hadith in practice forbids women from holding positions of authority.
Specific context or broad application? The hadith originated in response to a specific historical event: namely the people of Persia appointing a woman as their leader. Abu Bakrah, the companion who heard this from the Prophet ﷺ later applied this statement to a different situation - the Battle of Jamal (Camel) - where the people were following Ayesha (may Allah be pleased with her). He cited this hadith as his reason for not joining those who supported Ayesha, radhi Allahu anha.
However, the wording of the hadith is general, "Never will a people succeed..."; this opens the door to broader applications. There is a famous legal maxim: العبرة بعموم اللفظ لا بخصوص السبب "Consideration is granted to the generality of the language, not to the specificity of the reason" therefore, the general principle derived from specific incidents can be applied to a broader context; so while the hadith was originally said in the context of choosing the leader of a country (Persia), it could also be used to imply that women are discouraged from holding any position of authority.
Here is a summary of the opinions of the scholars regarding various aspects of female leadership 1:
Great leadership there is consensus that women cannot assume the role of the khalifah, or الإمامة الكبرى the leader of all Muslims. The vast majority of the scholars also hold that women must not become head-of-state; though dissenting opinions exist, both in the past and in contemporary times.
Judgeship: Shafi'i, Hanbali and Maliki schools hold that women cannot become judges. Hanafi scholars allow women to become qadis/judges in financial and civil matters, but not in cases involving qisas (retaliation), diya (blood money) or the like. Al-Tabari (though his madhhab doesn't survive today) stands alone in permitting women to be judges without restrictions.
Other positions of authority: As noted in your question, most scholars are of the opinion that women can hold any other position of authority, as long as this does not lead to violating any ruling or principle of Islam. This includes managing companies, organisations, educational institutions, etc.
That being said, in traditional Islam, a woman's success is not measured by her ability to 'climb the corporate ladder' but by seeking to become a better servant of Allah, and focusing on her duties as wife and mother (if she is married and/or has children).
Therefore, the principle remains that, all else being equal, it is preferable for women to stay at home and minimise their interactions with non-mahram men; as Allah ta'ala says in the Quran.
Stay in your homes, and do not display yourselves as women did in the
days of pre-Islamic ignorance. (33:33)
Wa Allahu a'lam.