I asked a related question earlier, trying to understand the sudden, catastrophic decline of the Bani Hashim from the high point during the early adulthood of Abd al-Muttalib to the low point at the time of Muhammad's birth.
The most plausible answer I've seen so far is posited by historian Hussain Mo’ones. He says that the problems started when Abd al-Muttalib retired and passed his duties to his son Az-Zubair. At that point the leaders of the other Qurayshi clans overpowered the efforts of Az-Zubair to enforce honest trade with the caravans. This made it impossible for Az-Zubair to make any profit. Because of his obligations of hospitality to pilgrims, he eventually ran the entire clan of Hashim out of money.
So the other Qurayshi clans were powerful enough, disrespectful enough, and dishonest enough, to drive the Bani Hashim from the high point of Zamzam into poverty and low status.
But if the other Qurayshi clans had the power and the lack of scruples to drive the Hashim into poverty and ignobility, and even to force a debilitating boycott/siege on the Hashim, why would they have respected Abu Talib's formal protection of Muhammad?