The covenant from the Prophets is essentially also a covenant from their ummah's, and it applies to whoever of them found the Messenger. This is apparent from the next verse:
فمن تولى بعد ذلك فأولئك هم الفاسقون
And whoever turned away after that - they were the defiantly disobedient.
As it is impossible for Prophets themselves to be disobedient as they are infallible.