In Surah Baqarah, Allah says:
Allah does not charge a soul except [with that within] its capacity.
It will have [the consequence of] what [good] it has gained, and it
will bear [the consequence of] what [evil] it has earned. "Our Lord,
do not impose blame upon us if we have forgotten or erred. Our Lord,
and lay not upon us a burden like that which You laid upon those
before us. Our Lord, and burden us not with that which we have no
ability to bear. And pardon us; and forgive us; and have mercy upon
us. You are our protector, so give us victory over the disbelieving
In another Hadith,
The Prophet (ﷺ) said, "A Muslim whose three children die before the
age of puberty will be granted Paradise by Allah due to his mercy for
Sahih al-Bukhari 1248
Aisha reported: The Messenger of Allah, peace and blessings be upon
him, said, “Be deliberate in worship, draw near to Allah, and give
glad tidings. Verily, none of you will enter Paradise because of his
deeds alone.” They said, “Not even you, O Messenger of Allah?” The
Prophet said, “Not even me, unless Allah grants me mercy from himself.
Know that the most beloved deed to Allah is that which is done
regularly even if it is small.”
Source: Ṣaḥīḥ al-Bukhārī 6099, Ṣaḥīḥ Muslim 2818
From the above Hadiths, we can infer that no soul will be tested more than that they can endure. Secondly, no matter how many good deeds we earn, we will never be able to do justice to God. It is only through His Mercy alone that all of us, including Prophet Muhammed, will be granted Paradise.
Hence, those who did no sin inshallah will not be punished (dead toddlers as in your question, but perhaps through them their parents gain access to paradise) and those who do good deeds will have their ranks elevated in paradise.