There were four sequential steps in Adam's story:
[ 2:35 ] And We said, "O Adam, dwell, you and your wife, in Paradise and eat therefrom in [ease and] abundance from wherever you will. But do not approach this tree, lest you be among the wrongdoers."
[ 2:36 ] But Satan caused them to slip out of it ...
Qortoby Tafseer states that this slip is considered a sin (Arabic reference: here). That sin caused them to be punished.
[2:36] and removed them from that [condition] in which they had been. And We said, "Go down, [all of you], as enemies to one another, and you will have upon the earth a place of settlement and provision for a time."
- Repentance, Acceptance and Mercy
[ 2:37 ] Then Adam received from his Lord [some] words, and He accepted his repentance. Indeed, it is He who is the Accepting of repentance, the Merciful.