Although the Qur'an does not specifically say the amount of prayers that people should pray, it has said multiple times to follow and obey the prophet's commands. And of the prophets commanded, praying five times a day was certainly one of them.
Obey Allah (Ta’ala) and His Beloved Prophet (Sallallahu ‘alayhi wa Sallam)
(3:132) (4:13) (4:59) (4:69) (4:80) (9:71) (24:52) (24:54) (24:56) (33:71) (49:14) (64:12)
Follow the Commands of Allah (Sub’hanahu wa Ta’ala)
and of His Beloved Prophet (Sallallahu ‘alayhi wa Sallam) (33:36)
The prophet was told in his journey isra and mi'raj to observe 5 prayers
Then the prayers were enjoined on me: They were fifty prayers a day. When I returned, I passed by Moses who asked (me), 'What have you been ordered to do?' I replied, 'I have been ordered to offer fifty prayers a day.' Moses said, 'Your followers cannot bear fifty prayers a day, and by Allah, I have tested people before you, and I have tried my level best with Bani Israel (in vain). Go back to your Lord and ask for reduction to lessen your followers' burden.' So I went back, and Allah reduced ten prayers for me. Then again I came to Moses, but he repeated the same as he had said before. Then again I went back to Allah and He reduced ten more prayers. When I came back to Moses he said the same, I went back to Allah and He ordered me to observe ten prayers a day. When I came back to Moses, he repeated the same advice, so I went back to Allah and was ordered to observe five prayers a day. When I came back to Moses, he said, 'What have you been ordered?' I replied, 'I have been ordered to observe five prayers a day.' He said, 'Your followers cannot bear five prayers a day, and no doubt, I have got an experience of the people before you, and I have tried my level best with Bani Israel, so go back to your Lord and ask for reduction to lessen your follower's burden.' I said, 'I have requested so much of my Lord that I feel ashamed, but I am satisfied now and surrender to Allah's Order.' When I left, I heard a voice saying, 'I have passed My Order and have lessened the burden of My Worshipers." - Sahih Al-Bukhari 5.227
full hadith can be found here
Also, the Qur'an does not only talk about 3 prayers
You shall observe the Contact Prayers (Salat) at both ends of the day, and during the night. The righteous works wipe out the evil works. This is a reminder for those who would take heed 11:114
This verse says both ends of the day and night, so the ends of the day must be fajr and maghrib, and the one at night is i'sha
You shall observe the Contact Prayer (salat) when the sun declines from its highest point at noon, as it moves towards sunset. You shall also observe (the recitation of) the Qur'an at dawn. (Reciting) the Qur'an at dawn is witnessed. 17:78
The highest point at noon is dhuhr prayer and dawn is fajr, once again.
Maintain with care the [obligatory] prayers and [in particular] the middle prayer and stand before Allah, devoutly obedient. 2:238
The middle prayer is salat asr
So as you can see the Qur'an did mention all the prayers, and even if the Qur'an did not name all of them, the prophet did, and specifically ordered us to pray 5 prayers. And furthermore, there is no evidence that we may only pray 3 prayers.