The verse you've mentioned only intends to express how the rain was falling or pouring down on the people. It's like if Allah said:
"The rain was pouring down as if the gates of heaven (or better the sky) were wide open"
It is simply -like- an analogy made since the rain was falling over 40 days without stopping as mentioned by many exegetes.
Sheikh Taher ibn 'Ashur in his at-Tahrir wa (aT)-Tanwir expressed this as follows:
وجملة ففتحنا أبواب السماء بماء منهمر مركب تمثيلي لهيئة اندفاق الأمطار من الجو بهيئة خروج الجماعات من أبواب الدار على طريقة :
The phrase “Then we opened the gates of heaven with rain pouring down” (a more exact translation, however, might be: “We opened the gates of heaven with pouring water”) is a representation of the form of rain pouring from the sky in the form of groups exiting the doors of the house in the manner of (citing a part of a rhyme mentioned by ibn al-Athir (the writer) ابن الأثير الكاتب in his al-Mathal as-Sa'ir المثل السائر في أدب الكاتب والشاعر without mentioning the author):
وسالت بأعناق المطي الأباطح (Source)
And the valleys flowed with the necks of the mounts.
The full poem part or rhyme is:
ولمّا قضَينا مِن مِنى كلَّ حاجةٍ
When we had finished all our needs in Mina
ومسَّحَ بالأركانِ مَن هوَ ماسحُ
and the one who wiped the corners had wiped them,
أخذنا بأطرافِ الأحاديث بيننا
we took up the conversations among ourselves
وسالتْ بأعناقِ المَطِيِّ الأباطحُ
and the valleys flowed on the necks of the mounts.
What do these two rhymes mean?
The first one indicates that people were returning from Hajj since they finished all the tasks in Mina and had done everything ordered for Hajj. They rode the road back of their camels and talked and were busy with conversation on their way home.
When these people talked while they were riding their mounts, the pleasure of talking distracted them from holding the reins, so they loosened from their hands. When the matter was like this and the reins loosened from their hands, the mounts sped up in their travels, and their necks were likened to the speed of a torrent passing over the face of the earth.
The above is a translated summary of this article on alukah.net, which claims that the poem is attributed to Abu Sakhr Kathir ibn 'Abdarrahman ibn al_Aswad al-Khozaa'y al-Madani أبو صخر كثير بن عبد الرحمن بن الأسود الخزاعي المدني (40a.H. -150 a.H.).
Note that some scholars hold the opinion that literally, some gates were opened in the sky (heaven), but didn't specify.