The Arabic in that hadith literally just says "…والروم…" ("…and Rome…") It doesn't say, or even imply, anything about conquering, much less being conquered by Muslims.
The sign that it's likely referring to is found in the Quran:
 Alif, Lam, Meem.
 The Byzantines (Rome) have been defeated
 In the nearest land. But they, after ...
The first verses of the Quran that were revealed about fighting are in Surah Hajj:
Permission (to fight) is given to those (believers) who are fought against, because they have been wronged; and surely, Allah is able to give them victory.
Those who have been expelled from their homes unjustly only because they said: "Our Lord is Allah.'' For had it ...
Sheikh al albani classified this hadith as hasan.
There's also a sahih Hadith in Muslim which may act as an evidence for the authenticity of this hadith.
Abu Huraira reported Allah's Apostle (may peace he upon him) saying:
You have heard about a city, one side of which is on land and the other is in the sea (Constantinople). They said: Allah's Messenger, yes....