Many hadith proves the verse of rajm(stoning) is abrogated. But my question is the verse is abrogated with which verse? Please give the reference in your answer.
Stoning does not appear in the Qur'an (ref.). It's instead mentioned in multiple hadith, e.g.:
The Jews brought a man and a woman who had committed illegal sexual intercourse, to the Prophet and the Prophet ordered them to be stoned to death, and they were stoned to death near the mosque where the biers used to be placed.
Sahih al-Bukhari 7332
Further ahadith are cited in: How many cases of stoning are documented in the hadiths? The closest the Qur'an mentions is:
The [unmarried] woman or [unmarried] man found guilty of sexual intercourse - lash each one of them with a hundred lashes ...
I don't believe stoning is considered "abrogated" by scholars (Wikipedia surveys the major schools of thought), e.g.:
...stoning is due upon a married person (men and women) who commit Zina if there is a clear evidence...
There are some arguments against stoning, although they go against the majority of scholars. Opposition to Rajm (Stoning): Analysis and Refutation debates this issue.
The verse of stoning was never abrogated in practice of Muslims almost all Muslim sects: Sunni, Shi'a and Ibadi confirm the punishment of stoning for married adulterers. If there was a verse abrogating it, this ruling would not remain or at least would be adjusted, which is not the case.
What was abrogated was the recitation this means in the final "update" of the qur'an the prophet () didn't confirm that this verse should remain in surat al-Ahzab and therefore in the qur'an we have at hand and people recite since 14 centuries this verse is nowhere mentioned. However several sahabah confirmed that it was part of the qur'an and mentioned similar wordings of this abrogated verse.
Further the ruling of stoning adulterers is still present in the old testament.
This might be regarded as a further evidence for the ruling. However both some early and contemporary scholars discussed and questioned the existence of such a verse due to different reasons like the wording mentioned in the ahadith which had small differences (this reason could rather be refuted easily) and doesn't seem to smoothly fit to the words of the qur'an. Beside this the remaining interpretation of the rule has an issue if it would be applied on slaves as they should get half the punishment of free people which doesn't make sense in case of a death sentence.