When you come into mosque late for Zuhr, and someone is busy with salah, without you knowing if he is performing Sunnah Zuhr or Fard Zuhr, can you join that person and perform your Fard Zuhr and carry on after he salaam and still get the benefit of congregation (jama'a) salaah?
If we keep aside the issue of some madhabs not allowing a second congregation in a masjid, there is nothing wrong with one who is offering an obligatory prayer praying behind one who is offering a nafil prayer. (Please refer below as pointed by Saif in his comment to know more about the issue)
Mu‘aadh ibn Jabal (RA) used to pray Isha with the Prophet (SAW), then he would go back to his people and lead them in praying Isha, and the prayer was nafil for him and obligatory for them.
حَدَّثَنِي مُحَمَّدُ بْنُ عَبَّادٍ، حَدَّثَنَا سُفْيَانُ، عَنْ عَمْرٍو، عَنْ جَابِرٍ، قَالَ كَانَ مُعَاذٌ يُصَلِّي مَعَ النَّبِيِّ صلى الله عليه وسلم ثُمَّ يَأْتِي فَيَؤُمُّ قَوْمَهُ فَصَلَّى لَيْلَةً مَعَ النَّبِيِّ صلى الله عليه وسلم الْعِشَاءَ ثُمَّ أَتَى قَوْمَهُ فَأَمَّهُمْ فَافْتَتَحَ بِسُورَةِ الْبَقَرَةِ فَانْحَرَفَ رَجُلٌ فَسَلَّمَ ثُمَّ صَلَّى وَحْدَهُ وَانْصَرَفَ فَقَالُوا لَهُ أَنَافَقْتَ يَا فُلاَنُ قَالَ لاَ وَاللَّهِ وَلآتِيَنَّ رَسُولَ اللَّهِ صلى الله عليه وسلم فَلأُخْبِرَنَّهُ . فَأَتَى رَسُولَ اللَّهِ صلى الله عليه وسلم فَقَالَ يَا رَسُولَ اللَّهِ إِنَّا أَصْحَابُ نَوَاضِحَ نَعْمَلُ بِالنَّهَارِ وَإِنَّ مُعَاذًا صَلَّى مَعَكَ الْعِشَاءَ ثُمَّ أَتَى فَافْتَتَحَ بِسُورَةِ الْبَقَرَةِ . فَأَقْبَلَ رَسُولُ اللَّهِ صلى الله عليه وسلم عَلَى مُعَاذٍ فَقَالَ " يَا مُعَاذُ أَفَتَّانٌ أَنْتَ اقْرَأْ بِكَذَا وَاقْرَأْ بِكَذَا " . قَالَ سُفْيَانُ فَقُلْتُ لِعَمْرٍو إِنَّ أَبَا الزُّبَيْرِ حَدَّثَنَا عَنْ جَابِرٍ أَنَّهُ قَالَ " اقْرَأْ وَالشَّمْسِ وَضُحَاهَا . وَالضُّحَى . وَاللَّيْلِ إِذَا يَغْشَى . وَسَبِّحِ اسْمَ رَبِّكَ الأَعْلَى " . فَقَالَ عَمْرٌو نَحْوَ هَذَا .
Jabir reported that Mu'adh b. Jabal used to pray with the Apostle (ﷺ), then came and led his people in prayer. One night he said the night prayer with the Messenger of Allah (ﷺ). He then came to his people and led them in prayer beginning with Surat al-Baqara.
A man turned aside, pronounced the taslim (salutation for concluding the prayer), then prayed alone and departed.
The people said to him: Have you become a hypocrite, so and so?
He said: I swear by Allah that I have not, but I will certainly go to Allah's Messenger (ﷺ) and will inform (him) about this.
He then came to the Messenger of Allah (ﷺ) and said: Messenger of Allah, we look after camels used for watering and work by day. Mu'adh said the night prayer with you. He then came and began with Surat al-Baqara.
Allah's Messenger (ﷺ) then turned to Mu'adh and said: Are you there to (put the people) to trial? Recite such and recite such (and such a surah).
It is transmitted on the authority of Jabir, as told by Sufyan, that he (the Holy Prophet) had said: "By the sun and its morning brightness" (Surah xci), "By brightness" (Surah xciii), "By the night when it spreads" (Surah xcii), and "Glorify the name of thy most high Lord" (Surah lxxxii). Sahih Muslim 465 a
In this hadith we see that it is permissible for one who is offering an obligatory prayer to pray behind one who is offering a nafil prayer, because Mu‘aadh used to offer the obligatory prayer with the Messenger of Allah (SAW), so he discharged his duty, then he would pray the second time with his people, so it was voluntary for him and obligatory for them. It can also been seen in this Hadith that is recommended for Imams to be considerate for their congregation and not prolong the salat keeping in mind the children, sick and elderly.
If a person starts to pray behind a man who has started to offer a nafil prayer, as in the case asked about here, then the imam has the choice as to whether he recites out loud or quietly (unless it is Zuhr or Asr). But if he intended to lead the prayer from the outset, then he should recite out loud, because of the hadith of Mu‘aadh (RA) quoted above.