The hadith is as follows:
إِنَّ أَعْتَى اَلنَّاسِ عَلَى اَللَّهِ ثَلَاثَةٌ: مَنْ قَتَلَ فِي حَرَمَ اَللَّهِ, أَوْ قَتَلَ غَيْرَ قَاتِلِهِ, أَوْ قَتَلَ لِذَحْلِ اَلْجَاهِلِيَّةِ
Three types of people are most hated by Allah, the Almighty: whoever kills another in the Sacred area of Haram (Sanctuary), whoever kills anyone other than the one who killed him or whoever kills anyone as revenge for the times of Jahiliyah (pre-Islamic times).
— Bulugh al-Maram, Musnad Ahmad 6681
It means that if someone is murdered then his heirs should only kill the murderer and not anyone else in Qisas. This is in contrast to the practice of Jahiliyah where powerful people would not be satisfied with just executing the murderer but would rather also kill the relatives or tribesmen of the murderer in their rage. Similarly if the murderer could not be killed they instead killed someone else from his tribe as a substitute.
The prohibition of the same is conveyed in the Quran:
ولا تقتلوا النفس التي حرم الله إلا بالحق ومن قتل مظلوما فقد جعلنا لوليه سلطانا فلا يسرف في القتل إنه كان منصورا
And do not kill the soul which Allah has forbidden, except by right. And whoever is killed unjustly - We have given his heir authority, but let him not exceed limits in [the matter of] taking life. Indeed, he has been supported [by the law].
— Quran 17:33
This applies to one whose blood is protected i.e. one who is a Muslim or Dhimmi or Musta’min or Mu‘aahid. It does not apply to a Harbi Kafir in war. Rather it is established that the blood of a Harbi Kafir is lawful to shed, regardless of whether they fight or kill anyone:
فاقتلوا المشركين حيث وجدتموهم ... فإن تابوا وأقاموا الصلاة وآتوا الزكاة فخلوا سبيلهم
Kill the polytheists wherever you find them ... But if they should repent, establish prayer, and give zakah, let them [go] on their way.
— Quran 9:5
أمرت أن أقاتل الناس حتى يشهدوا أن لا إله إلا الله وأن محمدا رسول الله ... فإذا فعلوا ذلك عصموا مني دماءهم وأموالهم
I have been ordered (by Allah) to fight against the people until they testify that none has the right to be worshiped but Allah and that Muhammad is Allah's Messenger ... so if they perform that, then they save their lives and property from me.