One should distinguish between two kinds of directives given in the Quran: 1) that are part of Sharia to be followed by Muslims and 2) that relate to the Ministry of a Messenger.
- Al-Ma'ida (5:32) is a representative verse of the first kind. This verse states that a human life can only be taken in two cases. (very limited reasons as you said)
- At-Tawba (9:5) is a representative verse of the second kind. The core idea is that a person who "denies the truth in spite of being convinced that it is the truth"  becomes a Kafir who can be punished by death. This is a Divine injunction to be carried out once the truth has been clearly communicated by a Messenger to the people he was sent to.
The proponents of killing Kafirs actually generalize the second kind which they should not.
Please visit the following links if you want to read more:
- Jihad against the Disbelievers
- Who is a Kafir?