You should have your co-worker with you when you tell your boss what you think of him.
Ghibah [غيبة] is a word derived from the root word Ghaib [غيب] which means "That which is hidden".
In an authentic hadith, Abu Hurairah (May Allah be pleased with him) said:
The Messenger of Allah (sallallahu 'alaihi wa sallam) said, "Do you know what is
ghibah (backbiting)?" The Companions said: "Allah and His Messenger know
better." Thereupon he said, "Ghibah is talking about your (Muslim)
brother in a manner which he dislikes." It was said to him: "What if
my (Muslim) brother is as I say." He said, "If he is actually as you
say, then that is ghibah; but if that is not in him, that is
[Sahih Muslim 32:6265]
وعن أبي هريرة رضي الله عنه أن رسول الله صلى الله عليه وسلم قال:
“أتدرون ما الغيبة؟” قالوا: الله ورسوله أعلم قال: "ذكرك أخاك
بما يكره" قيل: أفرأيت إن كان في أخي ما أقول؟ قال: "إن كان
فيه ما تقول، فقد اغتبته، وإن لم يكن فيه ما تقول فقد بهته”.