There's something i forgot in my original answer, but as it is important due to the word you used in your post, so I'm adding it now.
Note that accusing somebody to be an adulterer is a very grave accusation, as in Islam if you don't have at least four witnesses (four men or for each missing man two women, the accuser himself doesn't count) for that, you should be hardly punished according the shari'a. See (24:4-5 and 24:23) for the accusation of chaste women, which applies also for men. The Quran mentioned women in first place as falsely accusing them is more grave!
Note that those witnesses must have witnessed the act themselves (either seeing it, or maybe hearing it -with no doubt-), which usually would be hard! Of course if the person himself admits it freely then the witnesses won't be necessary
Please be aware of this!