Irony in the question noted :)
Classically, ijma' is defined as
the unanimous agreement of the mujtahidun of of the Muslim community of any period following the demise of the Prophet Muhammad on any matter (Amidi, Shaukani, and others).
So right away we see that there must be a plurality of mujtahids (who are taken to be representative of the ummah) who are discussing an issue and who unanimously agree on a certain ruling about the issue. The majority view is that all of the mujtahidun must agree, else no ijma' materializes.
In matters of fiqh, if a faqih is known to have actively invited people to innovation in religious matters, they are excluded from the requirements of ijma'.
This kind of ijma' has occurred in the past for example with groups of the Companions of the Prophet (saws) (there are examples from inheritance and other issues). However, apart from that era and some spotty examples from the Spanish Umayyads who set up official 'ulema councils and had the issue rulings, this standard has not been known to be achieved at other times.
Given the difficulty of identifying "every" mujtahid, and getting each one's opinion on a matter, clearly this is an extremely high standard to hold the Ummah to, so in reality the scholars impose certain other conditions to make it feasible. For example, Imam ash-Shafi'i restricts ijma' to matters that are obligatory only, saying that it's impossible to get ijma' on other matters. Imam Ahmad was of the opinion that ijma' only refers to the consensus of the Companions. Imam Malik preferred to base ijma' on the people of Madinah. The Shi'a base it on consensus of members of the Prophet (saws)'s family.
There are two kinds of ijma': Explicit (sarih) and tacit (sukuti). Explicit ijma', of course, is binding and definitive - it is where every mujtahid expressly expresses their opinion. With tacit ijma', a group of mujtahids of a particular time express their opinion, while others remain silent.
Now we come to the crux of the question: How is ijma' transmitted?
Ijma' can be transmitted either by tawatur (multiple channels, conclusive) or ahad (solitary). The only tawatur ijma' known is from the Companions' era; nothing after that. This is the main reason there is more disagreement in later ages about which issues have ijma' or not.
The 'ulema disagree on the value of solitary transmission of ijma' - the Ghazalian point of view is that solitary transmission is inherently "speculative" whereas ijma' by definition cannot involve any element of speculation. Other scholars accept the proof of ijma' on a matter by solitary reports.
In the end, it's not exactly a binary matter. The 'ulema of usul (principles of jurisprudence) maintain that if ijma' is transmitted through solitary reports, it loses its value and the matter must be proven from primary sources (where the ijma' was derived from in the first place).
Tl;dr answer: if someone claims there is no ijma' on a matter, ask them to produce a mujtahid who is agreed upon who clearly expressed his dissenting opinion. That is valid to break ijma'.
And Allah knows better. Please feel free to point out any mistakes.
This answer is sourced from Muhammad Hashim Kamali's book "Principles of Islamic Jurisprudence."