The basic principle is that the garment and body remains pure (taahir) until and unless you are certain that impurity has got onto it. If you are certain that drops have fallen, then you must wash the area which is touched by impurity and do wudoo, and wash off whatever gets onto your clothes of that. But if there is any uncertainty(might be/may be) about that, then you do not have to do anything, and you should ignore the doubts and uncertainty.
Shaykh al-Islam (may Allah have mercy on him) was asked: What if a
person does wudoo’ and starts to pray, then feels a drip during his
prayer: is his prayer invalidated or not?
He replied: Merely feeling something does not invalidate wudoo’ and it
is not permissible for him to stop an obligatory prayer because of a
mere doubt. But if he becomes certain that urine has escaped to the
exterior of the private part, then his wudoo’ is invalidated and he
has to clean himself with water (istinja’), unless he suffers from
urinary incontinence, in which case that does not invalidate his
prayer if he has done what he is enjoined to do.
End quote. Majmoo‘ al-Fataawa, 21/219-220