Today I had a bath and while taking bath I noticed that there was some dirt which was black. I had to remove that dirt by using soap and rubbing. If the the dirt was there before, are all my previous ghusl invalid because of the dirt?
Salam Brother. Regarding your question I would like to quote a Sahih Hadith of Prophet Muhammmad (PBUH)
The deeds (their reward) are dependent on the intentions
Secondly, The dirt (soil) is not impure according to sharia unless it has urine, poop or other kinds of impurities.
So, insha'Allah your Ghusal will be considered valid.
Allah says in Quran:
لَا يُكَلِّفُ اللَّهُ نَفْسًا إِلَّا وُسْعَهَا ۚ
Allah doesn't burden someone beyond its capacity