Al-‘Allaamah, al-Imaam, Shaykh al-Albaani (may Allah have mercy upon him) stated:
Indeed, the matter of having an abundance of followers or a small amount is not the measuring tool to know if the caller is upon truth or falsehood. The Prophets (‘alayhim as-salaatu was salaam), despite their call being the same and their religion being the same, differed in relation to the number of followers they had, some having many and others having few. There was even amongst them a prophet who only one man believed in. Rather, there was amongst them (a prophet) who didn’t have anyone with him. In that, there is a serious lesson for the caller and those who are being called in this (day and) time. It is upon the caller to remember this reality and put a foot forward for the sake of calling to Allah. He should not care about the amount of those who follow him being small. This is because nothing is upon him except clear conveyance (of the message). He has a beautiful example for himself in the Prophets of the past, those who didn’t have anyone except a man or two with them.