What was the religion of Muhammad before Islam?
According to Quran, Muhammad was neither a monotheist nor a polytheist before he received revelation. He was just an ordinary man. He carried no special interest to the common beliefs of his community, nor did he protest against it.
“And thus, We have revealed to you an inspiration of Our command. You did not know what is the Book or [what is] the faith before. We have made it a light by which We guide of Our servants who deserve. Indeed (now), you guide to a straight path.” (42:52)
He was not aware of the Bible and the Torah as he was illiterate.
“You were not reciting before this any scripture, nor did you inscribe one with your own hands. Otherwise the falsifiers would have had [cause for] doubt.” (29:48)
He probably knew that the common beliefs of his community were not right, but he were not aware of any better way. He was just lost like the majority of people today.
“He found you lost and guided you.” (93:7)
This is a common case for almost all of the messengers.
Abraham was also lost before receiving revelation. But unlike Muhammad, Abraham was in a pursuit of true faith and He became a natural monotheist through reasoning before receiving revelation. He tried to adopt things that are considered as divine in his time as Lords, but he quit believing in them one by one.
“Then, when the night outspread over him, he saw a star and said: This is my Lord. But when it faded, he said: I don’t like things that fade. Then, when he saw the moon shining, he said: This is my Lord! But when it faded, he said: I will be among the people who lost themselves unless my Lord guides me! Then when he saw the sun shining, he said: This is my Lord. This is the greatest of all. Then, when it faded, he said: Oh my people! I have no concern with whatever you associate (with Allah).” (6:76-78)
As an addition, let us remind that according to Quran, the greatest sin is shirk, which can be translated as “adopting mediators between man and the God”. All of the communities of messengers were mainly mushriks. They were not atheists. All of them knew and believed in a supreme being, the creator of all creation (see Quran 29:36, 31:25, 39:38, 43:87). Their mistake was to adopt some other things as mediators between them and the God. They put the mediators in the first place, and used to please them by worshiping icons which represent them, and were expecting them to be their intercessors in the Day of Judgment. Yes, they also believed in the concept of Last Day.
“The pure religion is the one devoted only to Allah. Those who have taken others from below of Allah for their guardians say: We only serve them for that they may bring us nearer to Allah. Allah will judge between them concerning what they differ about. Verily, Allah does not guide someone who lies and constantly ignores.” (39:3)
We meet this same type of belief in the majority of today’s Christians and -unfortunately- the majority of today’s Muslims. Christians believe that Jesus is/will be their advocate in the Day of Judgment and save them from the hands of Allah. Muslims(!) also believe that Muhammad will pray for them to Allah in the Day of Judgment, and Allah will forgive the sinful Muslims and let them enter to heaven for the sake of his “loved most” messenger Muhammad.
The pure religion of the God which He revealed to Noah, Abraham, Moses, Jesus and Muhammad accepts no mediators, no holy/divine men, no authority and no one to please except the one and only God Himself.