Mu'in was born to the large landowning Palestinian Arab family, al-Madi, in the town of Ijzim west of the Jezreel Valley. The al-Madi family rose to prominence in the first half of the 19th century under Sheikh Mas'ud al-Madi, originating from Ijzim where they owned considerable amounts of land. They expanded their holdings to the city of Haifa and dominated swathes of territory along the Mediterranean coast of Palestine, stretching from Jaffa to villages north of Haifa. Mu'in's father, Abdullah al-Madi, was a member of Haifa's municipal council from 1902-05 and 1912.

 

Mu'in received his primary education at the government-run al-Rushdiyya School in Haifa, finishing in 1908. He later enrolled in the al-Mulkiyya School (Royal College) in Istanbul where he learned the Turkish language. After graduating in 1912, he began his career as an Ottoman clerk in the town of Kashmuna in Anatolia.

In 1914, al-Madi was elected head of Acre's municipality and then leader of Baysan. On August 1, 1915, upon suspicion of helping Arab nationalists, he was transferred to Beirut (the Ottoman governor there, Jamal Pasha, had already hanged a number of Arab nationalists in the city) to serve as governmental employee. Following this demotion, he was tried by a military tribunal on charges of "conspiracy against the Ottoman state," but the charges were acquitted when his father interfered on his behalf. Al-Madi's demotion and trial left him embittered with the Ottoman authorities and after his release, he took up an alias and hid in the house of an Arab Christian, Najib al-Hakim, in Damascus.