Shaykh Usaamah al-‘Amri (may Allaah preserve him) mentioned that the scholars are in agreement that the one who leaves off the obligatory fast of Ramadhaan, denying the obligation of it, is a Kaafir. As for the one who leaves it off out of laziness, there are two statements. The first is that he’s a Kaafir. The second, which is the correct position, he’s not a Kaafir but a Faasiq. However Imaam adh-Dhahabi mentioned: The one who leaves off fasting is worst than the one who commits Zinaa and drinks Khamr”.
(Explanation of The Book of Fasting from Sunan at-Tirmidhee #1)