Gregory DiPippo to the Rescue!

Do Sundays "count"?
Which "Forty Days" are we talking about?
Well, New Liturgical Movement has most interesting information on this very question.

It is a universal custom of all historical Christian rites not to fast on the day of the Lord’s Resurrection, even in Lent and Holy Week. The original Roman Lent of six weeks therefore comprised forty-two days, but only thirty-six days of fasting, which St Gregory the Great describes as “the tithe of the year.” (Hom. XVI in Evang.) The Roman Missal preserves a reminder of this in the Secret for the Mass of the first Sunday of Lent, which speaks of the “sacrifice of the beginning of Lent.”

Not long afterwards, however, perhaps by [Pope Saint?] Gregory himself, the four days preceding the first Sunday were added to the fast to bring the number of days to exactly forty, the length of the fast kept by the Lord Himself, as well as by the prophets Moses and Elijah. This extension of Lent back to Ash Wednesday, once commonly known as “in capite jejunii – at the beginning of the fast”, is a proper custom of the Roman Rite, attested in the earliest Roman liturgical books of the century after St Gregory. 

And when a commentator reminds Mr DiPippo that Christ gave Himself no such break in the desert, he replies,

He had not yet risen from the dead, so there was as yet no need to observe Sunday. The sermon of Pope St Gregory the Great quoted above, in which he discounts the Sundays from the number of fast days, is from the end of the 6th century, but St Gregory was not much of an innovator, so we may safely assume that the idea was not a new one in his time.