There is a Pasuk in Eicha (1:3) גלתה יהודה מעוני ומרוב עבודה
"Yehuda has been exiled from misery and from an abundance of work"
The Pasuk teaches us that one of the reasons we were exiled was because of "oni" and "rov avodah".
The Chida explains this Pasuk in terms of spiritual priorities. When the important aspects of Halacha become oni-poor, unattended, and trivialized, while the less important parts, Chumras and Hidurim become rov avodah, when there is an overabundance of emphasis on the Hidurim while there is laxity in the primary of basic Kashrus, this causes the Galus to drag on.