The simple criticism of degrowth is that advocacy for such a stance is based on lies and climate woo woo. You don't need to make an argument about the philosophy behind it, except to explain where the climate woo woo arises and why it is so prevalent in our current world.
The truth of course is that meeting human needs is no longer a serious matter of growth. We could produce far more than we do, but we don't because we produce for the profit motive, for the market, rather than meeting human need. We have no shortage of labor, and we could build a lot of factories right now if there were political will to do so. The fundamental point is that (a) no one in power wants the poor to have things, because controlled deprivation underpins literally everything about our present social order, and (b) we don't really need a bunch of new factories to meet needs, because a lot of things people want are just sitting unsold, and (c) people don't actually need that much to live happy lives, but we are deprived specifically of basic comfort and security.
The problem facing people is fundamentally a political one - that we live in societies that beat and cajole people to modify their behavior, rather than societies that are oriented to meet needs. The market society is an indirect way of turning those beatings and cajolings into dollar signs, but markets are not the entirety of the story. We see enough beatings and ways in which people are disciplined, and where the market forces don't create deprivation, brute force and institutions exist to enforce that deprivation even amidst plenty. The demands of the people often aren't even demands for material wealth or more money, but demands for basic decency and the ability to actually live their life. Wanting money is only a means to achieve those ends, rather than simply coveting the things themselves. You could do a lot to alleviate homelessness just by abolishing the absurd levels of rent-seeking and leaving the landowners and finance capital fucked, but the reason the rent-seeking is encouraged is precisely because professional-class and upper-class people want to wipe out the undeserving outright. They embraced eugenic value a long long time ago, and it's been this way for over a century.