Making the middle class poor by taking away their ability to own a home, etc is absolutely not a “poor people don’t matter” argument. It’s a “let’s not destroy a stable part of society in a ham-fisted attempt to help the poor”.

Refuting dumb ideas to help the poor doesn’t mean I don’t care about helping them.