Then just don't use Adwaita/Breeze themes if that bothers you so much? And yes, it's tied to the theme. Ubuntu Unity's default "Ambiance" theme uses much less space. XFCE's default theme also uses less space. You can get themes like Chicago 95 for XFCE which also use less space.
Having visited the third circle of hell before, and having actually gone as far as writing my own theme before giving up the whole charade, I would rather avoid revisiting the fourth circle of hell that GTK theming is.
Edit: also, both Greybird and Ambiance are both really large compared to anything you get on other platforms. It's through no fault of their authors, GTK's layout rules break (or at least broke, I was on GTK 3.24 last I tried it) at low padding values, and it's extremely hard to keep alignment afloat on small widgets. I'm also not exactly young anymore, the text contrast in Greybird's unfocused windows is pretty awful, and fixing anything related to GTK's :backdrop attribute is a very long-winded affair.
You could also just use MATE!