>I don't understand coworkers who say silly things at the lunch table without a second thought (...) doing so seems to risk your upward mobility

Some people could not give less fucks about "upward mobility" (especially at the cost of some other things they value). They have what they want in life, and they'd rather be genuine...