The problem arises when someone in a position of authority cares. You might not invite your boss to that party of drunken debauchery, so why give them a sans context window to it on the internet? Better yet, why leave an insecure trail for future employers/family/networking? I'd rather err on the side of caution and keep shit locked down to avoid unwanted situations.
I don't think it's a question of 'who should care' so much as 'who will care'?