Well, if you’re a guy, fewer people see you as a potential prey, regardless of size.
In my experience, guys simply take it for granted they can walk wherever they feel like. Women take for granted they can’t.
It’s not a matter of Baltimore.
It’s just ingrained, to the point that if you’re a gentleman you walk a woman back if it’s late and dark. You don’t do that for your buddy because you don’t assume someone will see him and think “easy prey” or “walking sex toy”.
Note however that the “walk back” services on university campuses are offered to both men and women.