I like the burgers at Louis Lunch, but a lot of people don’t. I think it’s fun to go there, at least once. As for Modern, I never went there (or really heard much about it) until recently, but I thought it was competitive with Wooster Square Pizza. I still prefer Sally’s, though.
Shake Shack! https://www.shakeshack.com/location/new-haven/
(Admittedly I say this as somebody from the West Coast, where Shake Shack has not yet arrived.)
Shake Shack is pretty good, but my philosophy is always to try to do something that you can only do in the place you are visiting. One of the pizza places mentioned above, or Louis Lunch, qualifies.
Bring your mom to toads on a wednesday or saturday! she’ll love the family-friendly ambiance and the live entertainment is top notch.