Does Penn allow students to take summer school elsewhere?

If a student wants to lighten their semester load at Penn, are they allowed to take a summer class at a state university and then transfer in the credit?

Yes, with prior approval.
https://www.college.upenn.edu/credit-away

Is this common at all or pretty rare?

It’s certainly not rare.

But it is important to get prior approval before enrolling in a course