As a practical matter, caskets can be quite heavy, and the bearers need to be roughly uniform in height. I was quite surprised at the weight when I was a casket bearer. Going up stairs increases the physical demands quite a bit, so you’d want some pretty strong people for that part.
With the pic that @abasket posted, you can see everyone is relatively young and fit, and the guys are not 120 pound nerds.