It sounds like there are too many confounding factors (different tests being the biggest one) to draw a good conclusion from this data.
I will say, however, that when I taught college, I certainly looked at test scores as an indicator of my skill. If two students blew the midterm, shame on them. If the whole class blew the midterm, shame on me.