I don’t think it’s very helpful to use graduate rankings for an undergraduate program. At the very least, there are factors that can make up for the difference between 28 and 41. How involved are faculty members in undergraduate education? Do the professors that contribute to the high ranking mostly teach grad classes, or maybe they don’t even teach at all? What are the opportunities does the school give for undergraduate research? Many math professors do not believe that undergraduates can or should do research, so this can make it difficult to pursue research opportunities.
Notre Dame also has an honors program in their math department, btw.