Dos it matter? Valedictorian is a great honor - no matter how big your school is. Its not your fault the class is small. It won’t look bad - but is not as impressive as being top 5% of a very large very competitive school - but is still impressive. It won’t keep you out of top schools, but the rest of your application could be the part that gets you in.
IMO, depending upon your test scores, once you cross the threshold of what the college finds acceptable, other factors will come into play, with your essays, recs, EC"s, etc. Ranking 1 vs. 2, or top 6% vs. top 4%? No difference.
Most HS’s will not have named val/sal by the time apps are submitted, so for admissions, you’re right, probably not that impressive. It’s a nice accomplishment, but it’s one that probably only you and/or your parents will care about now and in the future.