Note that there are two ways for a college student to be involved in “sports culture”: as a fan, and as a participating athlete.
At LACs, the fan culture is generally muted compared to major DI universities. You probably won’t find rabid fan bases, elaborate tailgate parties, televised games, or huge packed stadiums or arenas. If you want to avoid such things, then a LAC would probably be a good choice.
However, LACs often have high levels (as much as 30-40%) of athletic participation. You could find that lots of your fellow students are on the lacrosse team or run cross-country or play softball, just as you might find that lots of them play in the orchestra or work on the newspaper or are involved in student government. LACs commonly attract students who did lots of extracurricular activities in high school, including athletics, and they keep up those extracurriculars in college.
In fact, some people are attracted to LACs specifically because they want the opportunity to play NCAA sports in college, but aren’t good enough to get recruited for an athletic scholarship at a Division I university.