Smith is going to have plenty of rigor. It’s not like you’re asking about Hamilton versus Directional State U. The users commenting on this are very experienced so I feel confident in saying you should trust them. Both schools are going to prepare you well for grad school. Both are rigorous. Both are highly regarded.
It almost sounds as though you want to say yes to Smith, but are looking for reasons not to. It’s a great school. Go and thrive.