Yale and all highly selective colleges do take into consideration trends in academic performance, but absolute GPA/class ranking and rigor of courses will probably matter more absent some highly unusual circumstances that can be substantiated through third party sources (e.g. counselor or teacher recommendations). I and many other posters on this site believe, unless there is a major hook like being a recruited athlete, famous person or child of famous person (but even then there are still fairly high minimum standards), there are minimum objective cut-offs in GPA/class rank and test scores that put you in the pile that merits serious consideration where they will take the time to do the full holistic evaluation. The schools publish those statistics on their admissions page for a reason. Here is the 2020 freshmen profile page from the Yale Admissions website https://admissions.yale.edu/sites/default/files/files/class_profile_2020_8-29.pdf.
While the profile page uses the top 10% of the high school class as a cutoff, I would guess a much greater part of the 95% are in the top 5%. You do not mention how badly you did freshmen year, but can you realistically climb up to the top 5%, at the very least top 10%? Further, as you scan the results pages here for all the highly selective schools, you will see candidates with perfect or near perfect GPA’s and test scores who have been rejected. With that, Yale is a great school to aspire to gain admissions, but the odds for even the top students to get in are low. Every student needs to set up a strategy of applying to reach, target and safety schools, and keep in mind that your success in college is much more dependent on your efforts there, not which college you attend. Good luck.