Look at the schools you are interested in. There may be required and recommended years of a foreign language. If the admissions site lists classes most successful applicants have taken that is also useful. For example a school requires x units (years) but most admitted students will have had x+ units. Here on CC I have seen this question asked in the UW (Wisconsin) forum. Student responses seem to indicate it did not matter for students who had stellar grades, test scores and other attributes. However, if you are in the middle of the pool for a given school it could matter.
Therefore the “it depends” responses.
btw- completing the fourth year/level of a single foreign language counts, not taking it all four years of HS. You may also want to check on the schools on your radar requirements for getting a degree. Again, taking that fourth year could exempt you from needing any foreign language in college. Eg- UW requires four years of one language or three of one and two of a second for a BA (three of one for a BS) in L&S. Depending on your proposed major taking the final year in HS could free up time in college.
If the German is really a burden of course you should drop it and enjoy your other academic classes- ie if the costs outweigh the benefits. Your world will not end, your future will not be ruined…