Will I be at a disadvantage? (very few extracurriculars due to circumstances)

Again, working is an excellent extracurricular, even more so if you’ve moved up by the time you apply to colleges (i.e. to shift supervisor, etc).