Are you getting the mRNA vaccines? I had side effects once from the Pfizer booster (3rd shot), extreme fatigue, slight body aches, sinus headache, cold chills, happened 18 hours after getting it. The first two doses and 4th shot, zero side effects.
I got the Novavax shot this time, after 24 hours, I got really sleepy and felt kinda off. I just went to bed early and was fine the next morning, nothing like the side effects from the Pfizer booster. Maybe try switching to that one next time.