Need help figuring how to not run into a Server to Server Rate Limit

[Saik1992]

I am building a Web-App that includes a Spotify Integration.

Essentially, I am grabbing a User's Followed Artists and building a cache to later display them and do stuff like automatic (complex) playlist generation, notifications, ect.

For this I've set up repeated checks for artist data in my Backend. An artist of interest (one of my user follows them) gets checked once every 24 hours, other artists just get their basic info (Name, Images) grabbed once and then updated every week/month (still figuring how often I should do this).

The past days, I've ran into heavy rate limits from Spotify. I am aggressively monitoring my spotify Integration (using both traces and prometheus metrics), but it's getting Retry-After Headers of  ~24 Hours after just a couple API calls repeatedly - meaning: The Retry-After timeout is over, and I get another one after just updating one or two artists fully.

Non-followed artists are just 1 API call, followed artists are:
GET artist
GET artists releases (singles, albums, compilations)
compare releases against my last updated time and batch GET releases that are new.

I have ~260 followed and ~20000 unfollowed artists.

Here's my question: How can I adjust this process so I am not hitting big rate limits?