I have multiple projects in Sentry and I want to upload source maps for each of them. Do I need multiple auth tokens (one for each project), or should I use only one auth token for both of them?

It depends on what works best for you. Using just one auth token for all your source map/debug file uploads is enough. We simply provide the option for multiple in case it is more helpful to your workflow (ie. auditing purposes).