I've never used FastAPI, but https://www.techempower.com/benchmarks/#section=data-r19 (which is the first thing TechEmpower shows me) says that for this specific test (and maybe others are different, you should look into it!), FastAPI does 52,080 RPS, and actix-core does 651,144.
One person's trivial 1-day of effort is another person's month. There's a solid jsonwebtoken library that could be used in a custom token authentication middleware, if the one in the ecosystem is unsatisfactory. It's not trivial to learn a web framework and how to write custom middleware, but if you know where to look for actix-web there is an example: https://github.com/actix/actix-extras/blob/master/actix-web-...