This may not be true for your website, but on many Gatsby-driven websites there's heavy reliance on JavaScript that doesn't actually add very much. The really good thing about Gatsby is that it makes moving between pages more efficient because it only downloads the content that changes and then it leverages React to seamlessly transition the parts that change when you move from one page to another. It does that really well. The problem is that a lot of websites that use Gatsby are blogs, and that's not how people read at blogs; most of the time people go straight to a page from Google, and then go back to Google after they've read that page. They don't move around the website. All that JS to make that happen nicely does nothing.
That's certainly the experience I had when my website was Gatsby based. I was sending people a chunk of JS that that just didn't benefit from. If Gatsby suits your visitor model then it's a great tech choice, but I found that something even more static (11ty in my case) worked better for people who came to my website.
Also, for what it's worth, I get 4 100s in Lighthouse. :)
Gatsby uses a lot of JavaScript but, importantly, the pages render without JavaScript enabled. It's just the transitions and prefetching which require JavaScript.
The point is, on my website at least, none of the transition or prefetching was being used by most visitors. About 85% of people hit one page, stayed for a few minutes, and then left. In that case all that JS is just a waste of downloaded data. That's why I moved my site to static HTML and CSS. Slightly annoyingly ever since I did that the number of users who only read one page has dropped to about 75%, but I still see it as a benefit.
I get 98 on all lighthouse scores.