Your assessment is missing the simple fact that FPGA can do things a GPU cannot faster
Yes, there are corner cases (many of which no longer exist because of software/compiler enhancements but...). But there is always the argument of "Okay. So we run at 40% efficiency but our GPU is 500% faster so..."
Nvidia is the Ford F-150 of the data center world, sure. It’s stupidly huge, ridiculously expensive, and generally not needed unless it’s being used at full utilization all the time. That’s like the only time it makes sense.
You are thinking of this like a consumer where those thoughts are completely valid (just look at how often I pack my hatchback dangerously full on the way to and from Lowes...). But also... everyone should have that one friend with a pickup truck for when they need to move or take a load of stuff down to the dump or whatever. Owning a truck yourself is stupid but knowing someone who does...
Which gets to the idea of having a fleet of work vehicles versus a personal vehicle. There is a reason so many companies have pickup trucks (maybe not an f150 but something actually practical). Because, yeah, the gas consumption when you are just driving to the office is expensive. But when you don't have to drive back to headquarters to swap out vehicles when you realize you need to go buy some pipe and get all the fun tools? It pays off pretty fast and the question stops becoming "Are we wasting gas money?" and more "Why do we have a car that we just use for giving quotes on jobs once a month?"
Which gets back to the data center issue. The vast majority DO have a good range of cards either due to outright buying AMD/Intel or just having older generations of cards that are still in use. And, as a consumer, you can save a lot of money by using a cheaper node. But... they are going to still need the big chonky boys which means they are still going to be paying for Jensen's new jacket. At which point... how many of the older cards do they REALLY need to keep in service?
Which gets back down to "is it actually cost effective?" when you likely need