There's a difference between sci-fi as a setting, and science fiction as a genre.

Science fiction, sometimes differentiated as speculative fiction, is all about exploring the implications of fictional science and technology. Frankenstein is often considered the first work of science fiction, because it truly is a "what if" story centered on a piece of fictional technology. Hard science fiction, such as the Martian, takes this principle to its limit by trying to rely only on known or highly plausible science, with the technology as close to currently feasible as possible. These kinds of stories are deeply grounded, and compel because they consider the consequences of everything they add. The story follows from the science. Black Mirror is a perfect example of this writing philosophy, looking at how people and technology change one another.

The "sci-fi" setting isn't really science fiction at all. Star Wars is set in space with lasers and robots, but nothing about it is driven by science. Star Wars is more interested in special effects set pieces, the politics of fascism, and pseudo-buddhist philosophy. Star Wars is space fantasy, and its philosophical themes almost make it good, but not quite. Star Wars would be amazing if it was good, but alas.

I think that all great fantasy is also about exploring the implication of fictional science. It's just, this science is less sciency-feeling to laypeople. Lord of the Rings is linguistic fiction. The entire world exists to serve a story about the evolution of language. It's awesome! Mistborn is also a really great "fantasy" series where the rules of magic are scientific, and clever readers can tell when unreliable narration is lying to them using the scientific method. These are the stories I love.

Harry Potter sucks because it isn't science fiction. After the second one came out, lots of fans were eagerly expecting the series to delve into the ethics of slavery using House Elves as a narrative device. "What if slavery is enforced by magic rules?" is an interesting sci-fi question. Instead of writing good political fiction, Robert Galbraith chose to say "Dobby's just weird for wanting to be free." That's boring. There's nothing for us to sink our curiosity into in that plot point. It's intellectually lazy. It's not science fiction.

I posit that all good works of fantasy fiction treat their magic the way good speculative fiction treats its science, at least a little bit. And if you're not going to include any of that, then what's even the point? Your universe is wasted on your story, and I'd rather read what AO3 has to say about your setting. I've read lots of HP fanfiction that was better than HP because it cared about science. Even Harry Potter and the Natural 20 is better than Galbraith's garbage.

And I really need to get around to reading Trans Wizard Harriet Porber and the Bad Boy Parasaurolophus...