Like, having less facial/body hair is something I want for sure

It's not super noticeable as far as I've seen. Most of us are still shaving, epilating/waxing, and/or using laser hair removal to take care of the majority of this. It will, however, stop male-pattern baldness in most cases.

I don't know if I'd be entirely comfortable if I had a lot of breast growth.

That's not something anyone can really predict for you. Anecdotally it seems to mostly hinge on family history, the age you start, dose method (people seem to think injections work best for breast growth), and whether you take progesterone. I've heard arguments for and against most of these. Whether or not your body decides to undergo major visible changes will be up to a roll of multiple dice, basically. In my case I look a little bit like my mom when she was my age.

For transfems in the very-binary mindset, I try to urge them to be content with the path their body takes them on (as surgery can be prohibitively expensive for most), but in your case I can't make any guarantees. Breast reductions in transfeminine people are probably a thing that exist, but I personally wouldn't know how to navigate that, as western medicine tends to have a very set path of progression designed for binary transpeople.

As an aside, I would personally not recommend edging your E extremely close to zero (I've heard people mention this to avoid the risk of major body changes) as it will negatively impact bone growth and make them quite brittle over time.

But I'm also worried that if I don't start HRT then my body will start changing in ways I don't like.

E won't stop you from changing or aging. I have grey hairs; you can still get female-pattern baldness. I didn't have stretch marks before. It's become quite difficult to maintain any muscle when it used to feel effortless, and you'll tend to hold onto more body fat. Some of these things may be desirable, some might not. You might be able to wrangle them to a small degree with diet, exercise, and lifestyle changes, but if you're spending all your time alive fighting your own gene expression I think there's a point where you'll have to sit down and decide what is a priority for you. I take hrt because it makes my life more bearable. If it didn't then I wouldn't have bothered.

One of the idle fantasies that I've had is to get to the point where I can pass as either male or female based on how I dress or style my hair.

This is pretty doable, possibly even right now. In the hegemonic culture, most people tend to base their judgement of someone else's gender presentation based on visual signifiers and vibes. They struggle to really understand identity in a way queer people do, so if you drastically change your presentation they tend to just go with what you show them instead of what you tell them. In queer spaces identity will trump presentation regardless of what your body is up to.

But as of right now, I don't have any to go out with IRL

Anything stopping you?

am I really going to get dressed up to stay home and play video games?

You could get dressed up to videochat with people? Also I wear women's pajamas when I stay at home and play video games.

I always get anxious about spending money and I don't want to buy stuff that I end up not liking.

The best way to handle this is to just continually try stuff on at the store until you find something you're happy with. Most of my early clothes were thrifted and I didn't make any larger purchases until I had a good idea what sizes and styles suited me. It's just something that takes a bit of patience, but once you figure it out it gets more intuitive.

living inauthentically is necessary to some degree anyways.

As someone who's spent the past decade pursuing this, I don't think so. I think it can seem necessary to live inauthentically if you value your identity like a fashion accessory. But I know people for whom this is a life or death question. For them, living authentically is something they are absolutely willing to give up a lot of comfort for, and for whom living inauthentically feels like a death sentence.

Taking on the pressure of being trans is daunting, of course. I'm not saying it isn't or that you're morally required to categorize yourself. I'm not saying that you shouldn't be scared to take the plunge, because most of us are. But at the heart of it, which more or less reiterates my last comment, this is about your very core, the definition of yourself, and what you value as a human being. Aesthetics are part of it but they aren't the whole of it. So if the question of what it means to live begins and ends at aesthetics, then I don't think hrt will bring you closer to your goals. Estrogen will fundamentally change the basic functioning of every cell in your body.

If you keep coming back to transitioning no matter how hard you try, I would sit with a little thought experiment for a while. Give yourself permission to transition and see if it changes anything. Once the anticipation is gone, the way you feel about it might surprise you.