You register with the election commission (the government) but in order to vote in party primaries, you have to register as their party. You're free to register as an independent.

You either register with the Post Office(USPS) or with your city's/county's/state's government. For California you can see the registration form and options here(just click register and say yes and you'll see the form with options): https://registertovote.ca.gov/

You register to vote and, depending on state/county, you choose what party you belong to(Republican, Democrat, No Party, Green Party, etc.). You may need to register with a specific political party because of open vs. closed primaries where you MUST be a registered Republican to vote in the Republican primary and/or register Democrat to vote in the Democrat primary, etc.

California has a good website/page with more details: https://www.sos.ca.gov/elections/political-parties/no-party-preference/

Again, different states have different ways of doing things.

Registration streamlines voting in that you don't need to bring ID, proof of eligibility, and residency to the polling place. You just show up and they check your signature.

Party declaration determines what party's primary you can vote it. It's to prevent people from casting spoiler votes to help a weak opposition candidate win their primary. Not all states do this: some ask which ballot you want at the polling place.