From the French sortir, "to go out".

So yeah, the "out" and "back" parts are important: if a unit is attaching while maneuvering in the field, it's not a "sortie". That's also why it's common in aviation, since airplanes inherently require a home runway that every attack begins and (ideally) ends at.