Yes, those are valid concerns. One alternative is to only reroll them once all artifacts have been generated, also increasing an "artifact reroll counter". All artifacts will also internally keep a copy of this counter.
If a chest is opened contaning an artifact where artifact.counter < current counter, then discard it and generate/drop a new artifact.