You don't clean plastic lens spectacles with paper as paper will scratch the lenses, though glass lenses can be wiped with paper. Wood is harder than plastic.

Toilet paper is wood, while the seats are plastic. Wood is a 2.5-ish on the mohs scale, and plastics tend to be too. So they scratch at a 2.5 with deeper grooves at a level 3, even on a micro scale. The alternative is to use ceramics for the seat, but the fragility means one rough drop of the seat and it could shatter. Metals are generally no better.