That cuts equally both ways. You could skim through a book without slowing down to intently focus on every single sentence and that would be functionally the same as drifting your attention in and out while listening to an audiobook.