I think that liking/not liking books doesn't have a lot to do with aphantasia. I love books and reading, but some authors are just terrible because they put too much visualization porn between interesting story elements (looking at you Tolkien).
For me to like a book, the story has to flow; Terry Pratchett is my favorite author, his stories flow in a really nice way and he tackles a lot of social issues in a great way.