I thought the issue was they wanted search engines to be able to see the content, but not non paying viewers? Hence slightly shitty paywalls.