Rich people have staff that knows all their secrets; assistants, bodyguards, maids, drivers, ... Their staff might abuse their position to steal, blackmail, or even sign book deals. They still have the staff.
When AI agents become useful for normal people, normal people will use them.