As someone who goes barefoot whenever possible, i can tell you that the idea of toes not being particularly useful for agility is very incorrect. If i have to restrict my toes i feel like dogs do when you put booties on their paws, so incredibly clumsy.

The toes play a large part in keeping your balance, making minor corrections to your weight distribution, and especially the big toe is pretty significant when pushing off the ground.
Try walking (or even running) around with your toes lifted off the ground, it's very awkward.

The problem is that a lot of people these days are constantly wearing shoes that make their toes useless, and even when not wearing such shoes their toes have been squeezed into a pointy shape (bunions are almost entirely caused by this) and become weak from atrophy. A healthy foot has toes splayed quite wide and the toes will be pretty strong and probably a lot thicker than you're used to, look at baby feet for a rough example.