You're free to remove it but it also adds texture. The taste of the rest is lightly sweet so I guess it should be good even without it.