That would be unpleasant, yes, and would require further efforts to minimize the consequences for those with low incomes (for instance, expansion of food subsidies, which yes would require even more taxation of those with wealth), however, if food prices aren't as high as that now because of near slavery conditions, then there's probably no ethical way to avoid it.