The "utility" of utilitarianism isn't that type of utility. IIRC it generally refers to the idea of maximizing happiness and minimizing harm, with a focus on outcomes of the whole, rather than the individual. Efficiency of labor doesn't explicitly factor into it.

Personally, I think you're just rationalizing being lazy and potentially causing harm to others, which isn't utilitarian at all.