Yes it would. Adding material generally makes thermal conductivity worse. There is no material with negative thermal resistance, so adding more will always increase thermal resistance by at least a bit.

The only exception is 3D geometries, where an increase in surface area due to the extra material can outweigh the effect of the added material, and lead to a net increase in thermal transfer.

This can lead to a paradoxical effect, where ADDING insulation around a pipe leads to HIGHER transfer than with less insulation material, because the added material increases the exterior diameter, and therefore surface area.

(Heatsinks are also a good example. It is more material than just the component by itself, but increases surface area by such a massive amount that it more than outweighs the added material)