Yes, thermal mass only serves to even out fluctuations in temperature. If the outside environment swings between hot and cold then a building with high thermal mass will tend to have a temperature in the middle of those two extremes. Like how a heavier ship is tossed about on the waves much less than a small boat.

But if a place is consistently hot or cold for a long time thermal mass doesn't really do anything. At most you can use it as a battery, so you can, for example, run a heat pump while electricity is the cheapest and use the thermal mass to maintain the temperature you established over the costly period.

So many people think that its a substitute for insulation though, which slows down the rate of heat transfer in or out, and does actually let you use less energy to maintain a given temperature.