Think of the atmosphere as a layer cake. The bottom of the cake is very dense and the higher up you go, these layers get less and less dense. Up on a mountain, the air is very thin.
In other words, there are fewer air molecules per cubic foot (volume of air). The molecules are farther apart and can hold less heat energy. Because "heat" is what we say when we mean molecules are moving around. The more they move, and the more molecules there are, the more heat you have.
It's also helpful to know that the air is heated by the ground and oceans. The heat from our sun mostly passes right through the atmosphere, not directly warming the air up very much. But the surface of the planet will warm up wherever the sun is shining on it. And in turn, the warm ground or the warm surface water then gradually warms the air from the bottom up. (This is because heat is transferred in different modes: radiant, convection, and conduction.)
And the warm air does indeed rise. As it rises, it gradually spreads out and cools off again. Some of the heat even radiates back out into space.
There are "fountains" of air constantly circulating throughout the atmosphere, and this creates weather patterns. It tends to snow on mountains because the warm air has carried some moisture with it on its way up. As it cools and thins, it can't carry the moisture any more, and the moisture precipitates out. Which is why we call it precipitation whenever it snows, rains, sleets, etc.
So by the time air reaches a high mountaintop, it's probably going to be cool or even frigid cold.
This is also why hotter regions, like the southern US, tend to get very humid in the summer. The warm air can carry a lot of moisture, and there is a lot of warm surface water. Our sweat is less efficient when the air is moist, because it takes longer to evaporate and carry the heat away with it.
Deserts have few water sources. So they also have hot dry air, and much less humidity, and therefore little to no precipitation. But they also get cold at night, because there's very little humidity to hold the heat overnight.
All of this is to illustrate the complex interactions between the sun, the atmosphere, and water (or lack of it) on the surface, and humidity in the air.
Inside an older building you're more likely to experience warmer air on higher floors than lower floors because the air is trapped in a nearly closed system and hot air rises. Of course, HVAC engineers try to compensate for this in modern buildings.