Basically, our current property tax system allows you to buy a vacant lot, sit on it for a number of years while paying pennies in taxes (because a vacant lot or a parking lot is very low value, even if the land is very high value), and then resell for a much higher price after the city has grown around it. Basically no work, but tons of free profit.
The remedy? Tax land instead. That way, your tax burden is based on the value of your land, not the value of your improvements, so that this form of land speculation becomes uneconomical, while also strongly incentivizing you to develop something more valuable, e.g., housing, offices, etc.