Smokers on average don't die that much younger. But they do have a much less healthy end of life.
The life expectancy of male smokers, ex-smokers, and never-smokers at age 40 years was 38.5, 40.8, and 42.4 years respectively. In women, the corresponding life expectancies were 42.4, 42.1, and 46.1 years.