If it wasn't for the impact on offer people, I think you could handle it basically like sugary drinks - there's some benefit in discouraging it for health reasons but not as much benefit comparatively, so a more modest tax is all I could really argue for, yeah. (Like how nicotine gum is treated essentially)
Since the impact is mostly annoyance (the smell) and most restaurants are either smoke-free or offer separate enclosures, why tax it at all (besides for the smell)? I am reducing my lifespan by about 8 to 10 years with smoking, sure. But why should the government force me to change that by taxing it? Why tax sugary drinks or ban or criminalize drugs other than the caffeine, nicotine and alcohol?
If the idea is to make everyone be healthy, live as long as possible and be productive for as long as possible, why not ban dangerous sports, too? I'm "the government" for my dog and I don't let him do anything dangerous or stupid, but he's a dog and we're people. With the supposed free will and agency we all like.