People without ADHD already get more than enough benefit from just coffee. Stimulants might be able to benefit them further, but if you let anyone take them, eventually it'd become expected of everyone due to their increased productivity, and the potential consequences of abuse are much higher than coffee.
Chronic use of caffeine blunts the effects, it doesn't do much to regular drinkers. You develop tolerance very quickly. I can drink several cups and get basically no effect. Apparently your body just increases the quantity of adenosine receptors in the brain to offset the caffeine.