Below is a transcript of a "smallest script" which runs in 328 ms total on my machine. And that includes loading an ubuntu OS image, which could be optimized depending on what dependencies the script needs.
Of course, if you're invoking it on a remote cluster, there are many reasons it can talk longer, especially if the cluster has to scale to satisfy the request. But all those reasons are nothing to do with containers specifically - it's things like cluster capacity, node provisioning time, container pull time, network latency, etc. If you architect things properly, you can get the number below + network latency even for a remote cluster invocation.
$ time docker run ubuntu echo hello world
hello world
real 0m0.328s
user 0m0.011s
sys 0m0.010s