logoalt Hacker News

nickjjyesterday at 12:17 PM4 repliesview on HN

> The startup time of a simple .py script can easily be in the 100 to 300 ms range

I can't say I've ever experienced this. Are you sure it's not related to other things in the script?

I wrote a single file Python script, it's a few thousand lines long. It can process a 10,000 line CSV file and do a lot of calculations to the point where I wrote an entire CLI income / expense tracker with it[0].

The end to end time of the command takes 100ms to process those 10k lines, that's using `time` to measure it. That's on hardware from 2014 using Python 3.13 too. It takes ~550ms to fully process 100k lines as well. I spent zero time optimizing the script but did try to avoid common pitfalls (drastically nested loops, etc.).

[0]: https://github.com/nickjj/plutus


Replies

zahlmanyesterday at 1:44 PM

> I can't say I've ever experienced this. Are you sure it's not related to other things in the script? I wrote a single file Python script, it's a few thousand lines long.

It's because of module imports, primarily and generally. It's worse with many small files than a few large ones (Python 3 adds a little additional overhead because of needing extra system calls and complexity in the import process, to handle `__pycache__` folders. A great way to demonstrate it is to ask pip to do something trivial (like `pip --version`, or `pip install` with no packages specified), or compare the performance of pip installed in a venv to pip used cross-environment (with `--python`). Pip imports literally hundreds of modules at startup, and hundreds more the first time it hits the network.

show 2 replies
randomtoastyesterday at 1:46 PM

Here is a benchmark https://github.com/bdrung/startup-time

This benchmark is a little bit outdated but the problem remains the same.

Interpreter initialization: Python builds and initializes its entire virtual machine and built-in object structures at startup. Native programs already have their machine code ready and need very little runtime scaffolding.

Dynamic import system: Python’s module import machinery dynamically locates, loads, parses, compiles, and executes modules at runtime. A compiled binary has already linked its dependencies.

Heavy standard library usage: Many Python programs import large parts of the standard library or third-party packages at startup, each of which runs top-level initialization code.

This is especially noticeable if you do not run on an M1 Ultra, but on some slower hardware. From the results on Rasperberry PI 3:

C: 2.19 ms

Go: 4.10 ms

Python3: 197.79 ms

This is about 200ms startup latency for a print("Hello World!") in Python3.

show 1 reply
maccardyesterday at 1:58 PM

A python file with

    import requests
Takes 250ms on my i9 on python 3.13

A go program with

    package main
    import (
       _ "net/http"
    ) 
    func main() {
    }
takes < 10ms.
show 1 reply
tlyleungyesterday at 1:12 PM

Just a guess - but perhaps the startup time is before `time` is even imported?

show 1 reply