16-bit Windows used the Pascal calling convention, with the documentation in the Windows 1.0 SDK only listing Pascal function declarations. (Most C programs for 16-bit Windows use FAR PASCAL in their declarations—the WINAPI macro was introduced with Win32 as a porting tool.) The original development environment for the Macintosh was a Lisa prototype running UCSD Pascal, and even the first edition of Inside Macintosh included Pascal declarations only. (I don’t know how true it is that Windows originated as a porting layer for moving (still-in-development) Excel away from (still-in-development) Mac, but it feels at least a bit true.) If you look at the call/return instructions, the x86 is clearly a Pascal machine (take the time to read the full semantics of the 80186’s ENTER instruction at some point). Hell, the C standard wouldn’t be out for two more years, and function prototypes (borrowed early from the still-in-development C++, thus the unhinged syntax) weren’t a sure thing. C was not yet the default choice.

>> Also, the stack is restored by the called procedure rather than the caller.

> What could possibly go wrong?

This is still the case for non-vararg __stdcall functions used by Win32 and COM. (The argument order was reversed compared to Win16’s __far __pascal.) By contrast, the __syscall convention that 32-bit OS/2 switched to uses caller cleanup (and passed some arguments in registers).

I loved OS/2 but I also remember the dreaded single input queue... but it didn't stop me using it until about 2000 when I realised it was time to move on.

Callee clean-up was (is? is.) standard for the 32-bit Win32 API; it's been pretty stable now for coming up on 40 years now.

> Did this choice of a small speed boost over compatibility ever haunt the decision makers,

...in the end it's just another calling convention which you annotate your system header functions with. AmigaOS had a vastly different (very assembly friendly) calling convention for OS functions which exclusively(?) used CPU registers to pass arguments. C compilers simply had to deal with it.

> What could possibly go wrong?

...totally makes sense though when the caller passes arguments on the stack?

E.g. you probably have something like this in the caller:

    push arg3      => place arg 3 on stack
    push arg2      => place arg 2 on stack
    push arg1      => place arg 1 on stack
    call function  => places return address on stack

(ok, x86 has the special `ret imm16` instruction which adjusts the stack pointer after popping the return address, but I guess not all CPUs could do that back then)

I remember learning this while I was on Microsoft's campus learning about OS/2 APIs. As I recall they said about 7% faster which is hard to leave on the table. I never had any issue with it the whole time I developed for OS/2 as well as Windows NT that used the same convention.

On x86, the RET instruction can add a constant to the stack pointer after popping the return address. Compared to the caller cleaning up the stack, this saves 3 bytes (and about the same number of clock cycles) for every call.

There is nothing wrong with using this calling convention, except for those specific functions that need to have a variable number of arguments - and why not handle those few ones differently instead, unless you're using a braindead compiler / language that doesn't keep track of how functions are declared?

Yup. If you call a function with the C calling convention with the incorrect number of parameters, your cleanup code still does the right thing. With the Pascal calling convention, your stack is corrupted.