The calendar was regularized to include a leap day during the reign of Julius Caesar (hence the name "Julian calendar"), which would have been 45 BC.

The Roman calendar moved to January as the first month of the year in 153 BC, over a hundred years before the leap day was added. The 10-month calendar may not have even existed--we see no contemporary evidence of its existence, only reports of its existence from centuries hence and the change there is attributed to a mythical character.

That's correct, the Romans had March as the first month of the year, so leap day was the last day of the year and September, October, November and December were the 7th (sept), 8th (oct), ninth (nov) and 10th (dec) months.

And (oct)ober was the 8th month of the year, (nov)ember the ninth, (dec)ember the tenth!

Technically, the leap day (bissextus) was the 24th. (Wikipedia tells me this is because that's when Mercedonius used to be, before the Julian reforms.)