Wow - there's free street parking in Lower Manhattan?! Yep, there's your problem!
Here in Sydney Australia, we don't have any congestion charge (there's been some talk about introducing one, but it's not really on the cards at this time). But it's pretty much impossible to find parking in the CBD (Mon-Fri 9-5) for less than around AUD$60 (USD$40) per day. There is literally no un-metered street parking anywhere in the CBD (also the parking inspectors are everywhere). Plus many of the routes in are tolled (although by no means all routes). Plus, things like the numerous one-way streets, bus-only lanes (with cameras), and ultra-low speed limits, makes it an extremely unpleasant driving experience (with a high risk of getting fines), for folks who are used to just driving in the suburbs. All of that effectively acts as a congestion charge - most people choose public transport over driving, when heading into the city centre, because in practice the cost of regularly doing the latter is prohibitive.
> most people choose public transport over driving
Is that true? I was surprised when looking at the actual stats (most people in Sydney drive to work), but maybe there are many more people working in places other than the CBD skewing this.
When I was cycling in from Balmain, there were a huge number of cars stuck in traffic getting into the CBD every morning. Despite it only being an easy 15 minute cycle (or a 15 minute bus ride) from the peninsula.