We work on this a lot at tscircuit, and we've used cassowary (i.e. flexbox-style) constraint solvers. The issue with cassowary/flexbox is PCBs are not as uniform as webpages w.r.t. alignment, and often have a much less nested structure (3 layers) vs web pages which have many many nested layers. CSS grid-style constraints are a much better fit IMO, but we eventually settled on sequential optimal packing[1] for "seeding" a placement so that AI can get initial positions before working in a feedback loop. I like sequential optimal packing because it's very very deterministic and the constraints are specified pretty close to how a human would specify them
[1] https://blog.autorouting.com/p/sequential-optimal-packing-fo...
The first thing that came to mind reading it was "backtracking". Which I guess is the recursive pathfinder piece?