The network-request-first approach is the right call. DOM parsing is fragile because it's scraping a rendering artifact — any style refactor, framework upgrade, or A/B test can silently break it. Intercepting the actual API calls the browser is already making is closer to testing the contract, not the presentation.
The healthcare context makes this especially sharp. Those portals are notoriously inconsistent and rarely built for automation. Would be curious how you handle cases where the app uses WebSockets or chunked responses instead of clean REST calls.
Right now libretto only captures HTTP requests, which the coding agent can use to determine how to perform the automation.
For more complex cases where libretto can't validate that the network approach would produce the right data (like sites that rely on WebSockets or heavy client-side logic) it falls back to using the DOM with playwright