alt Hacker NewsHow do you avoid downloading the whole haystack to search through the data? How does the hierarchy work? I have to keep a bunch of .txt files updated in my web root? Doesn't this require essentially mirroring the inventory db as text files (if the intent is for accurate counts of items, etc they would need to be updated in real time)?
You do not download the haystack. You traverse it. The architecture is fractal. The agent reads the Root. If the user wants "Headphones", it follows that specific link. It ignores the rest. It is lazy loading for context. Do not mirror your DB manually. For real stores, generate the files dynamically. It is a view layer, just like HTML or sitemap.xml. Real-time? Yes. Since it is a dynamic response, it reflects the DB state instantly. Cache-Control headers handle the freshness.