logoalt Hacker News

StackOptimisttoday at 2:41 AM0 repliesview on HN

The risk in relevance-based pruning is the same one summarization has: it's tuned to drop whatever's rare, and in a lot of domains the rare chunk is the whole answer. A contraindication, an "except when," the single row that contradicts the other forty. Those score low against the query precisely because they're phrased in the exception's language, not the question's, so a similarity cut throws them out first. The strongest version of the pro-pruning case is "prune by relevance to the query, not blindly," and I still don't trust it where being wrong is expensive, because relevance-to-the-query is exactly the signal that misses the buried caveat. If I ask whether drug A interacts with drug B and the warning is written under B's contraindications without naming A, semantic pruning helps me lose it. What's worked better for me is pruning by structure instead of by score: keep whole records or whole sections as units and drop by type (this entire category of source isn't relevant) rather than by ranking individual sentences. You give up some of the token savings of aggressive sentence-level pruning, but you stop silently deleting the one clause the answer depended on. If you must prune fine-grained, the honest test is recall on a set where the answer hinges on a rare chunk, not average-case QA where there's lots of redundancy to spare.