alt Hacker NewsShow HN: Screen memory without screenshots, just text to Markdown
It's a macOS menu bar app that reads the text of your focused window every few seconds through the Accessibility API. No screenshots, no video, or OCR. It writes plain markdown, one file per day, into a folder you choose.
Point Claude Code (or anything with file access) at the folder and ask "what did I work on Tuesday?" or have it build memory about your projects. The folder contains an AGENTS.md explaining the format to the model.
Thought it might be useful for someone so chucked it up on GitHub
Comments
Hmm. What would be really helpful for me would be an enhancement of the TypeWhisper app for dictation that would be able to get the context of what I am dictating into and send it along with the prompt to an LLM.
Probably much simpler and much less of a privacy problem (I run my own local LLM for that purpose so that nothing leaves the machine).
> reads the text of your focused window every few seconds through the Accessibility API
> It writes plain markdown
Where are the formatting decisions coming from?
neat. why not screenshot and tesseract (videos/images/viewport/etc)
I experimented with this exact same approach earlier this year.
It's barely sufficient, because, bluntly, most apps just aren't wired up right.
So you end up having to hand code a lot of specific profiles for specific apps to make this work well, and even then, you don't quite get the right level of detail to make it work out.
Will try this app, to see if it improved on my own approach, but man, the hope levels are low.
[dead]
One app that's using this technique (not exactly sure if it's the same) is Littlebird: https://littlebird.ai/
I also saw that HeyClicky started doing something similar but end up removing from the product.