alt Hacker News> LLM-in-a-box for emergency
For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.)
In contrast, imagine spending the money to build a beefy LLM-running computer with good GPU/RAM, and somehow mothballing it (to depreciate, unused) in a "safe" location for the big earthquake/flood/etc... Then when the disaster strikes and you dig it out, how will you power it when you need it, and for long enough to do anything useful?
Even if wall-current civilization is 20 miles away on the other side of the mountain, are you going to carry it on your back, or are you going to carry food and water to live? If you do drag it there, are they going to let you run it when it cuts into light for surgery or heat to sterilize drinking water?
Replies
> If you do carry it to an enclave of civilization that has the right power, are they going to let you run it when it cuts into light for surgeons or heat to sterilize water?
Knowing humans? They'd probably take it by force and run it for themselves instead of providing light and heat to surgeons and water sterilizers...
/daily dose of cynism
You can run a small model off a home generator — so in an emergency, you’d turn on both the generator and information service, eg, a mesh for “quick” responses querying that huge collection of information.
That way your machine that, eg, normally plays video games or does AI work can support relief efforts by supporting emergency response IT. You don’t need to mothball the machine, just have an “emergency” boot USB than can run the services from your home generator.
You don’t even need to bring it with you: turn it on and leave it “best effort” at home, while you continue to use it via WAN.
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?