alt Hacker NewsI put my whole life into a single database
Comments
A simple back of the envelope calculation shows that Felix causes between 70 and 110 tonnes of CO2 emissions per year just from flying.
Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.
In my experience, tracking objective things like "nutrition" and "sleep hours" is immensely useful to reflect on what went wrong, and tracking subjective things like "mood" or "stress" is useless given hedonic adaptation or heavy swings that make problems obvious, and not need tracking.
What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.
I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.
As someone who dislikes being tracked, I find it disconcerting to have this level of data stored in 3rd party services, but joining up multiple services to give one coherent picture is cool. How cool is questionable, trying to correlate health data with location data is going to give a strange picture, and I question how relating health data to weather is useful. (Do I have lower blood pressure on rainy days?)
Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.
Flying stats dashboards always amuse me. I get it, for the non-pilot it's kind of like a status thing, "Well I traveled X times in 2025!". As a pilot though, I have gobs of stats I could put up there but from flying for 15 years I realize there's not really anything meaningful in there other than "Gee whiz, I flew a little more/less than last year." I know some other professional pilots do track some of their stats a little closer as they try to optimize for hours flown:hours paid but I've never cared to hyper optimize my schedule in that way.
Why don’t you just query Palantir DB by your human ID? It shows your entire life data and much more.
I get that everyone wants to be cynical about this, but you really can't deny that both the visualization and sheer scale of data is impressive. The way the "my life in weeks" is done is also very cool, I'll be stealing that for myself.
The hardest part of this kind of personal data system is retrieval not storage. At some point you have more data than fits in a prompt, so you need to decide what's relevant per query. Did you build any ranking or filtering logic, or do you query specific tables directly?
> Probably obvious for many, but I didn't realize ACs don't transport any air into the room, but just moves it around
I had the same epiphany as you days after acquiring a CO2 monitor. Most people notice poor indoor air quality from proxies such as humidity and temperature. AC (without ventilation) eliminates these and tricks our senses very effectively, giving us cool and fresh feeling indoor spaces full of CO2 and devoid of oxygen.
The value rarely shows up where you expected it to.
I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.
Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.
How do you sync the steps data from iOS to your own server automatically?
This looks like it requires a heavy amount of discipline to track everything consistently over time. How do you build that into your daily routine?
I store my travel in YAML files in git.
https://edwardbetts.com/agenda/trip/past https://edwardbetts.com/agenda/trip/stats
> Back in 2019, I started collecting all kinds of metrics about my life. Every single day for the last 3 years I tracked over 100 different data types - ranging from fitness & nutrition to social life, computer usage and weather.
I know this is the type of person i would not like.
This was far more interesting than I first thought it would be when clicking the link. In particular, the place/time and life events and such being presented this way told a story and was fun.
"GitHub Open Source Contributions" aside, everything else falls into the consumption/intake/internal categories; thin data on production/output/external ones.
An interesting experiment, I think I'm too uncomfortable leaking data I don't yet know why someone would curate to me free of charge until I knew. If there was a FOSS suite like Home Assistant that would do a few of those things I might try it out, especially the weather (I would add air quality) correlation to mood and other subjective states.
I thought you created a database from scratch, got me excited! (I’m a db guy)
This is fascinating. One question I always have with large personal datasets like this: at what point did you start getting genuinely surprising insights versus confirming things you already suspected?
380k datapoints sounds incredible but I imagine the real challenge is turning that into decisions that actually change behavior
The step count in NYC stands out like One World Trade Center compared to the rest of the building when looking at the skyline ;-)
Wow, this sounds amazing! I am a statistics and chart lover, I love tracking various types of data, but this? I can't even imagine how much time it must have taken to input all the data. Huge respect to you! Keep going.
I'd like to make the same but with Owntracks instead of Swarm and ActivityWatch instead of RescueTime.
I wonder how much time you spend daily on tracking things / data entry
Taking “Know thyself” to a whole new level. I’d love to have these stats on me, if it could be done by inference, rather than conscious effort.
Related to this, I highly recommend anyone to install github.com/ActivityWatch/activitywatch, it's an amazing tool to keep track of your computer use completely locally. I think there are lots of possibilities with data analysis/AI aimed to improved one self's life
No need to spy on him, he publishes everything.
I had an idea similar to this where you could add information about yourself and answer daily questions and get paid by companies who access this data. This could be an ethical way to share information benefitting all parties.
It's all nice and all, but I'm just sitting here thinking "How can one afford all this flying around"?
Dude. Backup that database.
Hope they made backups :)
Bro queried his entire database, added a LIMIT 10 and it only produced 2 rows. Just a little humor in good faith.
[dead]
So it didnt end up working too well seeing that the latest data is from 4 years ago.
This might sound harsh but for someone who is keen on investing time to track so many things, you should invest some time in learning how to make better visualizations. A few tweaks here and there would really improve what you have.
Yeah, we've all put our whole lives into a single database. It's called the United States Government.
The takeaways at the very bottom of the page are valuable:
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
The whole "quantified self" movement might be more about OCD and perfectionism than anything else.
/edit: quantified, not qualified