I've built something very similar but am further along (https://github.com/stravu/crystal). I understand the desire to do your own thing, but if you are interested in joining forces and contributing I would love to have you. I think we were thinking along very similar lines.
I tried your project in my exploration and ran into an issue but I can't quite remember what it was. Anyways, I would be interested in connecting and learning more about your project, I dropped my X handle in the first issue on my Github repo