You can't take notes before you've learned language, so if that's how you propose to learn then you are doomed to build something with less learning capability than an human brain.

Maybe you don't care - if you just want to push to see how far an LLM can go that's fine. but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.