What you're saying can only be true to the extent that natural language doesn't look like formal logic.
Unfortunately, there are some obvious discrepancies. My favorite is that "can't" really means something closer to "not can".
This can be demonstrated with a close analysis of the statement, "I can't not do that." To get our usual understanding of the sentence we need to parse it as, "I (not can) (not do that)." And then turn that into, "I must not (not do that)." And now cancel the double negative to get, "I must do that."
Suppose that you try to parse it as, "I can not not do that." You quickly get, "I can do that." Which is not at all what that sentence actually means.