> If basically everything is <X>, does that word mean much then/is it a useful term?
 

Let's let X = "chair", to make my point a little more fun.

Obviously we know what a chair is, use the Potter definition[0]. Everyone has several of them in their houses, offices, and all around. But can you state a well defined definition? Is a table a chair? I can sit on it (I often do), it has four legs, and practically all things we'd consider a chair. Okay, it has no back (usually), but then we got to throw out the stool. It has no back. It doesn't even have to have 4 legs! So what about that stump on the ground? Is that a chair? I think most people would argue through a definition of convenience. Or maybe closer to a cat's definition of "if I fits, I sit."

You can replace X with just about anything and you'll find this ambiguity runs rampant. A cup, a table, cereal, sandwich, a tomato; you name it and I'm sure someone here will point out an instance that makes that definition look a lot fuzzier than you initially thought.

So that really shows us the crux of it all: how well defined is something. That only tells us our ability to distinguish something, though it doesn't tell us how well that thing fits into the category. So the stump is a little chair-like, the table is pretty chair, a stool is usually considered a chair, and that thing I'm sitting on is unambiguously a chair[1]. s/chair/intelligence/g and you have your answer. But the if you needed that last sentence, well, there's two types of people: those that can infer.

All models are wrong, but some models are useful

[0] Incidentally this is also a good definition for explaining RLHF and has all the same problems: https://en.wikipedia.org/wiki/I_know_it_when_I_see_it

[1] Though that doesn't mean it also isn't other things!

Simple chemistry couldn't be considered intelligent- it's basically just falling down a potential energy gradient. It gets more interesting when you implement chemical logic (like transistors) and graphs of molecules that convert between each other, like autocatalytic networks.

Good question. It's a useful but incomplete term. Just like white is incomplete. There are always things to be found more white.

Yes, words that label parts of continua are useful, even if they don't have well defined boundaries.

Kind of amazing how simple that fact is and how many arguments are caused by people not realising that it's true.

Here are some examples: town, artery, yellow, loud, tall, kind, woman, road...

Honestly it's hard to think of words that do not exist on a continuum. The exceptions are mostly low level maths/physics things.