Those are completely separate concepts. Enslaved people are very much still agents in the sense used here. An agent is simply any entity that interacts with the environment in a way that's not fully determined by other parts of the environment (at least, not in a way that is very easily observed/derived).

That is, a falling rock is not an agent, because its movement is fully determined by its weight, its shape, the type of atmosphere, and the spacetime curvature. An amoeba in free-fall is likewise not an agent, for the same reasons. But an amoeba in a liquid environment is an agent, because its motion is determined to at least some extent by things like information it is sensing about where food might be available, and perhaps even by some simple form of memory and computation that leads it to seek where food may have been available in the past.