Criminals need to be isolated from peaceful society anyway, and if we’re going to the expense of warehousing them, the least we can expect is for them to contribute something back. The ethical justification for prison labor is obvious and the analogy to heritable chattel slavery is ludicrous.
Society is not peaceful. It’s just that certain kinds of violence are legal.