Unfortunately, anti-androgens have myriad effects beyond basic T suppression.

There are two primary drivers behind why anti-androgens would cause loss of libido beyond effect on T:

1) AA's cause androgen receptor blockade systemically. This blocks action at the AR that would be residual across systems from adrenal production. Most important for libido are the AR activity that occurs inside of neurons and astrocytes in the brain

2) AA's have a two-punch effects on the Prolactin/Dopamine system + Progesterone system. Chronically elevated prolactin causes down-regulation of dopamine, which by itself is enough to kill libido. Progestins modulate GABA, which can cause "flat affect" and "emotional flatlining".

The combo punch of neuronal/adrenal AR blockade + Prolactin/Dopamine dysregulation + GABA dysregulation would require a miracle to have preserved libido on.

Yeah, your comment squares with (and the GP's point #2 contradicts) what I learned in my college Science & Gender class, which was a combined neuroscience/psychology offering where we read a bunch of papers. Most of them supported that testosterone was the primary driver of libido in both men and women, with higher T levels corresponding to higher sexual desire and lower T levels corresponding to the opposite.