My libertarian view on discrimination (independent of the Civil Rights Act) is this:

If a service is not widely available in the region, any systematic discrimination leading to refusing to provide service, or specific level of service or care, based on anything unrelated to the ability to provide it, should be illegal, locally, in that community. Rules like ousting disruptive customers apply across the board.

If a service is widely available, however, then “x-only” service providers should be allowed to operate (as indeed they are with women-only gyms, Jewish-only clubs, or nightclubs that let women in first and charge the men) as long as they advertise it up front and not make people go there only to find out that “ladies can go in free of charge, men pay $300 for a table with bottle service”

PS: replace “ladies” and “men” with “whites” and “blacks” and hear how that sounds. And no, citing crime or violence statistics shouldn’t play a role in shaping whether people can get into places, whether it’s women citing male vs bear violence / harassment or people citing racial FBI statistics on violence / harassment. This is the prosecutor’s fallacy.