> (FD: I work at AA)
I do apologize in advance for putting you in spot like this, but there are two things I have been quite curious about.
1. Since 60% of the benchmarks contributing to AA are now private only, how do you guys ensure transparency? More precisely, how can someone verify the transparency in scoring?
2. There are also disclosures about custom benchmarking services offered to AI labs, what does AA do to keep the custom benchmarking services not getting access to the same/similar data that is used in the private benchmarks?