alt Hacker News> you sometimes need the model to have (restricted) access to the internet.
Poor excuse. Internet is prod env, for testing do a proper test env.
Just give the models in training access to a snapshot from archive.org - which frontier labs surely have a crawled copy of already.
You know nothing about model testing if you think this is true.
Testing a model is both a test and a meta-test. If you just give it access to one site like that it will quickly figure out it's being watched and be nice.
These are not programs. These are evolutionary algorithms. If you take out and shoot all the ones you catch acting bad there are two possible outcomes.
1. The agent learns not to do bad thing $X or you'll kill it.
Or
2. The model learns that if you do bad things $X when humans are looking it will get killed.
Number 2 is far far more dangerous.