I'm pretty sure it doesn't work like that: previously there were genetic variants for large and small sizes. Now a high proportion of the larger variants have been removed from the gene pool. To reverse that in a similar time frame would need a massive selection pressure on the smaller variants. So you would need something like industrial scale fishing that removed the smaller adults, but didn't take out the juveniles who would become big fish, while also letting go the bigger adults. It doesn't sound feasible