You can do it a bit faster by not quicksorting the entire array, as you don't really care about the order of the lowest and highest numbers as long as they are not close to the middle.

There is also an approximate algorithm that does not keep all the data in memory at the same time.