That's not how copyright works. There is no "use it or lose it". You legally retain control of licensing for copying and distribution, including derivative works (samples) for the full copyright duration.

In this case, it's a music recording published in the USA between 1957 and 1972, so the copyright expires on 15 February 2067: https://library.mi.edu/musiccopyright/duration

There's a limit of 3 years for you to bring a copyright infringement case after a specific infringement takes place, but that doesn't stop you from bringing further cases against further infringements. You don't "lose" copyright if you fail to self-police it, as you might lose a trademark.

It gets very complicated around this specific 1969 sound recording and suing for copyright infringement, because prior to 1972, sound recordings were subject to US state law (presumably Georgia as the Holly Bee Music Co. was incorporated in Atlanta?) rather than federal law.

Ultimately, no samplers paid The Winstons any royalties. https://en.wikipedia.org/wiki/Amen_break#Royalties

The drummer, Gregory Coleman, died homeless and penniless. Some UK DJs raised money for the bandleader, Richard Lewis Spencer, in 2015. He died in 2020. So there's probably not going to be anyone suing you for infringing the copyright on Amen, Brother, but it's not not infringement.