You're right - there isn't one way to do things or one way that always works. There are issues that are outside a team's control, for example, that make it an impossible struggle. If you're rigidly forced to do something and cannot adapt then you're ... not really agile. But you have to understand why you're doing what you're doing and keep adjusting to see what works for you.
#1 IMO is that if the company you're in is non-agile in its general attitude, which is influenced by its own customers, then everything is geared against you.
That isn't to say that something like Kanban might not be usable or better than no plan at all but certainly scrum is not some universal solution.