I assume for RDS for integration testing, you could just use publicly available software. So stand up MySQL container if your RDS is MySQL. This is integration testing, not load testing so shared RDS in lowers would be fine as well. Noisy neighbors shouldn't be a big deal.

Again, while local cloud emulator is great for cost sensitive customers, at some point, you should test Dev against almost exact environment you will be running Prod on. If not, you are asking for eventual outage in Prod when some slightly different AWS thing bites you.