Yeah, you really need to structure your code around working with the tilemap system.

I did a small racing prototype with both vertical and horizontal scrolling and segmented my updates to 4x4 blocks of tiles per-frame (160x144 resolution so 20x18 of 32x32 tiles is visible at any point in time, so stippled updating 4x4 blocks outside of view is within the budget together with updating some of the tiles each frame)