Look back to the olden days of "drag proxies" or outline/wireframe drags.

Specifically for your example of maps, I would stick mostly to border buttons or UDLR gestures to "page" around rather than than trying to show a smoothly refreshing drag area.

Since you can _see_ the screen refreshing/drawing, and assuming you're using +/- buttons instead of "pinch to zoom", try drawing animations "in layers", ie: spend 100-300ms slapping "zoom tunnel boxes" trying to keep the user oriented while interacting.

Break the tiles into "thick => thin => letters => detail" and wait for them to quiesce before stacking more layers.

Rule #1 of optimization, you can never do something faster, you can only "do less"!

Rule #2 of optimization (amdahls law?)- optimize the slowest parts before you optimize the faster parts (and there's a limit to each!).

...from an end-user perspective, I would love if you respected my time and concentration when doing something "interactive".

Imagine scrolling through an e-ink address book. Imagine you spam only the (properly placed / laid out) first letter of first name and first letter of last name, and then after ~500ms of no movement, only THEN write out the remaining letters, phone numbers, icons, etc.

For e-ink interactive: every (slow, expensive) pixel that you write that gets scrolled off the screen is wasted time.

#1 - do less. In interactive cases, draw less pixels.

#2 - keep context. Use proxies/wireframes, "first letters", and "bold strokes".

#3 - fill in layers. Think "iterative Mona Lisa" and "progressive jpeg"

#4 - actually being slower is okay if you end up "feeling faster" or "seeming cooler"

Lean in to the defects and quirks of e-ink, make them feel intentional rather than trying to duplicate 60fps 16M colors.