It's not the rendering speed, per se. There's no API to get IP addresses after requests, and fetches of cached requests leave their IP field blank. A manual refresh forces a new actual request and thus fetch of the IP, at which point it works. If you're getting the globe when a refresh will cause it to change to a flag, then either Flagfox was loaded before the page was or it somehow got confused and forgot a cached IP, somehow.

Due to the API limitations here, I've got to listen to IPs ahead of time and cache them for when needed. Caching is easy; caching without keeping everything cached forever and becoming a giant memory leak, is more difficult. Flagfox 6.0.1 has some cache improvements, with more likely to come in future updates. TL;DR: Working on it.