AsyncImage Looks Like Free Image Loading. It Isn't Caching Anything.

I put a photo grid in a SwiftUI list once, thirty rows, one AsyncImage per row, and called it done. Then I scrolled down, scrolled back up, and watched every image I’d already loaded flash back to a gray placeholder before reloading. Same URL. Same image. Downloaded twice, three times, however many times a row happened to leave and re-enter the screen.

AsyncImage looks like it should just handle this. It has a URL, it fetches, it shows the picture — that’s the whole pitch. Caching feels like it should be bundled in for free. It isn’t, and once you know why, the flicker stops being surprising.

---

What AsyncImage is actually doing

AsyncImage is a view, not a cache. Every time SwiftUI creates a fresh instance of it — which, in a List or LazyVStack, happens constantly as rows scroll offscreen and get recycled — it starts from a blank AsyncImagePhase and kicks off a brand new URLSession data task. There’s no static store of “images I’ve already fetched” living anywhere. The download that finished thirty seconds ago when the row first appeared is completely forgotten the moment that row’s AsyncImage instance gets torn down.

AsyncImage(url: brew.photoURL) { phase in
    switch phase {
    case .empty: ProgressView()
    case .success(let image): image.resizable().scaledToFill()
    case .failure: Image(systemName: "photo")
    @unknown default: EmptyView()
    }
}

This is the textbook example from every SwiftUI tutorial, and it works fine for a handful of images shown once. Put it in a scrolling list and you’ve built a view that refetches its own content on every reappearance — which is exactly the same shape of bug as SwiftUI silently deciding a view is a brand-new one and dropping its state. Different mechanism, same lesson: SwiftUI recreating a view is not a rare edge case, it’s the normal cost of doing business in anything scrollable, and any state you assumed would “just persist” needs somewhere to live that isn’t the view itself.

There is one layer of caching involved here, and it’s easy to overrate it: URLSession.shared uses the default URLCache, which does cache HTTP responses according to their cache-control headers. If your image host sets sane caching headers, a re-fetch can be a cheap disk-cache hit instead of a real network round trip. That helps with network cost. It does nothing for the visible flicker, because AsyncImage still throws away the decoded image and rebuilds its view state from .empty first — you’ll get the placeholder-then-fade every single time, even on a cache hit that took two milliseconds.

---

The fix: cache the decoded image yourself, above AsyncImage

The actual fix is boring in the best way: keep your own in-memory cache of decoded images, keyed by URL, and check it before ever handing AsyncImage a URL to fetch.

@MainActor
final class ImageMemoryCache {
    static let shared = ImageMemoryCache()
    private let cache = NSCache<NSURL, UIImage>()

    func image(for url: URL) -> UIImage? {
        cache.object(forKey: url as NSURL)
    }

    func store(_ image: UIImage, for url: URL) {
        cache.setObject(image, forKey: url as NSURL)
    }
}

struct CachedAsyncImage: View {
    let url: URL?

    var body: some View {
        if let url, let cached = ImageMemoryCache.shared.image(for: url) {
            Image(uiImage: cached).resizable()
        } else {
            AsyncImage(url: url) { phase in
                if case .success(let image) = phase {
                    image.resizable()
                } else {
                    Color.secondary.opacity(0.15)
                }
            }
            .task(id: url) {
                guard let url, let (data, _) = try? await URLSession.shared.data(from: url),
                      let uiImage = UIImage(data: data) else { return }
                ImageMemoryCache.shared.store(uiImage, for: url)
            }
        }
    }
}

NSCache is the right tool here, not a plain dictionary — it evicts entries under memory pressure automatically, which a hand-rolled [URL: UIImage] dictionary won’t do, and it’ll happily grow until your app gets killed for it. The .task(id: url) modifier matters too: keying the task to the URL means SwiftUI cancels and restarts it correctly if the same view instance gets recycled for a different row’s URL mid-flight, which is exactly what happens during fast scrolling in a recycled list.

Once an image is in the cache, re-scrolling past its row is a synchronous dictionary lookup and an immediate Image — no placeholder, no flash, no second network request even on a cold URLCache.

---

Where this still isn’t enough

A view-level memory cache solves the scroll-flicker problem, but it’s still eager rather than smart. It fetches when the view appears, not before, so the first pass down a long list still shows placeholders in whatever order rows happen to appear on screen. If you want actual prefetching — kicking off downloads for rows about to scroll into view, before the user gets there — you’re building a request queue with priorities, which is a meaningfully bigger piece of infrastructure than a cache dictionary.

It’s also memory-only. Kill the app and the cache is gone; the next cold launch refetches everything, URLCache’s disk layer notwithstanding. For most feeds and grids that’s a fine tradeoff — a two-frame flash on cold launch is a very different problem than one on every scroll. If it isn’t fine for your app, that’s the point where reaching for something like Nuke or Kingfisher stops being overkill and starts being the sane choice: disk persistence, request coalescing, and downsampling are all things those libraries have already gotten right, and re-deriving them by hand rarely pays for itself.

---

The takeaway

AsyncImage handles exactly two things: issuing the request and rendering whatever phase it’s currently in. It was never going to remember what it already downloaded — that was never its job. The moment you put it somewhere views get recreated on the regular, which in SwiftUI is nearly everywhere, the cache has to live one layer up, in something that outlives the view. A dozen lines of NSCache wrapper is usually the entire fix.