Sending Images to a Model From a Small Container

Adding vision to an app is a small code change and a large memory change. There are two documented ways to hand a model an image — "by providing a fully qualified URL to an image file" or "by providing an image as a Base64-encoded data URL" (OpenAI docs) — and on a small container the choice is mostly about memory rather than convenience. The API will accept far more than you can hold: the documented limits are up to 512 MB of payload and 1500 images per request, against a container with 0.25 GB of RAM at a small preset. The provider's limit is not your limit, and the gap is where apps get killed. Resize before sending, prefer a URL when the image is already hosted somewhere stable, and never hold the original and the encoded copy at the same time if you can avoid it.

The memory arithmetic#

Base64 is not a compression format. It encodes three bytes into four characters, so the encoded form is about a third larger than the file. That's before your HTTP client copies it into a request body.

Walk through a single 6 MB photo:

• The original, read into memory: 6 MB
• The base64 encoding of it: about 8 MB
• The JSON request body containing that string: another copy, unless your client streams

That's roughly 20 MB in flight for one image, and it's transient — which is exactly the kind of allocation that looks fine in testing and kills you under concurrency. Two users uploading at once triples it, and your RAM limit is a hard ceiling: cross it and the process is killed outright rather than slowed, which surfaces as an app that vanished without an error.

This is the general file upload problem with an extra copy bolted on.

Resize before you send, always#

The single highest-value line of code here, and it helps twice.

It cuts memory proportionally to the pixel count — halving each dimension quarters the bytes.

It cuts cost. The docs are explicit that image tokens scale with size: high-detail processing scales the image "to fit in a 2048px x 2048px square" and charges per 512-pixel tile, and for newer patch-based models "large images can use more input tokens than they did with earlier models." Sending a 4000-pixel photo when the model will scale it down anyway means paying to transmit detail that gets discarded.

The docs name resizing as a cost control directly, alongside choosing a lower detail level. Do both. For most tasks — reading a receipt, describing a scene, classifying a screenshot — a modest resolution is enough, and you can test that against your eval set rather than guessing.

Resize on disk with a streaming image library rather than loading the full bitmap where you can. A decoded image is far larger in memory than its compressed file, which is the step people forget.

URL or base64?#

Pass a URL when the image is already hosted somewhere public and stable. Your app never holds the bytes at all — the provider fetches it directly, and your memory cost is a string. For a small container that is a large win.

Send base64 when the image is private, temporary, or something the user just uploaded and you don't want to publish.

The trap in the URL approach is that the provider has to be able to fetch it, later, from their network. That rules out anything behind your own authentication, and it rules out URLs that expire — which is precisely the situation with Discord attachment URLs, which carry expiry parameters and stop working. Passing one to a model is a race you will eventually lose, and the failure will be intermittent.

If you're building a bot that analyses images people post, download it yourself, resize it, and send the bytes. It costs memory, and it works every time.

Don't keep what you don't need#

Two habits, both about the same thing:

Write temporary files to disk, not memory, and give each one a unique name. A fixed path like /tmp/upload.jpg collides the moment two people use your app at once — the same single-user assumption that breaks generated code.

Delete originals once you have the answer. If your app stores every image it processes, that's growth with no ceiling, and disk is measured rather than allocated with an account-wide limit. Keep the model's output, which is small; discard the input, which isn't. If you genuinely need the originals, that's a decision to make deliberately and put on a volume.

Image requests are slower#

Uploading megabytes and processing them takes longer than a text prompt, so the timeouts matter more.

Set an explicit per-attempt timeout rather than inheriting a generous default — an SDK default measured in minutes will hold a worker for the duration. And be careful about retries: retrying an image request re-uploads the whole payload, so a retry storm is bandwidth and memory as well as money.

If a person is waiting, acknowledge first and do the work after. If they're not, be generous with the deadline and strict with the concurrency — one image at a time is a perfectly good design on a small container.

FAQ#

How do I send an image to a model?#

Either give a fully qualified URL the provider can fetch, or send the image as a Base64-encoded data URL.

Why does my app crash when users upload images?#

Base64 is about a third larger than the file, and you typically hold the original, the encoding and the request body at once. That transient spike crosses a hard memory limit and the process is killed.

Does the 512 MB payload limit mean I can send big images?#

Not on a small container. That's the provider's limit; yours is the RAM you allocated, which is far lower.

Should I resize images before sending?#

Yes. It reduces memory proportionally to pixel count and reduces tokens, since image cost scales with size and large images are scaled down anyway.

Can I pass a Discord attachment URL to the model?#

Not reliably. Those URLs expire, so the provider's fetch can fail later. Download, resize and send the bytes instead.

Should I store the images my app processes?#

Only if you need them. Keep the model's output and discard the input, since stored images grow without limit and count toward the account-wide disk limit.

Wrapping up#

Vision features expose the difference between what an API permits and what your container can survive, and the gap is wide enough — 512 MB against a fraction of a gigabyte — that the API's limits are no guide at all. The whole discipline fits in a sentence: make the image smaller before it enters your process, avoid holding two copies of it, and prefer letting the provider fetch it when that's genuinely an option. Do that and images become an ordinary feature rather than the thing that kills your app on a busy afternoon.