Vision inputs

Send image URLs or inline images to a vision-capable model.

For a vision-capable model, send a user message containing text and image content parts. The endpoint remains POST /v1/chat/completions.

Check support first

Model capabilities are separate from the HTTP request format. A text model does not gain vision support because an image object was included in its request.

Set SIMPLELLM_MODEL to a model whose vision support you have verified for your application. This guide deliberately does not label every chat model as vision-capable.

Send an image URL

The image must be reachable by the model service. A URL on localhost or your private network will not generally be accessible from the service.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.simplellm.eu/v1",
    api_key=os.environ["SIMPLELLM_API_KEY"],
    max_retries=0,
)
response = client.chat.completions.create(
    model=os.environ["SIMPLELLM_MODEL"],
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe the visible objects in this image."},
            {"type": "image_url", "image_url": {"url": os.environ["IMAGE_URL"]}},
        ],
    }],
    max_tokens=2048,
)
print(response.choices[0].message.content)

Use a direct image URL, not an HTML gallery page. Avoid credentials in URLs; use inline input for private images if the model's backend supports it.

Send a local image inline

Replace the image URL with a data URL containing the matching MIME type and base64 bytes. This example assumes a JPEG file:

import base64
from pathlib import Path

encoded = base64.b64encode(Path("photo.jpg").read_bytes()).decode("ascii")
image_part = {
    "type": "image_url",
    "image_url": {"url": "data:image/jpeg;base64," + encoded},
}
# Use image_part in the content array of the previous request.

Do not label PNG data as JPEG or pass a local filename as the remote URL. Inline data increases request size; resize unnecessarily large images before sending them.

Multiple images and detail

Where supported by the selected model, multiple image_url parts can accompany a question. Preserve the intended order and explicitly tell the model which image you are referring to.

The optional image_url.detail field uses values such as auto, low, or high, but support and its effect are backend-dependent. There is no single documented V1 image-count or resolution guarantee across all models.

Diagnose incorrect results

If the model ignores the picture, first confirm model support and that the image is reachable and decodable. If the service rejects the request, inspect the error and request size; do not repeatedly resend the same invalid payload.

Text extraction through a vision chat request is not a separate OCR API. The V1 public router documented here does not expose the direct GPU-host OCR endpoint.