• English

      • AI Gateway

      Estimated reading time: 2 minutes 预计阅读时间: 2 分钟

      Full Chinese version: /guide/core/ai-gateway.

      Sub-crateRole
      summer-ai/coreProtocol core types & traits
      summer-ai/modelSeaORM entities + DTO/VO
      summer-ai/relayRelay engine + OpenAI / Claude / Gemini routers
      summer-ai/adminAI admin APIs
      summer-ai/billing3-stage billing
      summer-ai/agentrig-core agent (optional)

      Three protocol entrypoints

      ProtocolPathAuthError flavor
      OpenAI/v1/chat/completions, /v1/responses, /v1/modelsAPI key{"error": {...}}
      Claude/v1/messagesAPI key{"type": "error", ...}
      Gemini/v1beta/models/{target}API keyGemini-style JSON

      Each protocol has its own ApiKeyStrategy::for_group(_, ErrorFlavor::*) and *_panic_guard so failures stay in the matching shape.

      6-dim routing

      DimensionTablePurpose
      Protocol familystatic enumOpenAI / Claude / Gemini
      Endpointai.routing_targetchat_completions / messages / etc.
      Credentialsai.channel_accountupstream API keys / OAuth tokens
      Model mappingai.model_configclient model → upstream model
      Extra headersai.routing_rule.headersorg id, project id, etc.
      Strategyai.routing_rule.strategyround_robin / weighted / priority / fallback

      3-stage billing

      Reserve avoids over-charging under concurrency; Settle reconciles to real usage; Refund covers upstream failures.

      Hot reload

      Everything that drives routing lives in DB tables (ai.channel, ai.channel_account, ai.routing_rule, ai.routing_target, ai.model_config, ai.channel_model_price, ai.token, ai.user_quota). relay/src/service/channel_store.rs polls and refreshes on a fixed interval — no restart required.

      Request log

      ai.request_log captures request_id, flavor, endpoint, client/upstream model, channel id, prompt/completion tokens, latency, status, attempts, and cost. Available via /api/ai-admin/request-log.

      Calling examples

      # OpenAI
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer sk-xxx" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}'

# Claude
curl -X POST http://localhost:8080/v1/messages \
  -H "Authorization: Bearer sk-xxx" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"claude-3-5-sonnet-latest","max_tokens":1024,"messages":[{"role":"user","content":"Hello"}]}'

# Gemini
curl -X POST "http://localhost:8080/v1beta/models/gemini-1.5-pro:generateContent?key=sk-xxx" \
  -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"Hello"}]}]}'

      Source files

      • crates/summer-ai/relay/src/router/{mod,openai,claude,gemini}.rs
      • crates/summer-ai/relay/src/{auth,service,convert,extract,pipeline}/
      • crates/summer-ai/admin/src/router/ (15+ files)
      • crates/summer-ai/billing/src/