#AI Gateway Integration

Summerrs Admin can serve as the admin backend foundation for an AI gateway. It provides login authentication, RBAC, menu buttons, resource permissions, dictionaries, operation logs, and MCP tools that can support AI channels, models, API keys, usage, audit, and operations.

An AI gateway usually has two parts:

Admin APIs fit naturally into the /api system, reusing admin JWTs, button permissions, resource permissions, and ApiResult responses. Relay protocol entrypoints should be mounted as a separate route group, such as /v1/chat/completions, /v1/messages, and /v1beta/models/*, with API keys, model routing, streaming responses, and protocol-specific error models.

#Reusable Admin Capabilities

The AI gateway admin side can reuse Summerrs Admin infrastructure:

This lets the AI module focus on model-gateway domain logic while the admin foundation stays consistent with the rest of the system.

#Recommended Module Boundaries

AI admin APIs and relay protocol entrypoints have different auth, error, and response models. Keep the modules clear:

crates/
  summer-ai-core/        # protocol types, error models, shared traits
  summer-ai-admin/       # /api/ai/* admin APIs
  summer-ai-relay/       # /v1/* protocol entrypoints and upstream forwarding
  summer-ai-model/       # SeaORM entities, DTOs, VOs
  summer-ai-billing/     # quotas, billing, request logs

The main app can register both admin and relay plugins:

app.add_plugin(SummerAuthPlugin)
   .add_plugin(ResourcePermissionPlugin)
   .add_plugin(AiAdminPlugin)
   .add_plugin(AiRelayPlugin);

Design the two entrypoint families separately:

This prevents admin login state, button permissions, API keys, streaming responses, and third-party protocol errors from being mixed into one request chain.

#Admin Features

An AI admin backend usually includes these pages:

Admin handlers can follow the existing system style:

#[log(module = "AI 渠道", action = "创建渠道", biz_type = Create)]
#[has_perm("ai:channel:create")]
#[post_api("/ai/channel")]
pub async fn create_channel(...) -> ApiResult<()> {
    // ...
}

Menu buttons can be organized by business object:

ai:channel:list
ai:channel:create
ai:channel:update
ai:channel:delete
ai:model:list
ai:model:update
ai:token:list
ai:token:create
ai:token:disable
ai:request-log:list
ai:usage:list

If you want the resource-permission layer to apply, register AI admin APIs in sys.resource and bind them to the corresponding Button through sys.action_resource. You can reload the policy with:

POST /api/system/resource-permission/reload

#API Key Auth

Relay callers are programs, external applications, or agents. Use a dedicated API key strategy instead of admin JWTs.

Common request format:

Authorization: Bearer sk-xxxx

An API key should contain at least:

The auth flow can be:

parse Authorization -> hash lookup -> check enabled/expiry/model scope -> calculate limits and quota -> inject caller context

Show only key prefixes/suffixes in the admin UI. Return the plaintext key only once at creation time, then replace it through rotation.

#Model Routing

Model routing maps client-requested models to actual upstreams:

client model: gpt-4o-mini
        |
        v
routing rule: openai-primary 80%, openai-backup 20%
        |
        v
upstream model: gpt-4o-mini / gpt-4.1-mini / custom alias

Common routing dimensions:

Write the routing result into request logs so operators can answer "which model did the client request, and which upstream did it actually hit?"

#Streaming Responses

LLM relays often return SSE or chunked bodies. Streaming differs from normal admin JSON:

Continue using #[log] for admin actions. Relay requests should write to a dedicated AI request log, for example:

This keeps admin operation audit intact while giving relay traffic the high-frequency, streaming, billing-oriented log model it needs.

#Rate Limits And Quotas

summer-common::rate_limit provides #[rate_limit], RateLimitEngine, and cost-based rate limiting. Admin APIs can use declarative limits; relay traffic is better controlled by API key and token cost.

For LLM requests, reserve quota before calling the upstream:

let reservation = rate_limit_ctx
    .reserve(&token_key, config, estimated_tokens, "请求过于频繁")
    .await?;

// Commit with real usage after upstream success; release/refund on failure or over-estimation.

Common strategies:

If the API key is in Authorization, the relay auth layer can parse it first, then call RateLimitContext with the resolved token identifier.

#Relationship To MCP

MCP and AI relay solve different problems:

MCP can help develop AI admin modules: generate entities, CRUD, frontend bundles, then plan menus and dictionaries with menu_tool and dict_tool. Relay handles online model calls, API keys, quotas, routing, logs, and streaming responses.

#Integration Checklist

You can integrate an AI gateway module in this order:

1. Define AI domain models: channels, models, API keys, routing rules, request logs, usage stats.
2. Create admin menus and button permissions with the ai:* permission namespace.
3. Add admin APIs with #[log], #[has_perm], and resource-permission bindings.
4. Design API key auth: store only hashes and show plaintext only once.
5. Implement model routing from client model names to real upstream channels.
6. Add rate limits and quotas for keys, tenants, models, and token cost.
7. Create dedicated request logs for streaming responses, final status, and usage.
8. Use MCP to generate or validate CRUD, menus, dictionaries, and frontend page drafts.

This lets the AI gateway fit into the admin system while keeping relay protocol entrypoints independent.