diff --git a/internal/cmd/root/verbs/install/skills_test.go b/internal/cmd/root/verbs/install/skills_test.go index 2264164e1..a55f4e2fd 100644 --- a/internal/cmd/root/verbs/install/skills_test.go +++ b/internal/cmd/root/verbs/install/skills_test.go @@ -51,7 +51,7 @@ func TestInstallBundledSkillsWritesVersionedSkillsAndManifest(t *testing.T) { assert.Equal(t, canonicalDir, result.CanonicalDir) assert.Equal(t, "v9.9.9", result.CLIVersion) - expectedSkills := []string{"kongctl-declarative", "kongctl-extension-builder"} + expectedSkills := []string{"kongctl-ai-gateway", "kongctl-declarative", "kongctl-extension-builder"} assert.ElementsMatch(t, expectedSkills, result.SkillNames) for _, skillName := range result.SkillNames { diff --git a/skills/README.md b/skills/README.md index eb077ffba..7875f8aec 100644 --- a/skills/README.md +++ b/skills/README.md @@ -5,6 +5,8 @@ with `kongctl`. ## Included Skills +- [kongctl-ai-gateway](kongctl-ai-gateway/SKILL.md) + - Deploy native AI Gateway models, caller access and auditable CI/CD. - `kongctl-declarative` - Generate declarative config and manage plan/apply/sync/delete/adopt flows. - `kongctl-extension-builder` @@ -28,8 +30,10 @@ kongctl install skills --dry-run By default, the installer writes skill files to `.kongctl/skills/` and creates symlinks for supported agent tooling: +- `.agents/skills/kongctl-ai-gateway` - `.agents/skills/kongctl-declarative` - `.agents/skills/kongctl-extension-builder` +- `.claude/skills/kongctl-ai-gateway` - `.claude/skills/kongctl-declarative` - `.claude/skills/kongctl-extension-builder` @@ -50,6 +54,7 @@ tool's skills directory. - Target path: `.claude/skills/` - Example: + - `ln -s ../../skills/kongctl-ai-gateway .claude/skills/kongctl-ai-gateway` - `ln -s ../../skills/kongctl-declarative .claude/skills/kongctl-declarative` - `ln -s ../../skills/kongctl-extension-builder .claude/skills/kongctl-extension-builder` @@ -57,5 +62,6 @@ tool's skills directory. - Target path: `.agents/skills/` (some setups use `.agents/skills/`) - Example: + - `ln -s ../../skills/kongctl-ai-gateway .agents/skills/kongctl-ai-gateway` - `ln -s ../../skills/kongctl-declarative .agents/skills/kongctl-declarative` - `ln -s ../../skills/kongctl-extension-builder .agents/skills/kongctl-extension-builder` diff --git a/skills/embed.go b/skills/embed.go index 711bbca22..2a6fcc71c 100644 --- a/skills/embed.go +++ b/skills/embed.go @@ -4,5 +4,5 @@ import "embed" // BundledFS contains the built-in kongctl skills distributed with the CLI. // -//go:embed kongctl-declarative kongctl-extension-builder +//go:embed kongctl-ai-gateway kongctl-declarative kongctl-extension-builder var BundledFS embed.FS diff --git a/skills/kongctl-ai-gateway/SKILL.md b/skills/kongctl-ai-gateway/SKILL.md new file mode 100644 index 000000000..61aa9e899 --- /dev/null +++ b/skills/kongctl-ai-gateway/SKILL.md @@ -0,0 +1,148 @@ +--- +name: kongctl-ai-gateway +description: >- + Build and evolve native Konnect AI Gateway deployments with kongctl + declarative configuration, from local provider-backed inference to model + access controls and auditable CI/CD. Use for AI Gateway setup, extending + an existing deployment, or diagnosing its configuration and traffic. +license: Apache-2.0 +metadata: + product: kongctl + category: ai-gateway +--- + +# kongctl AI Gateway + +Help the user reach a working AI Gateway through declarative files they can +review and operate again. This skill is self-contained; a kongctl source +checkout or another skill is unnecessary. + +## Choose the next outcome + +- For a new local OpenAI deployment, read + [local OpenAI setup](references/local-openai.md). Adapt the bundled + [manifest](assets/openai/ai-gateway.yaml) and + [data plane helper](assets/openai/data-plane.sh) into the user's project. +- For more models or caller authorization, read + [model access](references/model-access.md). Extend the existing gateway + and preserve its namespace, refs, names, public certificate and model + aliases unless the requested change requires otherwise. +- For a deployment pipeline, read [CI/CD](references/cicd.md). Combine it + with model access only when that expansion is requested. Start with the + bundled single-workflow example and one existing gateway; for a + speed-focused task, get the first CI deployment working before expanding + models or callers. Keep the user's required approval boundary, but do not + add enterprise controls or custom audit machinery by default. +- For a failure, use the diagnostic table in the local setup reference; + inspect the relevant installed schema before changing configuration. + +Use the user's chosen provider, CI system and hosting model. The local +OpenAI example is a starting point, not a prerequisite for other use cases. +If account details or credentials are unavailable, still prepare concrete +files, local checks and the commands needed to finish. Ask only for choices +that block the next action; reuse existing authorization. A files-only task +does not authorize Konnect mutations, Docker startup or paid inference. + +## Discover the installed contract + +Start with `kongctl version --full` and targeted schema discovery: + +```sh +kongctl explain ai_gateways -o text +``` + +Then inspect only the children and fields needed for the requested setup, +using `-o text` for summaries. Request JSON or extended output only for a +specific field that needs detail; a full gateway schema includes all nested +resources and can overwhelm the useful context. For example, use +`kongctl explain ai_gateways.models.access -o text` for access controls. +Use `kongctl scaffold --help` when a different resource shape needs a +starter. Read the schema's required +fields, union branches and relationship annotations; do not infer required +names from `ref`. If the installed CLI lacks these subjects, report that +capability gap and arrange a compatible CLI before attempting deployment. + +Native AI Gateway uses `ai_gateways`, nested `model_providers`, `models`, +`data_plane_certificates`, `auth_strategies` and `consumers`. A Gateway +control plane with services, routes and AI plugins is a different workflow. +Use that architecture when the user explicitly requests it or is already +operating it. Native AI Gateway configuration does not require decK. + +The bundled example follows kongctl 1.15.1 native declarative support and +the AI Gateway 2.0 quickstarts. Verify the installed schema when adapting it; +an example version is not a claim of compatibility with every environment. +For other providers or policies, use targeted discovery and the current +[AI Gateway documentation](https://developer.konghq.com/ai-gateway/). + +## Keep the deployment contract explicit + +- Set `_defaults.kongctl.namespace` to the project's ownership scope. Use + explicit `ref`, `name` and required API fields. Preserve identity during + expansion so the plan updates the existing deployment. +- Keep file inputs within the project. `!file` resolves relative to its + containing YAML file, constrained by `--base-dir`. Ordinary `!env` values + resolve during loading; deferred `!secret` values resolve during apply. +- Use `!secret` for write-only provider and caller credentials. Public + `Bearer ` decoration belongs in `parts`; secret bytes do not. Do not print + environment values or put private keys, credentials or response dumps in + committed configuration. +- Separate the public model alias used in request routing from the upstream + model ID. Give aliases unique values within the gateway. Select upstream + models available to the user's provider account before planning. +- Prefer an explicit `--mode apply` plan for additive setup and expansion. + `plan` defaults to sync mode, which can propose deletions. A saved plan's + execution verb must match its mode. Use sync/delete only for the intended + ownership scope and show their proposed removals. +- Plan and diff require remote state and Konnect authentication. Schema + inspection and local file checks can run without them. Do not invent an + offline validation flag, substitute a fake API endpoint, or claim generic + YAML lint proves native schema, references or deployment success. + +Once the target and execution are authorized, produce and inspect a saved +plan, then execute that same file: + +```sh +mkdir -p .plans .artifacts +kongctl plan --mode apply -f ai-gateway.yaml \ + --require-namespace ai-demo --output-file .plans/apply.json +kongctl diff --plan .plans/apply.json +kongctl apply --plan .plans/apply.json -o json --auto-approve \ + --execution-report-file .artifacts/apply-report.json +``` + +Adapt the namespace and use the same explicit profile and region/base URL +for planning and execution. Record the target organization ID using +`kongctl get organization -o json --jq '.id' --jq-raw-output` with that +same context; a profile or organization name alone is not its identity. +Changes to inputs or target require a new plan and review. Secrets +referenced in that plan must be supplied to execution. +Use `--auto-approve` only after the saved plan's execution is authorized; +it avoids a second interactive CLI prompt in an agent's noninteractive +shell. Explicit JSON output also works around versions that skip writing +`--execution-report-file` with text output. Verify that the report exists +and inspect its summary; a missing report does not mean apply failed and +is not a reason to repeat mutations. + +## Finish with evidence and a usable handoff + +Deliver the project files and concise instructions to configure required +inputs, plan/apply, run the data plane, verify traffic and clean up the +project's resources. Keep local data plane operations separate from remote +Konnect operations so the user can run each on its intended host. + +Report what was actually checked: + +- **Authored:** files and execution instructions exist. +- **Locally checked:** name the schema, file or script checks performed. +- **Provisioned/connected:** resources exist and the data plane connects. +- **Functional:** a matching request returns a real model completion; + access changes also pass the intended allowed and denied cases. + +A plan, container startup, node listing or HTTP status alone does not prove +inference. Record remaining steps when live execution is unavailable. For a +repeatable demo, inspect an unchanged follow-up plan and rehearse scoped +cleanup. Keep the execution report, follow-up plan and sanitized node and +inference check results as local evidence; do not rely only on a transcript +summary. Inspect artifacts for secrets before sharing them. +Determinism here concerns deployment inputs and approved actions; +model replies and remote state can vary. diff --git a/skills/kongctl-ai-gateway/agents/openai.yaml b/skills/kongctl-ai-gateway/agents/openai.yaml new file mode 100644 index 000000000..78ac9237a --- /dev/null +++ b/skills/kongctl-ai-gateway/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Kongctl AI Gateway" + short_description: "Deploy AI Gateway with models, access and CI/CD" + default_prompt: "Use $kongctl-ai-gateway to build a declarative AI Gateway + deployment with working model traffic and a reviewable deployment workflow." diff --git a/skills/kongctl-ai-gateway/assets/github-actions/deploy.yaml b/skills/kongctl-ai-gateway/assets/github-actions/deploy.yaml new file mode 100644 index 000000000..afb8a07fa --- /dev/null +++ b/skills/kongctl-ai-gateway/assets/github-actions/deploy.yaml @@ -0,0 +1,84 @@ +name: Deploy reviewed AI Gateway plan + +on: + workflow_dispatch: + inputs: + plan_sha256: + description: SHA-256 of the committed ci/plan.json you reviewed + required: true + type: string + +permissions: + contents: read + +concurrency: + group: ai-gateway-ai-demo + cancel-in-progress: false + +env: + KONGCTL_VERSION: "1.16.0" + KONGCTL_NO_TELEMETRY: "true" + KONNECT_BASE_URL: https://us.api.konghq.com + KONNECT_ORG_ID: REPLACE_WITH_ORGANIZATION_ID + KONNECT_NAMESPACE: ai-demo + +jobs: + deploy: + runs-on: ubuntu-24.04 + timeout-minutes: 5 + defaults: + run: + shell: bash + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + with: + persist-credentials: false + - name: Check the reviewed plan + env: + EXPECTED_PLAN_SHA256: ${{ inputs.plan_sha256 }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + run: | + set -euo pipefail + test "$GITHUB_REF" = "refs/heads/$DEFAULT_BRANCH" + [[ "$EXPECTED_PLAN_SHA256" =~ ^[0-9a-f]{64}$ ]] + printf '%s ci/plan.json\n' "$EXPECTED_PLAN_SHA256" | sha256sum --check - + jq -e --arg ns "$KONNECT_NAMESPACE" --arg cli "kongctl/$KONGCTL_VERSION" ' + .metadata.mode == "apply" and .metadata.generator == $cli and + all(.changes[]; .namespace == $ns and + (.action == "CREATE" or .action == "UPDATE")) + ' ci/plan.json >/dev/null + - uses: Kong/setup-kongctl@659d5c2b0f4db2e25d63797035358a22907421db # v1 + with: + kongctl-version: ${{ env.KONGCTL_VERSION }} + - name: Apply the approved plan + env: + KONGCTL_DEFAULT_KONNECT_PAT: ${{ secrets.KONNECT_PAT }} + OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} + run: | + set -euo pipefail + test -n "$KONGCTL_DEFAULT_KONNECT_PAT" + test "$(kongctl version --full | cut -d ' ' -f 1)" = "$KONGCTL_VERSION" + actual_org="$(kongctl get organization --profile default \ + --base-url "$KONNECT_BASE_URL" -o json --jq '.id' --jq-raw-output)" + test "$actual_org" = "$KONNECT_ORG_ID" + mkdir -p evidence + { + printf 'Source: %s\nActor: %s\n' "$GITHUB_SHA" "$GITHUB_ACTOR" + printf 'Organization: %s\nNamespace: %s\n' "$actual_org" "$KONNECT_NAMESPACE" + sha256sum ci/plan.json + kongctl diff --plan ci/plan.json + } > evidence/review.txt + cat evidence/review.txt + cat evidence/review.txt >> "$GITHUB_STEP_SUMMARY" + kongctl apply --plan ci/plan.json --profile default \ + --base-url "$KONNECT_BASE_URL" --auto-approve -o json \ + --execution-report-file evidence/apply.json + jq -e '.summary.status == "success"' evidence/apply.json + - name: Save execution evidence + if: always() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: ai-gateway-${{ github.run_id }}-${{ github.run_attempt }} + path: evidence/ + if-no-files-found: warn + retention-days: 7 diff --git a/skills/kongctl-ai-gateway/assets/openai/ai-gateway.yaml b/skills/kongctl-ai-gateway/assets/openai/ai-gateway.yaml new file mode 100644 index 000000000..5c4dd616d --- /dev/null +++ b/skills/kongctl-ai-gateway/assets/openai/ai-gateway.yaml @@ -0,0 +1,58 @@ +_defaults: + kongctl: + namespace: ai-demo + +ai_gateways: + - ref: ai-demo + name: ai-demo + display_name: AI Demo + description: Local OpenAI-backed AI Gateway + deployment_type: hybrid + proxy_urls: + - host: localhost + port: 8000 + protocol: http + data_plane_certificates: + - ref: ai-demo-data-plane + title: ai-demo-data-plane + description: Local Docker data plane + cert: !file ./certs/data-plane.crt + model_providers: + - ref: openai + name: openai + display_name: OpenAI + type: openai + config: + auth: + type: basic + headers: + - name: Authorization + value: !secret + parts: + - "Bearer " + - !env OPENAI_API_KEY + models: + - ref: demo-chat + name: demo-chat + display_name: Demo Chat + type: model + enabled: true + formats: + - type: openai + config: + route: + paths: + - /v1 + model: + body_param: model + values: + - demo-chat + targets: + - name: !env OPENAI_MODEL + provider: openai + allow_auth_override: false + config: + type: openai + policies: [] + capabilities: + - generate diff --git a/skills/kongctl-ai-gateway/assets/openai/data-plane.sh b/skills/kongctl-ai-gateway/assets/openai/data-plane.sh new file mode 100644 index 000000000..156f73eba --- /dev/null +++ b/skills/kongctl-ai-gateway/assets/openai/data-plane.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +set -euo pipefail + +project_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cert_dir="${project_dir}/certs" +cert_file="${cert_dir}/data-plane.crt" +key_file="${AIGW_DATA_PLANE_KEY:-${cert_dir}/data-plane.key}" +container_name="${AIGW_CONTAINER_NAME:-ai-demo-data-plane}" +image="${KONG_AI_GATEWAY_IMAGE:-kong/kong-ai-gateway:2.0.3}" +proxy_port="${AIGW_PROXY_PORT:-8000}" +proxy_tls_port="${AIGW_PROXY_TLS_PORT:-8443}" + +fail() { + echo "Error: $*" >&2 + exit 1 +} + +require_command() { + command -v "$1" >/dev/null 2>&1 || fail "$1 is required" +} + +check_certificates() { + require_command openssl + [[ "${key_file}" == /* ]] || fail "AIGW_DATA_PLANE_KEY must be absolute" + [[ -f "${cert_file}" && -f "${key_file}" ]] || + fail "Missing certificate pair; supply the existing key or generate a new pair explicitly" + local cert_public_key key_public_key + cert_public_key="$(openssl x509 -in "${cert_file}" -pubkey -noout | + openssl pkey -pubin -outform DER | openssl dgst -sha256)" + key_public_key="$(openssl pkey -in "${key_file}" -passin pass: \ + -pubout -outform DER | openssl dgst -sha256)" + [[ "${cert_public_key}" == "${key_public_key}" ]] || + fail "The private key does not match certs/data-plane.crt" + openssl x509 -in "${cert_file}" -noout -checkend 0 >/dev/null || + fail "The data plane certificate has expired" +} + +generate_certs() { + require_command openssl + [[ "${key_file}" == /* ]] || fail "AIGW_DATA_PLANE_KEY must be absolute" + if [[ -e "${cert_file}" || -e "${key_file}" ]]; then + check_certificates + echo "Using the existing certificate pair" + return + fi + umask 027 + mkdir -p "${cert_dir}" "$(dirname "${key_file}")" + # LibreSSL requires a distinguished_name section even with -subj. + # Supply our own config so host OPENSSL_CONF settings cannot change it. + openssl req -config <(printf '%s\n' \ + '[req]' 'distinguished_name = req_distinguished_name' \ + '[req_distinguished_name]') \ + -new -x509 -nodes -newkey rsa:2048 -days 365 \ + -subj "/CN=ai-demo-data-plane/C=US" \ + -keyout "${key_file}" -out "${cert_file}" + chgrp "$(id -g)" "${key_file}" + chmod 640 "${key_file}" +} + +check_ports() { + local port + for port in "${proxy_port}" "${proxy_tls_port}"; do + [[ "${port}" =~ ^[1-9][0-9]{0,4}$ ]] || + fail "Proxy host ports must be integers from 1 to 65535" + (( port <= 65535 )) || + fail "Proxy host ports must be integers from 1 to 65535" + done + [[ "${proxy_port}" != "${proxy_tls_port}" ]] || + fail "HTTP and HTTPS host ports must differ" +} + +show_status() { + require_command docker + check_ports + docker context show + docker ps -a --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' + printf 'Requested container: %s; loopback ports: %s / %s\n' \ + "${container_name}" "${proxy_port}" "${proxy_tls_port}" +} + +preflight() { + require_command docker + check_ports + local names published name ports port + # Failed daemon access is an error, not evidence of a clean host. + names="$(docker ps -a --format '{{.Names}}')" + while IFS= read -r name; do + [[ "${name}" != "${container_name}" ]] || + fail "Container ${container_name} already exists; inspect status and choose resume or scoped reset" + done <<< "${names}" + published="$(docker ps --format '{{.Names}} {{.Ports}}')" + while read -r name ports; do + for port in "${proxy_port}" "${proxy_tls_port}"; do + [[ "${ports}" != *":${port}->"* ]] || + fail "Host port ${port} is published by ${name}; reset its deployment or choose ports before planning" + done + done <<< "${published}" + if command -v lsof >/dev/null 2>&1; then + for port in "${proxy_port}" "${proxy_tls_port}"; do + if lsof -nP -iTCP:"${port}" -sTCP:LISTEN >/dev/null 2>&1; then + fail "Host port ${port} has a listener; inspect its owner before proceeding" + fi + done + fi + echo "Preflight passed; Docker will check port availability again at startup" +} + +run_data_plane() { + preflight + check_certificates + local key_group + key_group="$(stat -c '%g' "${key_file}" 2>/dev/null || \ + stat -f '%g' "${key_file}")" + : "${AIGW_CONTROL_PLANE:?Set the discovered configuration hostname}" + : "${AIGW_TELEMETRY:?Set the discovered telemetry hostname}" + for endpoint in "${AIGW_CONTROL_PLANE}" "${AIGW_TELEMETRY}"; do + [[ "${endpoint}" != null && + "${endpoint}" =~ ^[a-zA-Z0-9][a-zA-Z0-9.-]*$ ]] || + fail "Endpoints must be hostnames, without a scheme, port or path" + done + docker run --detach --rm --name "${container_name}" \ + --label "com.kongctl.ai-gateway.project=${project_dir}" \ + --group-add "${key_group}" \ + --env KONG_ROLE=data_plane --env KONG_DATABASE=off \ + --env KONG_VITALS=off --env KONG_CLUSTER_MTLS=pki \ + --env "KONG_CLUSTER_CONTROL_PLANE=${AIGW_CONTROL_PLANE}:443" \ + --env "KONG_CLUSTER_SERVER_NAME=${AIGW_CONTROL_PLANE}" \ + --env "KONG_CLUSTER_TELEMETRY_ENDPOINT=${AIGW_TELEMETRY}:443" \ + --env "KONG_CLUSTER_TELEMETRY_SERVER_NAME=${AIGW_TELEMETRY}" \ + --env KONG_CLUSTER_CERT=/etc/kong/certs/data-plane.crt \ + --env KONG_CLUSTER_CERT_KEY=/etc/kong/certs/data-plane.key \ + --env KONG_LUA_SSL_TRUSTED_CERTIFICATE=system \ + --env KONG_KONNECT_MODE=on \ + --volume "${cert_file}:/etc/kong/certs/data-plane.crt:ro" \ + --volume "${key_file}:/etc/kong/certs/data-plane.key:ro" \ + --publish "127.0.0.1:${proxy_port}:8000" \ + --publish "127.0.0.1:${proxy_tls_port}:8443" \ + "${image}" +} + +case "${1:-}" in + certs) generate_certs ;; + check) check_certificates ;; + status) show_status ;; + preflight) preflight ;; + run) run_data_plane ;; + stop) + require_command docker + docker stop "${container_name}" + ;; + *) fail "Usage: bash data-plane.sh certs|check|status|preflight|run|stop" ;; +esac diff --git a/skills/kongctl-ai-gateway/references/cicd.md b/skills/kongctl-ai-gateway/references/cicd.md new file mode 100644 index 000000000..208801f13 --- /dev/null +++ b/skills/kongctl-ai-gateway/references/cicd.md @@ -0,0 +1,112 @@ +# Get a working deployment workflow first + +For a small GitHub Actions setup, start with the bundled +[workflow](../assets/github-actions/deploy.yaml). Copy it to +`.github/workflows/deploy-ai-gateway.yaml`. Keep the existing gateway, +namespace, model and local data plane. Get one deployment working before +adding models or authorization, unless those are the user's immediate goal. + +The first version plans locally and deploys in CI. The user reviews that +saved plan, commits it with the source, then approves execution by manually +dispatching the workflow with its exact SHA-256. CI verifies the bytes and +organization and applies that file. It never replans during deployment. +This needs one workflow, one Konnect token and no custom audit scripts. + +## Check prerequisites once, before building + +Check `gh auth status`, the repository's default branch, and existing +secret **names** using `gh secret list`. Reuse an existing authorized CI +token; the example calls its repository secret `KONNECT_PAT`. If the repo +already has `KONNECT_APPLY_PAT`, change the workflow's secret reference. +Do not create two new tokens or require separate read/write credentials +for this small setup. Ask for a missing CI credential with an explicit +scope; do not mint broad tokens merely to finish setup. + +Use a working, released local kongctl version and the same fixed version +in the workflow. Edit its version, organization UUID, base URL, namespace +and concurrency group to match the project. The bundled action revisions +and inputs are already selected; adapting this template does not require +researching or writing a new installer. + +Repository writers can [manually dispatch][dispatch] this workflow. This +is an explicit operator approval, not independent reviewer enforcement. +Do not configure an environment reviewer gate unless the user requests +that policy or the repository already requires it. [Required reviewers] +have plan/visibility restrictions; check support before building that path. +Keep existing repository controls. If publishing requires a PR, create it +directly and surface the merge prerequisite early. The workflow must exist +on the default branch before its first manual dispatch. + +## Prepare one small change and its saved plan + +Start with the current working manifest. For a visible first deployment, +make a user-requested small change such as updating the gateway description. +An unchanged plan is also a valid automation smoke test; label it as such. +Do not add a new model or caller key solely to prove CI works. + +Use the project's known profile/target and namespace in these commands: + +```sh +mkdir -p ci +kongctl plan --mode apply -f ai-gateway.yaml \ + --profile default --base-url https://us.api.konghq.com \ + --require-namespace ai-demo --output-file ci/plan.json +kongctl diff --plan ci/plan.json +shasum -a 256 ci/plan.json +``` + +Inspect the plan before committing it: provider and caller credentials +must be deferred `!secret` environment sources, never literal bytes. +Commit `ci/plan.json`, the manifest, the public certificate and the workflow. +Keep `.env` and private keys ignored. Plans can contain operational IDs and +configuration; use a repository appropriate for that content. The public +certificate resolves `!file` for future planning; the private key stays on +the data plane host. Do not regenerate or move it when enabling CI. + +The initial workflow maps `OPENAI_API_KEY` for plans that write that secret. +Existing resources with no secret write need no new provider credential. +When later adding caller credentials, add only their required secret/env +mappings. Do not require unrelated secrets to execute an unchanged plan. + +## Approve, run, and verify + +After reviewing the exact plan and publishing through the repository's +existing source-review process, the authorized operator runs: + +```sh +gh workflow run deploy-ai-gateway.yaml --ref main \ + -f plan_sha256=THE_REVIEWED_SHA256 +``` + +Use the actual default branch. An agent may dispatch after the user has +authorized that exact plan; do not treat the hash as approval by itself. +Open the resulting run and confirm the execution report succeeded. The +workflow records source revision, actor, target, plan hash and diff, and +retains its execution report. `-o json` also avoids CLI versions that skip +the report file in text output. + +Run one inference check from the existing data plane host. A GitHub-hosted +runner's `localhost` is not the user's gateway. Generate a fresh local +apply-mode plan afterwards to check convergence; do not blindly rerun an +old create plan. For later changes, replace `ci/plan.json`, review the new +diff and hash, commit, then dispatch again. + +## Keep the first delivery small + +One working run and one short README section are the first milestone. +Use available lint tools; do not install a Go toolchain or build a bespoke +test/audit framework to validate this copied workflow. Check adapted shell +syntax and plan inputs, then exercise the real workflow when authorized. + +The minimal path assumes one trusted repository operator and a recently +reviewed plan. It does not enforce a second reviewer, plan expiry, or +detect all remote drift. Add those controls, CI-side planning, multi-target +promotion, separate tokens, or a full access matrix when requested. If an +existing policy needs a stronger gate, honor it instead of using this path. + +For a speed-focused task, report time to the first successful run separately +from later feature expansion. Surface missing login, token, source merge +or runner availability immediately; never call authored YAML functional CI. + +[dispatch]: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/manually-run-a-workflow +[Required reviewers]: https://docs.github.com/en/actions/reference/workflows-and-actions/deployments-and-environments diff --git a/skills/kongctl-ai-gateway/references/local-openai.md b/skills/kongctl-ai-gateway/references/local-openai.md new file mode 100644 index 000000000..ca9ab1d67 --- /dev/null +++ b/skills/kongctl-ai-gateway/references/local-openai.md @@ -0,0 +1,203 @@ +# Local OpenAI deployment + +Use the installed schema as the field contract. The bundled manifest and +helper adapt the native [OpenAI quickstart][quickstart] and use the +`kong/kong-ai-gateway:2.0.3` image from the AI Gateway 2.0 examples. Select and +record a compatible image for the user's environment; pin its digest for a +rehearsed deployment. + +## Discover existing state before planning + +A fresh project can share Docker and Konnect with an existing deployment. +Before generating a certificate, inspect existing project files and the +intended gateway under the selected profile, organization and region. +Inspect `docker context show` and `docker ps -a` for existing containers +and published ports. A permission or network error is not an empty result. + +If state remains and intent is unclear, resolve one choice with the user: +resume that deployment, replace it after scoped cleanup, or run in parallel. +For resume, preserve the namespace, gateway identity and certificate pair; +verify the container's Konnect endpoints, certificate mounts, image and +port mappings before reusing it. A matching name alone is insufficient. +For replacement, review the intended removals before cleanup. For a +parallel deployment, choose distinct identities and free ports before planning. +Do not silently switch namespaces or replace certificates to escape a +collision. If access is unavailable, record this preflight as pending. + +## Prepare the project + +Copy `assets/openai/ai-gateway.yaml` and `assets/openai/data-plane.sh` from +this skill into the project root. Adapt their names to the user's project. +The helper is invoked with `bash`; it need not have executable permissions. +Provide a README with the chosen profile/region, inputs and commands below. +Create `.gitignore` before generating keys: + +```gitignore +.env +.env.* +!.env.example +certs/*.key +.plans/ +.artifacts/ +``` + +An `.env.example` may contain empty values and descriptions. Do not assume +kongctl automatically loads `.env`; document how the shell supplies inputs. +For a trusted, shell-compatible local `.env`, load it in each shell that +needs its values without printing them: + +```sh +set -a +. ./.env +set +a +``` + +Persist the chosen container name, `AIGW_PROXY_PORT` and +`AIGW_PROXY_TLS_PORT` in the project's local settings and document them in +`.env.example`. Defaults are 8000 and 8443; both remain loopback-only. +Set the manifest's `proxy_urls[].port` and all request URLs to the chosen +HTTP host port before generating the saved plan. Host port overrides do +not change the container's internal listeners or update Konnect for you. + +Run `bash data-plane.sh status` to inspect local containers, then +`bash data-plane.sh preflight` before planning a new deployment. Preflight +rejects an existing container name and Docker port conflicts, and checks +host listeners when `lsof` is available. It is read-only and needs neither +certificates nor Konnect endpoints. Docker still checks for port races +when starting. On a resume, inspect and reuse the matching running node +instead of calling `run` again; the helper never replaces it automatically. + +Required inputs and tools: + +| Input | When needed | Purpose | +| --- | --- | --- | +| `OPENAI_MODEL` | Before loading/planning | Available upstream model ID | +| `OPENAI_API_KEY` | At apply | OpenAI provider credential | +| Konnect login or PAT | Plan, apply, discovery | Intended org and region | +| Docker and OpenSSL | Local data plane | Container and mTLS key pair | +| curl and jq | Inference check | HTTP request and completion assertion | + +`demo-chat` is the client-facing alias; `OPENAI_MODEL` is its upstream +target. Keep the alias stable when changing provider models. For CI, prefer +committing the selected, non-secret model IDs to YAML so review shows them. + +Generate the local certificate pair once: + +```sh +bash data-plane.sh certs +``` + +Only `certs/data-plane.crt` goes into the declarative manifest. The private +key stays on the data plane host. This certificate authenticates the data +plane to Konnect; it is distinct from a proxy's public HTTPS certificate. +The helper checks and reuses an existing pair. It refuses mismatched or +incomplete pairs without replacing either file. Run +`bash data-plane.sh check` to check a pair without starting Docker. +If a failed generation left only a key, preserve it and explicitly repair +the pair or authorize a fresh pair before retrying. Do not replace a +certificate that has already been planned or registered without replanning. + +By default the key is `certs/data-plane.key`. For an existing key outside +the checkout, set `AIGW_DATA_PLANE_KEY` to its absolute path on the laptop. +The helper mounts that file separately from the public certificate. Keep +the key readable by its runtime group (the generated key uses mode 0640); +the container receives that file's group ID. Preserve this path setting +when handing the project over to CI/CD. + +For a files-only request, generating a local pair is optional; leaving the +explicit generation command is sufficient. Explain that resolving the +manifest's `!file` requires that certificate. The helper's Docker group and +permissions follow the Linux example; adapt them if the target host differs. + +## Plan and provision when authorized + +Select the user's Konnect profile and region and use them consistently. +Authenticate with `kongctl login` or supply the profile's PAT environment +variable. For the default profile it is `KONGCTL_DEFAULT_KONNECT_PAT`. +Keep the token out of command arguments and logs. + +Use the saved apply-plan sequence in `SKILL.md`, including the project's +namespace guard. `OPENAI_MODEL` must already be set; `OPENAI_API_KEY` is +deferred until execution. Inspect the plan's resource scope before applying. +Do not add `--write-secrets` to routine runs: new resources write their +configured secrets once; subsequent secret rotation is an explicit change. + +## Discover endpoints and run locally + +After apply, retrieve the gateway using its actual display name. Add the +same profile and region/base URL flags used during provisioning: + +```sh +AIGW_CONTROL_PLANE="$(kongctl get ai-gateway 'AI Demo' -o json \ + --jq '.endpoints.configuration | sub("^https://"; "") | + sub(":443$"; "")' --jq-raw-output)" +AIGW_TELEMETRY="$(kongctl get ai-gateway 'AI Demo' -o json \ + --jq '.endpoints.telemetry | sub("^https://"; "") | + sub(":443$"; "")' --jq-raw-output)" +export AIGW_CONTROL_PLANE AIGW_TELEMETRY +bash data-plane.sh run +kongctl get ai-gateway nodes --gateway-name 'AI Demo' +``` + +Check that both results are nonempty real hostnames, not `null`, before +starting Docker. Do not synthesize endpoints from a gateway ID or region. +The helper mounts the pair read-only, configures both Konnect channels, +and binds the unauthenticated demo proxy to loopback. The provider key is +sent through declarative apply; it is not a client request header or a +required Docker environment variable in this example. + +Allow a bounded interval for the node to connect and configuration to +propagate. Inspect its status and `docker logs ai-demo-data-plane` if it +does not. Do not use repeated paid chat requests as a readiness loop. + +## Prove inference + +Send an OpenAI-format chat request through the local gateway using the +configured alias and path: + +```bash +set -o pipefail +curl --fail-with-body --silent --show-error --max-time 60 \ + "http://127.0.0.1:${AIGW_PROXY_PORT:-8000}/v1/chat/completions" \ + -H 'Content-Type: application/json' \ + -d '{"model":"demo-chat","messages":[ + {"role":"user","content":"Reply with a short greeting."}]}' \ + | jq -e '.choices[0].message.content | + select(type == "string" and length > 0)' +``` + +Verify a successful HTTP response with a nonempty assistant completion +(for this text example, `choices[0].message.content`). Report the request +path, alias and observed result without logging credentials. A non-404 +response or a running container is not this check. Use one bounded request +per smoke test unless the user authorizes a larger traffic test. + +| Symptom | Check before changing configuration | +| --- | --- | +| No node connection | Discovered endpoints, cert/key match, read access | +| Connection fails | Docker logs, DNS and outbound TLS to both channels | +| Proxy 404 | Applied config, `/v1` route, unique matching body alias | +| Proxy 401 | Caller key/auth strategy; distinguish upstream auth error | +| Proxy 403 after auth | Model ACL names and caller/group membership | +| Provider error | Model availability, key, upstream quota and response | +| Success without content | Response format and actual completion body | + +Keep diagnostic logs local and inspect them for credentials before sharing. +If the provider rejects the request, stop and fix that cause; switching +models or weakening access controls silently is not a valid verification. + +## Cleanup and repeatability + +Stop only this project's container with `bash data-plane.sh stop`. This +does not remove Konnect resources. For requested teardown, generate a +`kongctl plan --mode delete -f ai-gateway.yaml` with the same namespace +guard and target, save and inspect it using `diff --plan`, then execute +`kongctl delete --plan` after the intended removals are authorized. +Deletion of a gateway also affects its children; inspect that scope. + +Keep the certificate pair for reuse until reset is deliberate. For an +unchanged deployment, generate another apply-mode plan and inspect its +actions. Do not claim idempotence merely because apply exited successfully. + +[quickstart]: + https://github.com/Kong/kongctl/tree/main/docs/examples/declarative/ai-gateway/openai-llm diff --git a/skills/kongctl-ai-gateway/references/model-access.md b/skills/kongctl-ai-gateway/references/model-access.md new file mode 100644 index 000000000..c967f9f7f --- /dev/null +++ b/skills/kongctl-ai-gateway/references/model-access.md @@ -0,0 +1,166 @@ +# Expand models and authorize callers + +Start from the existing deployment. Preserve its gateway identity, namespace, +certificate and original model alias. Read the installed contracts: + +```sh +kongctl explain ai_gateways.auth_strategies +kongctl explain ai_gateways.consumers.credentials +kongctl explain ai_gateways.models.access +``` + +For a simple platform demo, use three aliases and two API-key consumers: +`full-access` may use all three, while `limited-access` may use only the +original model. Choose two additional upstream model IDs available to the +account. Keep one `/v1/chat/completions` endpoint and select the model via +the request body. API keys demonstrate caller access without an external +identity provider; use OIDC when the user asks for federated identity. + +## Add caller identity, then attach it to each model + +Merge this fragment into the existing `ai_gateways` entry. It is not a +standalone manifest. The caller keys are distinct from `OPENAI_API_KEY`: + +```yaml +auth_strategies: + - ref: client-key-auth + name: client-key-auth + display_name: Client Key Authentication + type: key-auth + config: + key_names: [apikey] + hide_credentials: true +consumers: + - ref: full-access + name: full-access + display_name: Full Access + custom_id: full-access + type: api-key + credentials: + - ref: full-access-key + name: full-access-key + display_name: Full Access Key + type: api-key + ttl: 0 + api_key: !secret {source: !env FULL_ACCESS_API_KEY} + - ref: limited-access + name: limited-access + display_name: Limited Access + custom_id: limited-access + type: api-key + credentials: + - ref: limited-access-key + name: limited-access-key + display_name: Limited Access Key + type: api-key + ttl: 0 + api_key: !secret {source: !env LIMITED_ACCESS_API_KEY} +``` + +For the existing model, add: + +```yaml +access: + auth_strategies: + - !ref client-key-auth + acls: + allow: [full-access, limited-access] +``` + +For each additional model, use `allow: [full-access]` with the same auth +strategy. Copy the existing model's supported shape with a distinct `ref`, +`name` and `config.route.model.values` alias, then set the selected upstream +target. Do not overwrite the original model list when adding models. + +Use `!ref` for the auth strategy relationship. For model access, kongctl +normalizes this reference to the strategy's **name**; do not substitute a +literal UUID. ACL entries are **consumer or group names**. Choose either +`allow` or `deny` for a model; the schema permits only one. Creating an auth +strategy alone does not protect a model: attach it through +`access.auth_strategies`. +Authentication identifies the caller; ACLs decide which model it may use. +See [AI Consumers][consumers] and [AI Consumer Groups][groups]. + +For group-based variants, inspect +`kongctl explain ai_gateways.consumer_groups`. Declarative consumer +membership and authenticated OIDC claims have different setup needs; +verify group names and claim mapping against the intended caller token. +Do not copy an SE environment's issuer, audience or group prefixes. + +## Verify the complete access matrix + +Prepare a bounded verification script using the same endpoint, model +aliases and `apikey` header. Read caller keys from the environment; do not +print them, enable shell tracing or embed them in saved examples. + +| Caller | Original model | Added model 1 | Added model 2 | +| --- | --- | --- | --- | +| Missing key | 401 | 401 | 401 | +| Invalid key | 401 | 401 | 401 | +| Full access | 200 + completion | 200 + completion | 200 + completion | +| Limited access | 200 + completion | 403 | 403 | + +These are the intended results for this key-auth/allow-ACL example. Make +both status and response semantics executable assertions. An upstream 401, +routing 404 or server failure does not count as an authorization denial. +First establish the allowed requests, then verify missing/invalid +credentials and denied model access. Do not accept any non-200 as proof +that the policy works. + +For example, after checking the expected HTTP status, assert the expected +gateway rejection message in the JSON body: + +```sh +jq -e --arg expected "$expected_message" \ + '.message == $expected and .error == null' "$response_file" >/dev/null +``` + +Kong's [Key Auth implementation][key-auth] reports a missing key as +`No API key found in request` and an invalid key as `Unauthorized`. +[ACL validation][acl] expects `You cannot consume this service`. Use these +as starting assertions and confirm the signatures for the selected runtime +during rehearsal; an unknown body fails the check and needs diagnosis. + +Check the verifier offline with synthetic responses before using it live: +a 401 provider error must fail even though its status matches the missing +key case, and a 200 without a completion must fail. Use request parameters +supported by the selected upstream models; token-budget options can differ. + +Make an unexpected acceptance fail the self-test process explicitly: + +```sh +if assert_response "$provider_response" 401 401 missing; then + echo "Verifier accepted a provider error as missing client credentials" >&2 + exit 1 +fi +``` + +Adapt that call to the verifier's interface. A bare `! assert_response ...` +does not enforce failure under `set -e`: Bash exempts negated commands from +errexit. Check the self-test by temporarily making the response assertion +always succeed in a disposable copy; the self-test must then fail. + +If configuration propagation delays a check, wait a bounded interval and +retry the affected case; stop after the stated bound. Record the model, +caller label, status and outcome. Do not persist credential-bearing request +logs. Re-run this matrix after the approved pipeline changes the gateway. + +## Credential lifecycle + +Supply distinct caller keys when creating these credentials, using deferred +`!secret`. This avoids depending on a server-generated key that may be +returned only once. Consumer `api_key` is create-only: rotating it requires +a new credential identity and deliberate retirement of the old credential. +Changing its environment variable on an unchanged apply is not rotation. + +Existing provider secrets also do not reconcile through ordinary drift. +For a requested provider rotation, inspect `kongctl plan --help` and use +the specific `--write-secret` selector while generating a new reviewed +plan. The resulting plan owns that write intent; do not add secret-selection +flags to `apply --plan`. Avoid blanket `--write-secrets` for routine CI. + +[consumers]: https://developer.konghq.com/ai-gateway/entities/ai-consumer/ +[groups]: https://developer.konghq.com/ai-gateway/entities/ai-consumer-group/ +[key-auth]: + https://github.com/Kong/kong/blob/master/kong/plugins/key-auth/handler.lua +[acl]: https://developer.konghq.com/how-to/configure-oidc-with-acl-auth/