test(docs): automated documentation quality checks (ENG-23) - #89
Open
YacineMK wants to merge 12 commits into
Open
test(docs): automated documentation quality checks (ENG-23)#89YacineMK wants to merge 12 commits into
YacineMK wants to merge 12 commits into
Conversation
…ftest token handling
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (27)
🚧 Files skipped from review as they are similar to previous changes (16)
WalkthroughThe pull request adds Python and TypeScript documentation validation to CI, generates and executes TypeScript snippets, centralizes fixtures and model inventory checks, and updates documentation links, metadata, provider mappings, and model examples. ChangesDocumentation quality and execution
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🟡 Moderate · up to The PR adds live documentation validation but leaves 11 unsupported or nonexistent provider/model references unresolved, and four new linkup identifiers still need confirmation. Users could copy examples that fail, so the affected documentation should be corrected or explicitly accepted before merging. Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant PythonValidators
participant Documentation
participant EdenAIAPIs
GitHubActions->>PythonValidators: run documentation validators
PythonValidators->>Documentation: scan pages and navigation
PythonValidators->>EdenAIAPIs: fetch specifications and inventories
PythonValidators-->>GitHubActions: report validation results
sequenceDiagram
participant GitHubActions
participant Bun
participant GlobalSetup
participant SnippetExtractor
participant GeneratedSnippets
GitHubActions->>Bun: start TypeScript tests
Bun->>GlobalSetup: load environment and preload setup
GlobalSetup->>SnippetExtractor: generate snippets and fixtures
Bun->>GeneratedSnippets: execute generated snippets
GeneratedSnippets-->>Bun: return execution status
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/test-snippets.yml:
- Around line 33-36: Restrict API-token environment variables in
.github/workflows/test-snippets.yml:33-36 to trusted events such as
protected-branch pushes, scheduled runs, or approved protected environments; at
.github/workflows/test-snippets.yml:45-46 run Python snippets without
credentials or move credentialed execution to a trusted workflow; at
.github/workflows/test-snippets.yml:67-70 remove the sandbox token from the
pull-request TypeScript job; at .github/workflows/test-snippets.yml:84-86 run
credentialed TypeScript snippets only in the trusted workflow; and in
tests/ts/snippets.test.ts:10-20 replace the process.env spread with an explicit
minimal environment.
In `@tests/config_validator.py`:
- Around line 103-111: Update test_config_guide so path existence, content
loading, and local JSON/YAML/TOML/URL validation always run regardless of
EDEN_AI_SANDBOX_API_TOKEN. Restrict the token-based skip to the
get_model_inventory() and subsequent live-inventory model validation, preserving
the existing behavior when the token is available.
In `@tests/link_checker.py`:
- Around line 96-104: Update _is_checkable and the external-link
request/redirect flow to reject private or otherwise non-global destinations,
including IPv4, IPv6, and link-local literals, before any request is made.
Resolve hostnames and validate their resulting addresses as well, and reapply
the same validation to every redirect target; alternatively, route all checks
through an established egress-controlled proxy or allowlist.
In `@tests/README.md`:
- Line 142: Update the stale-reference behavior description in tests/README.md
to match tests/model_provider_validator.py: unmatched candidates must fail
unless they are placeholders or have an excluded MIME prefix. Do not claim that
known stale references are tracked unless an explicit allowlist is implemented.
In `@tests/snippet_extractor.py`:
- Around line 336-350: Update extract_all_ts() so each generated
TypeScript/JavaScript file from a block includes the required standalone setup,
including url and headers when referenced, rather than relying on declarations
from earlier blocks on the page. Apply this during the path.write_text
generation flow while preserving existing filename, skip-marker, and
snippet-rewrite behavior.
In `@v2/index.mdx`:
- Line 3: Update the documentation links in v2/index.mdx (lines 3-3) and
v3/overview/ai-gateway.mdx (lines 162-162) to point V2 or previous-version
documentation text to https://www.edenai.co/docs/v2, preserving the surrounding
wording.
In `@v3/expert-models/features/audio/tts.mdx`:
- Line 65: Update the Google TTS model entry for gemini-3.1-flash-tts-preview to
use the correct Gemini 3.1 model identifier instead of
audio/tts/google/gemini-2.5-flash-tts, and adjust its listed price if required
by the intended 3.1 model.
In `@v3/expert-models/features/image/background-removal.mdx`:
- Line 50: Update the Stability AI row in the model table to replace the Api4ai
model identifier with the correct Stability AI model ID, while preserving the
provider label and pricing.
In `@v3/expert-models/features/image/generation.mdx`:
- Around line 76-78: Keep each catalog row’s provider, model identifier,
image-generation ID, and price aligned: in
v3/expert-models/features/image/generation.mdx lines 76-78, preserve distinct
IDs and prices for gpt-image-1, gpt-image-1.5, and gpt-image-1-mini, or remove
those rows and retain only gpt-image-2; in
v3/expert-models/features/web/research-async.mdx line 58,
v3/expert-models/features/web/scraping.mdx line 60, and
v3/expert-models/features/web/search.mdx line 71, keep each Firecrawl entry
under its corresponding /firecrawl ID or remove/rename it consistently as
linkup.
In `@v3/integrations/codex-cli.mdx`:
- Around line 110-115: Update the Codex smoke-test request in the documented
command to use the Responses API: replace the chat-completions messages payload
with an input payload while retaining the full azure/gpt-5.1-codex model
identifier.
In `@v3/integrations/pi.mdx`:
- Line 52: Update the configuration example to use the required production or
sandbox token variable: replace the EDEN_AI_API_KEY reference with the
appropriate api_token or sandbox_api_token field, and update the adjacent
environment or test fixture consistently so tests cannot consume a production
credential.
In `@v3/llms/image-generation.mdx`:
- Around line 49-52: Update the image model table to use unique provider/model
IDs matching the /v3/images/models format, replacing Universal AI-prefixed
values with IDs such as google/imagen-4.0-generate-001 and openai/gpt-image-2.
Remove the duplicate OpenAI entry and replace the Amazon row’s incomplete ID
with its provider/model-specific model ID.
In `@v3/llms/listing-models.mdx`:
- Line 107: Update the Stable alias entry in the model listing to replace
anthropic/claude-sonnet-4-6 with the catalog’s alias_of-backed
anthropic/claude-sonnet-latest, or remove the former from this entry and list it
under the versioned models section.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c382557f-f1ae-4ae1-af2c-2b3cd8f12dd3
⛔ Files ignored due to path filters (1)
tests/ts/bun.lockis excluded by!**/*.lock
📒 Files selected for processing (41)
.github/workflows/test-snippets.yml.gitignoreindex.mdxtests/README.mdtests/api_reference_validator.pytests/config_validator.pytests/conftest.pytests/helpers/edenai_inventory.pytests/helpers/file_generators.pytests/helpers/model_names.pytests/link_checker.pytests/model_provider_validator.pytests/snippet_extractor.pytests/ts/bunfig.tomltests/ts/globalSetup.tstests/ts/package.jsontests/ts/snippets.test.tsv2/index.mdxv3/.claude/settings.local.jsonv3/expert-models/features/audio/tts.mdxv3/expert-models/features/image/background-removal.mdxv3/expert-models/features/image/generation.mdxv3/expert-models/features/web/research-async.mdxv3/expert-models/features/web/scraping.mdxv3/expert-models/features/web/search.mdxv3/integrations/claude-code.mdxv3/integrations/codex-cli.mdxv3/integrations/continue-dev.mdxv3/integrations/librechat.mdxv3/integrations/open-webui.mdxv3/integrations/openai-sdk-python.mdxv3/integrations/openai-sdk-typescript.mdxv3/integrations/openclaw.mdxv3/integrations/pi.mdxv3/llms.txtv3/llms/chat-completions.mdxv3/llms/image-generation.mdxv3/llms/listing-models.mdxv3/overview/ai-gateway.mdxv3/quickstart/first-expert-model-call.mdxv3/quickstart/first-llm-call.mdx
💤 Files with no reviewable changes (1)
- v3/.claude/settings.local.json
Comment on lines
+33
to
+36
| env: | ||
| EDEN_AI_BASE_URL: ${{ vars.EDEN_AI_BASE_URL || 'https://staging-api.edenai.run' }} | ||
| EDEN_AI_SANDBOX_API_TOKEN: ${{ secrets.EDEN_AI_SANDBOX_TOKEN }} | ||
| EDEN_AI_PRODUCTION_API_TOKEN: ${{ secrets.EDEN_AI_PRODUCTION_TOKEN }} |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Do not execute pull-request content with API credentials.
A same-repository pull request can add a documentation snippet that reads the sandbox token and sends it to an external service. The TypeScript runner also forwards the complete process environment to each generated snippet.
.github/workflows/test-snippets.yml#L33-L36: bind API tokens only for trusted events, such as protected-branch pushes, scheduled runs, or approved protected environments..github/workflows/test-snippets.yml#L45-L46: run Python snippet execution without credentials on pull requests, or move credentialed execution to a trusted workflow..github/workflows/test-snippets.yml#L67-L70: remove the sandbox token from the pull-request TypeScript job..github/workflows/test-snippets.yml#L84-L86: run credentialed TypeScript snippets only in the trusted workflow.tests/ts/snippets.test.ts#L10-L20: pass an explicit minimal environment instead of spreadingprocess.env.
📍 Affects 2 files
.github/workflows/test-snippets.yml#L33-L36(this comment).github/workflows/test-snippets.yml#L45-L46.github/workflows/test-snippets.yml#L67-L70.github/workflows/test-snippets.yml#L84-L86tests/ts/snippets.test.ts#L10-L20
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/test-snippets.yml around lines 33 - 36, Restrict API-token
environment variables in .github/workflows/test-snippets.yml:33-36 to trusted
events such as protected-branch pushes, scheduled runs, or approved protected
environments; at .github/workflows/test-snippets.yml:45-46 run Python snippets
without credentials or move credentialed execution to a trusted workflow; at
.github/workflows/test-snippets.yml:67-70 remove the sandbox token from the
pull-request TypeScript job; at .github/workflows/test-snippets.yml:84-86 run
credentialed TypeScript snippets only in the trusted workflow; and in
tests/ts/snippets.test.ts:10-20 replace the process.env spread with an explicit
minimal environment.
Source: Linters/SAST tools
Comment on lines
+103
to
+111
| @pytest.mark.parametrize("guide", CONFIG_GUIDES, ids=lambda p: Path(p).stem) | ||
| def test_config_guide(guide: str) -> None: | ||
| if not os.environ.get("EDEN_AI_SANDBOX_API_TOKEN"): | ||
| pytest.skip("EDEN_AI_SANDBOX_API_TOKEN not set") | ||
|
|
||
| path = DOCS_ROOT / guide | ||
| assert path.exists(), f"Guide not found: {path}" | ||
| content = path.read_text(encoding="utf-8") | ||
| inventory = get_model_inventory() |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Run local validation when the token is absent.
Line 105 skips JSON, YAML, TOML, and URL validation when EDEN_AI_SANDBOX_API_TOKEN is absent. Only get_model_inventory() requires the token. Run the local checks unconditionally, and skip only the live-inventory model validation.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/config_validator.py` around lines 103 - 111, Update test_config_guide
so path existence, content loading, and local JSON/YAML/TOML/URL validation
always run regardless of EDEN_AI_SANDBOX_API_TOKEN. Restrict the token-based
skip to the get_model_inventory() and subsequent live-inventory model
validation, preserving the existing behavior when the token is available.
Comment on lines
+336
to
+350
| for rel in TS_GUIDES: | ||
| mdx_path = DOCS_ROOT / rel | ||
| if not mdx_path.exists(): | ||
| raise FileNotFoundError(f"Guide not found: {mdx_path}") | ||
| blocks = extract_ts_blocks(mdx_path) | ||
| if not blocks: | ||
| continue | ||
|
|
||
| stem = sanitize_filename(mdx_path) | ||
| files: list[dict] = [] | ||
| for i, block in enumerate(blocks, start=1): | ||
| skip_marker = ".skip" if block["skip"] else "" | ||
| name = f"{stem}__block_{i}{skip_marker}.{block['ext']}" | ||
| path = GENERATED_TS_DIR / name | ||
| path.write_text(rewrite_ts_snippet(block["code"], block["ext"])) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
for guide in \
v3/integrations/openai-sdk-typescript.mdx \
v3/integrations/langchain.mdx \
v3/integrations/pi.mdx
do
echo "===== $guide ====="
nl -ba "$guide" | sed -n '/^.*```\(typescript\|ts\|tsx\|javascript\|js\|jsx\)/,/^.*```/p'
doneRepository: edenai/docs
Length of output: 244
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "===== Guide files ====="
for guide in \
v3/integrations/openai-sdk-typescript.mdx \
v3/integrations/langchain.mdx \
v3/integrations/pi.mdx
do
if [ -f "$guide" ]; then
echo "FOUND $guide"
else
echo "MISSING $guide"
fi
done
echo "===== All TS-like fenced blocks in TS_GUIDES files ====="
python3 - <<'PY'
from pathlib import Path
import re
files = [
Path("v3/integrations/openai-sdk-typescript.mdx"),
Path("v3/integrations/langchain.mdx"),
Path("v3/integrations/pi.mdx"),
]
fence_re = re.compile(r"^```(\w+)\s*$")
skip_re = re.compile(r"\{\s*\n?\s*{}\s*\n?\s*/\*\s*skip-test\s*\*/\s*\n?\s*\}\s*$|^{\s*\n?\s*{}\s*\n?\s*/\*\s*skip-test\s*\*/\s*\n?\s*\}\s*$", re.M)
for guide in files:
if not guide.is_file():
print(f"=== {guide} (missing) ===")
continue
print(f"=== {guide} ===")
lines = guide.read_text(encoding="utf-8").splitlines()
in_fence = False
fence_lang = None
block_lines = []
for no, line in enumerate(lines, 1):
if not in_fence:
m = fence_re.match(line)
if m:
in_fence = True
fence_lang = m.group(1)
block_lines = []
else:
if line.startswith("```"):
code = "\n".join(block_lines)
print(f"line {no - len(block_lines):4}-{no:4} {fence_lang}")
print(code or "<empty>")
if code and skip_re.search(code):
print("<<< SKIP-MARKED BLOCK >>>")
print("---")
in_fence = False
fence_lang = None
block_lines = []
else:
block_lines.append(line)
print()
PY
echo "===== Search for url/headers across guide files and generated test files ====="
python3 - <<'PY'
from pathlib import Path
import re
files = [
Path("v3/integrations/openai-sdk-typescript.mdx"),
Path("v3/integrations/langchain.mdx"),
Path("v3/integrations/pi.mdx"),
]
fence_re = re.compile(r"^```(\w+)\s*$")
identifier_re = re.compile(r"\b(url|headers|fetch)\b", re.M)
for guide in files:
if not guide.is_file():
continue
print(f"=== {guide} ===")
lines = guide.read_text(encoding="utf-8").splitlines()
in_fence = False
block_lines = []
env = {}
for no, line in enumerate(lines, 1):
if not in_fence:
m = fence_re.match(line)
if m:
in_fence = True
block_lines = []
else:
if line.startswith("```"):
print(f"block {env.get('lang')!r} lines {env.get('start')}-{no}:")
code = "\n".join(block_lines)
if identifier_re.search(line := "\n".join(block_lines)):
for match in reversed(list(identifier_re.finditer(code))):
print(f" {match.group()} at char {match.start()}")
env = {"lang": m.group(1), "start": no - len(block_lines)}
in_fence = False
block_lines = []
else:
block_lines.append(line)
generated = Path("generated-js")
if generated.exists():
print(f"=== generated-js ({len(list(generated.glob('*')))} files) ===")
for path in sorted(generated.glob("*")):
text = path.read_text(encoding="utf-8", errors="replace")
matches = list(identifier_re.finditer(text))
if len(matches) >= 2:
print(path.relative_to(generated.parent), "has", len(matches), "matches")
for label, text in [("url", "url"), ("headers", "headers"), ("fetch", "fetch")]:
print(f" {label}: {text in text}")
PYRepository: edenai/docs
Length of output: 663
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "===== snippet_extractor outline / relevant sections ====="
python3 - <<'PY'
from pathlib import Path
p = Path("tests/snippet_extractor.py")
print(f"exists={p.exists()}")
if p.exists():
lines = p.read_text(encoding="utf-8").splitlines()
for i, line in enumerate(lines, 1):
if "TS_GUIDES" in line or "extract_ts_blocks" in line or "rewrite_ts_snippet" in line or "GENERATED_TS_DIR" in line or "sanitize_filename" in line or "extract_all_ts" in line:
print(f"{i}: {line}")
PY
echo "===== Read relevant tests/snippet_extractor.py section ====="
python3 - <<'PY'
from pathlib import Path
p = Path("tests/snippet_extractor.py")
if not p.exists():
print("MISSING")
else:
lines = p.read_text(encoding="utf-8").splitlines()
start, end = 1, len(lines)
for needle in ("TS_GUIDES", "extract_ts_blocks", "rewrite_ts_snippet"):
for i, line in enumerate(lines, 1):
if needle in line:
start = min(start, max(1, i-30))
end = max(end, min(len(lines), i+60))
for i in range(start, end+1):
print(f"{i:4}: {lines[i-1]}")
PY
echo "===== Search exact fenced block syntax in target guides ====="
python3 - <<'PY'
from pathlib import Path
import re
target = Path("tests/snippet_extractor.py")
if not target.exists():
raise SystemExit("tests/snippet_extractor.py missing")
lines = target.read_text(encoding="utf-8").splitlines()
# Extract TS_GUIDES literal from file to avoid relying on hard-coded guides in this verifier.
text = "\n".join(lines)
m = re.search(r"TS_GUIDES\s*=\s*\[(.*?)\]", text, re.S)
if not m:
print("TS_GUIDES not found")
raise SystemExit(1)
glob_str = "[" + m.group(1).split("]",1)[0] + "]"
guides = re.findall(r"'([^']+)'|\"([^\"]+)\"", glob_str)
guides = [g for pair in guides for g in pair]
print("TS_GUIDES:", guides)
fence_re = re.compile(r"^```([A-Za-z0-9_]+)\s*$")
skip_re = re.compile(r"\{\s*\n?\s*{}\s*\n?\s*/\*\s*skip-test\s*\*/\s*\n?\s*\}\s*$|^{\s*\n?\s*{}\s*\n?\s*/\*\s*skip-test\s*\*/\s*\n?\s*\}\s*$", re.M)
ts_langs = {"ts", "typescript", "tsx", "tsx-typescript", "js", "javascript", "jsx"}
for guide_path in guides:
guide = Path(guide_path)
print(f"=== {guide_path} exists={guide.exists()} ===")
if not guide.exists():
continue
ns = guide.read_text(encoding="utf-8").splitlines()
blocks = []
in_fence = False
fence_lang = None
block_lines = []
block_start = 0
for no, line in enumerate(ns, 1):
if not in_fence:
fm = fence_re.match(line)
if fm and fm.group(1) in ts_langs:
in_fence = True
fence_lang = fm.group(1)
block_start = no
block_lines = []
else:
if line.startswith("```"):
blocks.append((block_start, no, fence_lang, "\n".join(block_lines), skip_re.search(block_lines)))
in_fence = False
fence_lang = None
block_lines = []
else:
block_lines.append(line)
for start, end, lang, code, skip in blocks:
print(f"block {start}-{end} lang={lang} skip={bool(skip)}")
print(code[:2000] if code else "<empty>")
print("---")
PYRepository: edenai/docs
Length of output: 16306
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import re
target = Path("tests/snippet_extractor.py")
text = target.read_text(encoding="utf-8")
m = re.search(r"TS_GUIDES:\s*list\[str\]\s*=\s*\[(.*?)\]", text, re.S)
if not m:
raise SystemExit("TS_GUIDES not found")
glob_text = "[" + m.group(1).split("]", 1)[0] + "]"
guides = re.findall(r"[\"']([^\"']+)[\"']", glob_text)
print("TS_GUIDES:")
for g in guides:
print(f" {g}")
skip_re = re.compile(r"(/\*\s*skip-test\s*\*/|{\s*\n?\s*{}\s*\n?\s*/\*\s*skip-test\s*\*/\s*\n?\s*}\s*)", re.S)
fence_re = re.compile(
r"^```(?P<lang>typescript|ts|tsx|javascript|js|jsx)"
r"(?:[ \t]+[^\n]*)?[ \t]*\n(?P<body>.*?)^\s*```",
re.MULTILINE | re.DOTALL,
)
for guide_path in guides:
guide = Path(guide_path)
print(f"\n=== {guide_path} exists={guide.exists()} ===")
if not guide.exists():
continue
all_identifiers = []
content = guide.read_text(encoding="utf-8")
for match in fence_re.finditer(content):
body = match.group("body")
preceding = content[: match.start()]
recent_lines = preceding.rsplit("\n", 3)[-3:]
skip = bool(any(skip_re.search(line) for line in recent_lines))
block_line = preceding.count("\n") + 2
block_identifiers = re.findall(r"\b(url|headers|fetch)\b", body)
all_identifiers.extend((block_line, body.count("\n") + 1, block_identifiers))
print(f"block line {block_line}: {len(all_identifiers)} identifier refs={block_identifiers} skip={skip}")
if body.strip():
for idx, line_no, _ in all_identifiers[-1:]:
# print last block body unless already printed in a long list
print(body[:180].replace("\n", "\\n"))
# Also print if identifiers appear outside TS fences because Python extractor can rewrite those later
all_refs = re.findall(r"\b(url|headers|fetch)\b", content)
print(f"total url/headers/fetch refs in file={len(all_refs)}")
PYRepository: edenai/docs
Length of output: 505
Add required setup to each standalone TS/JS file.
extract_all_ts() writes each fence body into its own generated file, so later files cannot access variables declared by earlier files. Add the minimum setup, such as url and headers, inside files that depend on them instead of relying on page-level declaration order.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/snippet_extractor.py` around lines 336 - 350, Update extract_all_ts()
so each generated TypeScript/JavaScript file from a block includes the required
standalone setup, including url and headers when referenced, rather than relying
on declarations from earlier blocks on the page. Apply this during the
path.write_text generation flow while preserving existing filename, skip-marker,
and snippet-rewrite behavior.
Source: Coding guidelines
Comment on lines
+110
to
+115
| -d '{"model": "azure/gpt-5.1-codex", "messages": [{"role": "user", "content": "ping"}]}' | ||
| ``` | ||
|
|
||
| ### Model not found | ||
|
|
||
| Use the full `provider/model` string (e.g. `openai/gpt-5.1-codex`, not `gpt-5.1-codex`). Confirm the ID is in the catalog returned by `GET /v3/models`. | ||
| Use the full `provider/model` string (e.g. `azure/gpt-5.1-codex`, not `gpt-5.1-codex`). Confirm the ID is in the catalog returned by `GET /v3/models`. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
git ls-files | rg '^v3/integrations/codex-cli\.mdx$' || true
echo "== relevant excerpt =="
if [ -f v3/integrations/codex-cli.mdx ]; then
nl -ba v3/integrations/codex-cli.mdx | sed -n '35,130p'
fi
echo "== search for codex references =="
rg -n "codex|wire_api|responses|chat/completions|gpt-5\.1-codex|Codex" v3/integrations/codex-cli.mdx || true
echo "== repo-wide codex occurrences (concise) =="
rg -n "codex|gpt-5\.1-codex|wire_api|v3/responses" . || true
echo "== git diff stat/name/status =="
git diff --stat || true
git diff -- v3/integrations/codex-cli.mdx | sed -n '1,220p' || trueRepository: edenai/docs
Length of output: 260
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path('v3/integrations/codex-cli.mdx')
if not p.exists():
raise SystemExit
text = p.read_text()
lines = text.splitlines()
for needle in ['wire_api', 'chat/completions', '/v3/responses', 'gpt-5.1-codex', '"input"', '"messages"']:
print(f'-- {needle!r} --')
for i,l in enumerate(lines, 1):
if needle in l:
print(f'{i}: {l}')
PYRepository: edenai/docs
Length of output: 1072
🌐 Web query:
OpenAI Codex responses endpoint vs chat completions wire_api responses responses endpoint input
💡 Result:
As of August 9, 2026, the OpenAI Codex ecosystem has fully transitioned to the Responses API for its primary interactions, with the legacy Chat Completions API deprecated and unsupported for Codex-specific workflows [1][2][3]. Key differences and status: 1. Relationship between APIs: The Responses API is an evolution of the Chat Completions API designed specifically to handle complex, agentic coding workflows [4]. While Chat Completions treats input as a simple list of messages, the Responses API uses a system of typed "Items" (e.g., messages, reasoning, function_call_outputs) and supports stateful features like previous_response_id for chaining interactions [4][2]. 2. Codex API Support: Codex models and the official Codex CLI/desktop app now exclusively use the wire_api = "responses" protocol [1][2]. The legacy Chat Completions API is considered an obsolete target for Codex and is no longer supported for new integrations [1][2][3]. Configurations or custom providers still attempting to use wire_api = "chat" will fail as support was fully removed in February 2026 [1]. 3. Endpoint and Implementation: - Official Codex traffic primarily utilizes relay endpoints such as https://chat-response.codex-openai.com/relay/responses or internal paths like https://chatgpt.com/backend-api/codex/responses [5][6][7]. - These endpoints are generally intended for first-party clients (the Codex or ChatGPT desktop apps) and require specific headers and authentication tokens (e.g., ChatGPT-Account-ID, OpenAI-Beta: responses=v1) [5][6]. - There is no publicly documented, stable contract for third-party clients to interface directly with these subscription-based relay endpoints; they are subject to internal changes, version coupling, and circuit-breaker protections [6][7]. Developers building integrations should use the standard OpenAI platform API rather than attempting to route through internal Codex relay endpoints, as the latter are designed for internal application state management rather than public API consumption [6].
Citations:
- 1: Deprecating `chat/completions` support in Codex openai/codex#7782
- 2: https://wavespeed.ai/blog/developer-friction/responses-api-vs-chat-completions-codex/
- 3: https://community.openai.com/t/codex-api-role-issues-in-chat-and-responses-api/1373205/2
- 4: https://developers.openai.com/api/docs/guides/migrate-to-responses
- 5: ChatGPT.app Codex relay returns 403 'invalid or disabled credential' after v26.715.31925 update openai/codex#33969
- 6: Is there a documented auth contract for third-party clients using a ChatGPT subscription with the Responses API? openai/codex#36886
- 7: Unable to initiate or continue new or existing chats ChatGPT App. Complete service outage openai/codex#35315
Use the Responses API for the Codex smoke test.
The provider sets wire_api = "responses", but the test sends a chat-completions request with messages. Replace it with a Responses request that uses input so a passing check actually exercises the configured protocol.
Proposed smoke-test correction
-curl -X POST https://api.edenai.run/v3/chat/completions \
+curl -X POST https://api.edenai.run/v3/responses \
-H "Authorization: Bearer $EDENAI_API_KEY" \
-H "Content-Type: application/json" \
- -d '{"model": "azure/gpt-5.1-codex", "messages": [{"role": "user", "content": "ping"}]}'
+ -d '{"model": "azure/gpt-5.1-codex", "input": "ping"}'📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| -d '{"model": "azure/gpt-5.1-codex", "messages": [{"role": "user", "content": "ping"}]}' | |
| ``` | |
| ### Model not found | |
| Use the full `provider/model` string (e.g. `openai/gpt-5.1-codex`, not `gpt-5.1-codex`). Confirm the ID is in the catalog returned by `GET /v3/models`. | |
| Use the full `provider/model` string (e.g. `azure/gpt-5.1-codex`, not `gpt-5.1-codex`). Confirm the ID is in the catalog returned by `GET /v3/models`. | |
| curl -X POST https://api.edenai.run/v3/responses \ | |
| -H "Authorization: Bearer $EDENAI_API_KEY" \ | |
| -H "Content-Type: application/json" \ | |
| -d '{"model": "azure/gpt-5.1-codex", "input": "ping"}' |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@v3/integrations/codex-cli.mdx` around lines 110 - 115, Update the Codex
smoke-test request in the documented command to use the Responses API: replace
the chat-completions messages payload with an input payload while retaining the
full azure/gpt-5.1-codex model identifier.
| name: "Eden AI", | ||
| api: "openai-completions", | ||
| baseUrl: "https://api.edenai.run/v3", | ||
| apiKey: process.env.EDEN_AI_API_KEY, |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Use the required production and sandbox token variables.
The new example reads process.env.EDEN_AI_API_KEY. V3 examples must use api_token for production and sandbox_api_token for testing. Rename the variable and update the adjacent environment or test fixture so test execution cannot use a production credential.
As per coding guidelines, distinguish api_token for production from sandbox_api_token for testing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@v3/integrations/pi.mdx` at line 52, Update the configuration example to use
the required production or sandbox token variable: replace the EDEN_AI_API_KEY
reference with the appropriate api_token or sandbox_api_token field, and update
the adjacent environment or test fixture consistently so tests cannot consume a
production credential.
Source: Coding guidelines
…lidator/readme fixes
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/link_checker.py`:
- Around line 128-134: Update the request flow in the link-checking function
around _classify_host so redirects are not followed automatically; disable
redirect handling for both the HEAD request and GET fallback, then parse and
validate every Location target with _classify_host before issuing the next
request, rejecting private or link-local destinations.
- Around line 31-36: Update _classify_host so transient socket.gaierror results
are not retained by the process-wide `@cache`; ensure DNS failures can be retried
for later links while preserving caching for successful classifications,
preferably with a bounded lifetime as requested.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 9b28f335-6a30-4927-87f1-4b257148707d
📒 Files selected for processing (8)
.github/workflows/test-snippets.ymlindex.mdxtests/README.mdtests/config_validator.pytests/link_checker.pytests/ts/snippets.test.tsv2/index.mdxv3/overview/ai-gateway.mdx
🚧 Files skipped from review as they are similar to previous changes (6)
- index.mdx
- v2/index.mdx
- v3/overview/ai-gateway.mdx
- tests/README.md
- tests/config_validator.py
- .github/workflows/test-snippets.yml
- pass sandbox/production tokens unconditionally in CI: blanking them on pull_request events made every content check skip while reporting green (GitHub already withholds secrets from fork-originated PR runs) - when the production token is rejected (401), unset it and record the rejection in the shared state file so prod-token doc tests skip in xdist workers too, matching the comment's stated behavior - restore correct model strings where the stale-reference rename copied a neighboring row's string: tts gemini-3.1-flash-tts-preview, background removal stabilityai, image generation gpt-image-1/1.5/1-mini, and the firecrawl rows on the three web feature pages - llms/image-generation: dedupe the OpenAI row, restore the Amazon model ID, and keep table rows in the plain provider/model form the /v3/images endpoint expects - restore stable aliases: anthropic/claude-sonnet-latest in the stable alias example, cohere/command-r(-plus) across integration guides - prune stale generated_ts snippets before extraction; the bun runner discovers files by globbing the directory - README: document the strip_tool_alias inventory bypass; drop em dashes introduced in README and codex-cli lists
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automated documentation quality checks under ENG-23.
.ts/.tsx/.mjs/.cjsblocks fromopenai-sdk-typescript,langchain,piguides and executes them against staging.provider/modelstrings against live inventory.docs.jsonare reachable, cross-checks everyapi.edenai.run/v[23]/…URL in prose..mdxpages forprovider/modelrefs, cross-checks against/v3/models+/v3/info+ probed embeddings.docs.jsonnav path resolves.test-snippets.yml(Python + TS),cancel-in-progress: false, weekly cron.Bundled doc fixes: renamed stale model refs (
cohere/command-r-plus→-plus-08-2024,mistral/mistral-large→-large-latest, codex prefixopenai/→azure/, several image/audio/translation catalog rewrites); rewrotepi.mdxTS snippet against the realExtensionAPIshape; replaced brokennode-fetch + form-datavision snippet with nativefetch + Blob; replaced deadold-docs.edenai.codomain.Test plan
pytest tests/test_snippets_execute.py tests/config_validator.py tests/api_reference_validator.py tests/model_provider_validator.py tests/link_checker.py -n0 --no-cov— passes except the 11 flagged belowcd tests/ts && bun test— 13 pass / 1 skip / 0 failworkflow_dispatchrun in Actions❓ Question — 11 failing tests need a product decision
The model/provider validator flags 11 backticked
provider/modelstrings across 6 doc pages that don't exist in the live Eden AI catalog (/v3/modelsor/v3/info). These aren't code bugs — they're doc content bugs the validator surfaced. I need someone with platform context to decide the intent for each:A. `image/generation/bytedance/*` × 5 + `image/generation/minimax` × 2 — in `v3/expert-models/features/image/generation.mdx`. Live API says these providers don't support `image/generation`.
B. `video/generation_async/pixverse/*` × 5 — in `v3/expert-models/features/video/generation-async.mdx`. Live API says these models don't exist.
C. `web/crawl_async/firecrawl` + `web/map/firecrawl` — in the corresponding `v3/expert-models/features/web/*.mdx` pages. Subfeatures exist, firecrawl isn't a supported provider for them.
For each, which one applies?
Also flagging: `/v3/info` doesn't list `web/batch_scrape_async` or `web/structured_extraction_async` even though the API accepts them. Worked around via runtime probing in `tests/helpers/edenai_inventory.py`; backend may want to expose them in `/v3/info`.
Summary by CodeRabbit
Documentation
New Features
Chores