Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,20 @@ retirement. Target native background termination only by observed current-Turn
item/process identity, and verify Files plus retained interrupted history through
the maintained native client before any production ownership or public admission.

The optional Codex file installer runs through the existing native process interface
with bounded stdin chunks, declared length and a SHA-256 commit trailer. It fills
the demonstrated native hard-link overwrite and whole-message size gaps; it is
not a second filesystem service or public admission. Reuse held-directory traversal
and existing rustix directory-relative operations for replacement. Keep preparation, caller
authorization and uncertain mutation recovery in their existing owning layers.
The caller must prevent concurrent workspace writers during installation; native
tools and background processes can otherwise interfere with staging. Public upload
admission must establish this prerequisite or provide a separately verified safe
commit mechanism. Temporary-file cleanup is best effort.
A queued stdin receipt, missing helper result or process termination is not a file
commit receipt. See the [installer contract](packages/codex-executor/README.md#scoped-file-installer)
for private limits, cleanup, metadata and concurrency semantics.

The private [retirement qualification](services/agents-api/tests/native/retirement/README.md)
separates native connection/processor shutdown from already admitted filesystem
work. Its hashed test-only scheduling overlay is not a production native patch.
Expand Down
2 changes: 1 addition & 1 deletion contracts/agents-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ the Python SDK. Vault HTTP paths start at `/vaults`, not `/agents/vaults`.
| sessions.subagents.turns | retrieve, list | Missing |
| sessions.subagents.turns.items | list | Missing |
| environments | retrieve | Supported self-hosted profile: durable status and safe empty installation metadata; hosted/populated inventory remains missing |
| environments.files | create, list | Missing |
| environments.files | create, list | [Partial bounded Codex list](environment-files.md); create remains missing |
| environments.templates | create, retrieve, update, list, delete | Missing |
| vaults | create, retrieve, list, delete | Create/retrieve/list/delete with independent tenant persistence, stored status filtering, atomic Credential cascade and frozen Session attachments; archive semantics and full hosted lifecycle parity remain missing |
| vaults.credentials | create, retrieve, update, list, delete | Static-bearer create/retrieve/list/token replacement/deletion with scoped encrypted storage; Session attachment and exact-URL HTTPS MCP binding; OAuth, archive semantics and full hosted lifecycle parity remain missing |
Expand Down
2 changes: 2 additions & 0 deletions packages/codex-executor/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 7 additions & 2 deletions packages/codex-executor/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ publish = false
name = "agents-api-codex-directory"
path = "src/bin/directory.rs"

[[bin]]
name = "agents-api-codex-write"
path = "src/bin/write.rs"

[dependencies]
codex-api = { git = "https://github.com/openai/codex", rev = "3d2ee51ca2d5db578f328aa75e20aa22c0197c9a" }
codex-exec-server = { git = "https://github.com/openai/codex", rev = "3d2ee51ca2d5db578f328aa75e20aa22c0197c9a" }
Expand All @@ -18,12 +22,13 @@ http = "1.3.1"
rustix = { version = "=1.1.4", features = ["fs"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha2 = "=0.10.9"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "signal"] }
url = "2"
uuid = "1"
uuid = { version = "1", features = ["v4"] }

[dev-dependencies]
uuid = { version = "1", features = ["v4"] }
tempfile = "=3.27.0"

[patch.crates-io]
tokio-tungstenite = { git = "https://github.com/openai-oss-forks/tokio-tungstenite", rev = "0e5b2d73aa18dd9f0a50ee9ff199d5aef7594186" }
Expand Down
42 changes: 42 additions & 0 deletions packages/codex-executor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,48 @@ not change stock native filesystem methods, create a daemon connection, or enabl
public Files. The [native fixture](../../services/agents-api/tests/native/directory/README.md)
qualifies the standalone helper independently of later adapter/public wiring.

## Scoped file installer

The optional `agents-api-codex-write` helper addresses two pinned native write
limitations: hard-link targets are modified in place, and base64 encoding a
50 MiB file exceeds the native 64 MiB message bound. Install this helper outside
the writable workspace and invoke it directly through the native process API,
with restricted network and a filesystem policy limited to that workspace and
required helper/runtime reads. It does not authorize callers or enable Files.create.

Arguments are the authorized absolute root, a nonempty relative file path and
its declared byte count (0–50 MiB). Stream those bytes in bounded native stdin
chunks, followed by their 32-byte binary SHA-256 digest. This is one private frame;
there is no second request on that process. The native process protocol has no
stdin-close method, so the digest terminates the frame without waiting for EOF.
Extra bytes after the frame are not consumed. A process/write accepted receipt
means queued input, not committed file contents.

The helper reuses held-directory no-follow traversal, rejects existing nonregular
targets and requires an existing parent. It writes a fresh mode-0600 temporary
file with the existing rustix openat/renameat operations, checks the declared byte count and
digest, syncs the file, then replaces the destination directory entry and syncs
the parent. Existing hard links retain their original inode and contents. This
private replacement policy does not preserve destination mode/ownership metadata
or establish official overwrite semantics. The caller must prevent concurrent
workspace writers throughout installation, including native tools and background
processes that can modify the staging file or its directory entry. The digest
checks streamed input; it does not protect against another process replacing the
staging name or modifying its inode before commit. This prerequisite is not yet
established for public uploads. Later writers can change the installed file;
no snapshot or exactly-once guarantee is implied.

One version-1 JSON response reports `outcome: completed` with `size_bytes`,
`failed` before replacement, or `unknown` if the parent sync fails after replacement.
Errors contain only a fixed safe code. Require a complete response plus observed
native exit/output close; exit zero alone is insufficient. Input errors preserve
the old destination under that concurrency prerequisite. Temporary-file cleanup
is best effort: permission or I/O errors, as well as forced termination, can leave
a `.parsar-upload-*` staging file. Never interpret it as a completed upload.
A missing receipt remains unknown and must not trigger automatic replay. This
helper does not fence a replacement owner after remote transport or service loss;
public admission still needs operation ownership and recovery handling.

## Connect an executor

An operator creates an executor principal key with
Expand Down
5 changes: 4 additions & 1 deletion packages/codex-executor/src/bin/directory.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
use rustix::fs::FileType;
#[path = "../directory.rs"]
mod directory;
use directory::{anchor, directory, observe};
use directory::observe;
#[path = "../workspace_path.rs"]
mod workspace_path;
use serde_json::json;
use std::{io, path::Path};
use workspace_path::{anchor, directory};

fn run() -> io::Result<serde_json::Value> {
let args: Vec<_> = std::env::args().skip(1).collect();
Expand Down
33 changes: 33 additions & 0 deletions packages/codex-executor/src/bin/write.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#[path = "../workspace_path.rs"]
mod workspace_path;
#[path = "../write_file.rs"]
mod write_file;

use serde_json::json;
use std::io::{self, Write};
use std::path::Path;

fn run() -> Result<u64, write_file::Failure> {
let args: Vec<_> = std::env::args().skip(1).collect();
if args.len() != 3 {
return Err(io::Error::from(io::ErrorKind::InvalidInput).into());
}
let size = args[2]
.parse::<u64>()
.map_err(|_| io::Error::from(io::ErrorKind::InvalidInput))?;
write_file::install(Path::new(&args[0]), &args[1], size, io::stdin().lock())?;
Ok(size)
}

fn main() -> std::process::ExitCode {
let response = match run() {
Ok(size) => json!({"version": 1, "outcome": "completed", "size_bytes": size}),
Err(failure) => {
json!({"version": 1, "outcome": if failure.committed { "unknown" } else { "failed" }, "error": if failure.error.kind() == io::ErrorKind::InvalidInput { "invalid_input" } else { "write_failed" }})
}
};
if writeln!(io::stdout().lock(), "{response}").is_err() {
return std::process::ExitCode::FAILURE;
}
std::process::ExitCode::SUCCESS
}
66 changes: 2 additions & 64 deletions packages/codex-executor/src/directory.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use rustix::fs::{AtFlags, Dir, FileType, Mode, OFlags, open, openat, statat};
use rustix::fs::{AtFlags, Dir, FileType, statat};
use std::io;
use std::os::fd::OwnedFd;
use std::path::{Component, Path};

#[derive(Debug, PartialEq)]
pub(crate) struct Entry {
Expand All @@ -18,68 +17,7 @@ pub(crate) struct Observation {
}

fn invalid() -> io::Error {
io::Error::new(io::ErrorKind::InvalidInput, "invalid directory request")
}

pub(crate) fn anchor(root: &Path) -> io::Result<OwnedFd> {
if !root.is_absolute() || root.as_os_str().len() > 4096 {
return Err(invalid());
}
let mut fd = open(
"/",
OFlags::PATH | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?;
for part in root.components() {
match part {
Component::RootDir => (),
Component::Normal(name) => {
fd = openat(
&fd,
name,
OFlags::PATH | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?;
}
_ => return Err(invalid()),
}
}
Ok(fd)
}

pub(crate) fn directory(root: &OwnedFd, relative: &str) -> io::Result<OwnedFd> {
if relative.len() > 4096 {
return Err(invalid());
}
let mut fd = openat(
root,
".",
OFlags::PATH | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?;
if !relative.is_empty() {
for part in relative.split('/') {
if part.is_empty()
|| part == "."
|| part == ".."
|| part.contains(['\\', '\0', '\r', '\n'])
{
return Err(invalid());
}
fd = openat(
&fd,
part,
OFlags::PATH | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?;
}
}
Ok(openat(
&fd,
".",
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?)
io::ErrorKind::InvalidInput.into()
}

pub(crate) fn observe(fd: &OwnedFd, limit: usize) -> io::Result<Observation> {
Expand Down
1 change: 1 addition & 0 deletions packages/codex-executor/src/directory_tests.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use super::*;
use crate::workspace_path::{anchor, directory};
use std::fs;
use std::os::unix::fs::symlink;
use std::path::PathBuf;
Expand Down
69 changes: 69 additions & 0 deletions packages/codex-executor/src/workspace_path.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
use rustix::fs::{Mode, OFlags, open, openat};
use std::io;
use std::os::fd::OwnedFd;
use std::path::{Component, Path};

fn invalid() -> io::Error {
io::Error::new(io::ErrorKind::InvalidInput, "invalid directory request")
}

pub(crate) fn anchor(root: &Path) -> io::Result<OwnedFd> {
if !root.is_absolute() || root.as_os_str().len() > 4096 {
return Err(invalid());
}
let mut fd = open(
"/",
OFlags::PATH | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?;
for part in root.components() {
match part {
Component::RootDir => (),
Component::Normal(name) => {
fd = openat(
&fd,
name,
OFlags::PATH | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?;
}
_ => return Err(invalid()),
}
}
Ok(fd)
}

pub(crate) fn directory(root: &OwnedFd, relative: &str) -> io::Result<OwnedFd> {
if relative.len() > 4096 {
return Err(invalid());
}
let mut fd = openat(
root,
".",
OFlags::PATH | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?;
if !relative.is_empty() {
for part in relative.split('/') {
if part.is_empty()
|| part == "."
|| part == ".."
|| part.contains(['\\', '\0', '\r', '\n'])
{
return Err(invalid());
}
fd = openat(
&fd,
part,
OFlags::PATH | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
Mode::empty(),
)?;
}
}
Ok(openat(
&fd,
".",
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::CLOEXEC,
Mode::empty(),
)?)
}
Loading