Feat/databox v2 support

.claude/rules/api-client.md

@@ -0,0 +1,33 @@
+---
+paths:
+  - src/lib/api-client.ts
+  - src/base-command.ts
+---
+
+# API Client Contract
+
+## BaseCommand
+
+`BaseCommand<T>` provides:
+- `apiClient` getter — lazy-creates `ApiClient` from config or flag overrides. Errors with exit 1 if no key.
+- `requireNumericId(value, name)` — validates string is all digits, errors with exit 2.
+- `accountHeaders` getter — returns `{'x-account-id': id}` if `--account-id` set, else `{}`.
+- Base flags: `--json`, `--api-key` (hidden), `--api-url` (hidden), `--account-id` (hidden).
+
+`auth login` extends `Command` directly (not `BaseCommand`) because it works without an existing API key.
+
+## ApiClient
+
+- Wraps native `fetch()` — no external HTTP dependencies.
+- Methods: `get<T>`, `post<T>`, `patch<T>`, `put<T>`, `delete<T>` — all generic.
+- Unwraps V2 envelope: returns `response.data`, not the full `{data, requestId, status}`.
+- Auth: `x-api-key` header on every request. Content-Type set only when body is present.
+- Errors: parses `errors[]` from response, throws `ApiRequestError(message, statusCode, errors)`.
+- Network errors: rethrown as `'Could not connect to API. Check your internet connection.'`
+
+## Rules
+
+- Never add external HTTP dependencies (axios, got, node-fetch) — use native `fetch()`.
+- Never expose `ApiRequestError` internals (statusCode, raw error array) to the user beyond the message.
+- Never bypass `ApiClient` for API calls unless the protocol requires it (SSE streaming).
+- The `apiKey` property must not appear in any `this.log()`, `console.log()`, or error output.

.claude/rules/commands.md

@@ -0,0 +1,77 @@
+---
+paths:
+  - src/commands/**
+  - src/lib/output.ts
+---
+
+# Command Conventions
+
+## Structure
+
+Every command extends `BaseCommand<T>` (exception: `auth login` extends `Command` directly).
+Static members are ordered alphabetically: `args`, `description`, `examples`, `flags`.
+
+```typescript
+// Good
+export default class DatasetGet extends BaseCommand<typeof DatasetGet> {
+  static args = { ... }
+  static description = 'Get details of a specific dataset'
+  static examples = [ ... ]
+  static flags = { ... }
+  async run(): Promise<void> { ... }
+}
+
+// Bad — wrong order, missing examples
+export default class DatasetGet extends BaseCommand<typeof DatasetGet> {
+  static description = '...'
+  static flags = { ... }
+  static args = { ... }
+  async run(): Promise<void> { ... }
+}
+```
+
+## Flags and arguments
+
+- Flag names: kebab-case (`page-size`, `data-source-id`). Body/query params: camelCase (`pageSize`, `dataSourceId`).
+- Use `Flags.string()`, `Flags.integer()`, `Flags.boolean()` — match the data type.
+- `required: true` on mandatory flags, `options: [...]` for enums, `exclusive: [...]` for mutual exclusion.
+- Boolean flags: `default: false`.
+- Examples use `<%= config.bin %>` template, never hardcoded `databox`. At least 2 examples per command.
+- Args use `Args.string({ required: true })` — even numeric IDs are accepted as strings and validated later.
+
+## Output by command type
+
+| Type | Output | Functions |
+|---|---|---|
+| List | Table + pagination | `formatOutput(data, columns, json)` + `showPagination(pagination, json)` |
+| Get / Create / Update | Single record | `formatSingle(data, json)` |
+| Delete / Purge / Clear | Confirmation message | `this.log('Resource ID action.')` |
+| Set (permissions, timezone) | Confirmation message or single record | `this.log()` or `formatSingle()` |
+
+## Destructive operations
+
+Delete, purge, and clear commands require:
+1. `--force` flag with `default: false`
+2. `confirm()` from `../../lib/prompt.js` when not forced
+3. `this.log('Aborted.')` when user declines
+4. Success message: `"Resource ID past-tense."` (e.g., `"Dataset 123 deleted."`)
+
+## Error codes
+
+- `this.error(msg, {exit: 1})` — general errors (missing auth, API failures)
+- `this.error(msg, {exit: 2})` — input validation errors (`requireNumericId`)
+
+## API calls
+
+- Always pass `this.accountHeaders` as the last argument to `apiClient.get/post/patch/put/delete`.
+- No try/catch around API calls — errors propagate to oclif's handler.
+- No direct `fetch()` calls (exception: `ask-genie.ts` for SSE streaming).
+
+## Update commands
+
+Update commands with optional flags must guard against empty bodies:
+```typescript
+if (Object.keys(body).length === 0) {
+  this.error('Provide at least one field to update (--name or --title).', {exit: 1})
+}
+```

.claude/rules/e2e-testing.md

@@ -0,0 +1,105 @@
+# E2E Testing Conventions
+
+End-to-end suites live in `test/e2e/` and spawn the **built** CLI against a **real** API.
+They are separate from the mocked unit suite in `test/commands/`. Full usage:
+`test/e2e/README.md`.
+
+## Boundaries
+
+| | Unit (`test/commands/*.test.ts`) | E2E (`test/e2e/*.e2e.ts`) |
+|---|---|---|
+| Runs | `runCommand` in-process | `node bin/run.js` as a child process |
+| API | `global.fetch` mocked | Real, over the network |
+| Asserts | stdout string contents | exit code + stdout/stderr |
+| Command | `npm test` | `npm run test:e2e` |
+
+The `.e2e.ts` suffix is what keeps them apart — `npm test`'s glob is `test/**/*.test.ts`.
+Never rename an e2e file to `.test.ts`, and never add `test/e2e` to `.mocharc.yml`.
+
+## Rules
+
+- **Everything goes through the CLI.** No suite makes a direct HTTP call. Setup,
+  assertions and teardown all use `cli()`. The e2e layer has no API client of its own,
+  so it cannot drift from the one under test.
+- **One file per command group**, named for the group: `test/e2e/<group>.e2e.ts`.
+- **Name every created resource** with `e2eName(label)` and register it on a
+  `ResourceTracker`; tear it down in the suite's `after()`.
+- **Always pass `--force`** to destructive commands. Child stdin is `'ignore'`, so an
+  interactive `confirm()` would hang until the mocha timeout.
+- **Match error text with `errorText(result)`**, never `result.stderr` directly — the
+  CLI hard-wraps messages, so a phrase can be split across lines with padding.
+- **Never add mocha `--retries` or `--parallel`.** Mocha retries re-run the whole test,
+  creating resources twice; parallel suites collide on shared account state. Retry belongs
+  in `retryRead` (poll a read until the API's cache catches up) and `cliWithRetry` (re-run
+  a command whose failure matches a transient environment fault). `cliWithRetry` is safe
+  for assertions too — a genuine failure does not match a transient pattern, and a matched
+  one is still returned once attempts run out. Shared dev environments do fail in bursts,
+  including spurious 401s on a valid key.
+- **No API key outside `helpers/environments.ts`**, and never a production key.
+- **Never mutate a resource the suite did not create without `withRestore()`.** It
+  records the undo on disk before the change, so an interrupted run can be repaired
+  with `npm run test:e2e:cleanup`. A bare `finally` does not survive Ctrl-C.
+
+## Classifying a failure
+
+A failing e2e test means one of three things. Say which, in the test:
+
+1. **A CLI defect** — **fix the command.** A skipped test is green and CI cannot tell it
+   from a passing one, so parking a known-broken command as a skip makes the suite lie.
+   Only if the fix is genuinely deferred, use `it.skip` with `[BROKEN: <what>]` in the
+   title plus a comment giving the API's real contract, the observed symptom, and the
+   source file that fixes it — and keep that list short.
+2. **An environment outage** — `this.skip()` at runtime via `serviceUnavailable(result)`,
+   which recognises the API's own 5xx/service-down messages. Never hard-code an outage
+   as expected behaviour.
+3. **A capability the account lacks** — `this.skip()` with a logged reason: no agency
+   account, no Advanced Security add-on, no databoards, no connections.
+
+A skip must always print or carry its reason. A silent skip is worse than a failure.
+
+## Verifying a suspected CLI defect
+
+Before marking anything `[BROKEN]`, confirm it against the raw endpoint with `curl`,
+so the report distinguishes a CLI bug from an API one. Two defect families found so far,
+both invisible to the unit suite:
+
+- **Envelope drift** — a command reads `response.items` where the API returns a bare
+  array, or hands `{items: […]}` straight to `formatOutput`. The unit mock defines the
+  shape, so it always agrees with itself. Only a real response settles it.
+- **Request-body drift** — a command sends a field name the API does not accept
+  (`interval` vs `syncInterval`, `status` vs `isVerified`, `tags` vs `synonyms`). These
+  commands could never succeed, and every one of them had a passing unit test.
+
+**Any command that sends a request body needs both**: an e2e test, and a unit test
+asserting the body via `lastBody(method, path)` from `test/helpers.ts`. The unit
+assertion is the cheap guard that runs on every `npm test`; the e2e test is what proves
+the field names are the ones the API actually wants.
+
+```typescript
+it('sends syncInterval, not interval', async () => {
+  await runCommand(['dataset', 'set-sync-frequency', '123', '--interval', '60'], {root: process.cwd()})
+  expect(lastBody('PUT', '/v2/datasets/123/sync-frequency')).to.deep.equal({syncInterval: 60})
+})
+```
+
+Note that `@oclif/test`'s `runCommand` refuses an **empty-string** flag value even though
+the real binary accepts one, so assertions about clearing a nullable field belong in the
+e2e suite.
+
+## The API is the source of truth
+
+`ingestion-api` defines the contract; the CLI follows it. Concretely:
+
+- A command exposes **every field** of its request contract in
+  `IngestionApi.Core/Contracts/Request/V2/`, and every query parameter its controller
+  action declares.
+- `--json` returns **what the endpoint returned**. Do not hand-pick a subset into a new
+  object, and do not reshape values. List commands unwrap `response.items` to a bare
+  array — that is the one established convention — but the item objects pass through whole.
+- Optional string fields are guarded with `!== undefined`, never truthiness, so an empty
+  string can clear a nullable field.
+- Response interfaces mirror the response contract, including inherited members (a
+  `…Detail` type extends its `…ListItem`).
+
+Re-derive the mapping from the API source rather than from memory or from the existing
+CLI code, and confirm anything surprising against a live endpoint with `curl`.

.claude/rules/security.md

@@ -0,0 +1,44 @@
+---
+paths:
+  - src/**
+---
+
+# Security Rules
+
+## API key handling
+
+- `apiClient.apiKey` must never appear in `this.log()`, `console.log()`, or error messages.
+- The only legitimate direct access to `apiKey` is in `ask-genie.ts` for the SSE `x-api-key` header.
+- Hidden flags (`--api-key`, `--api-url`, `--account-id`) must remain `hidden: true` — they do not appear in help output.
+
+## Config file
+
+- Config lives at `~/.config/databox-cli/config.json` with `apiKey` and optional `apiUrl`.
+- Never log config file contents. Never weaken file permissions.
+
+## URL interpolation
+
+- User-provided IDs are interpolated into API paths: `/v2/datasets/${args.datasetId}`.
+- Dataset commands validate with `requireNumericId()` (digits only — safe).
+- Other commands do not validate — a malicious ID could produce unexpected API paths.
+- New commands that interpolate user input into URL paths should validate the input.
+
+```typescript
+// Safe — validated
+this.requireNumericId(args.datasetId, 'Dataset ID')
+const response = await this.apiClient.get(`/v2/datasets/${args.datasetId}`, ...)
+
+// Risk — unvalidated
+const response = await this.apiClient.get(`/v2/connections/${args.connectionId}`, ...)
+```
+
+## Input handling
+
+- `JSON.parse()` on user-provided flag values (--schema, --records, --data) must be wrapped in try/catch with a user-friendly error message. A bare `JSON.parse` leaks a raw `SyntaxError`.
+- `fs.readFileSync` on user-provided paths (--file flag) should check file existence first.
+- No unbounded stdin reads without size limits in new commands.
+
+## Error messages
+
+- Do not include request headers, full URLs, or API keys in error output.
+- `ApiRequestError` messages are displayed to the user — they come from the server and are acceptable.

.claude/rules/testing.md

@@ -0,0 +1,54 @@
+---
+paths:
+  - test/**
+---
+
+# Testing Conventions
+
+## Framework
+
+Mocha + Chai (expect style) + `@oclif/test` (`runCommand`). ESM with ts-node loader.
+
+## Test structure
+
+```typescript
+import {runCommand} from '@oclif/test'
+import {expect} from 'chai'
+import {cleanupTestConfig, mockApi, restoreApi, setupTestConfig} from '../../helpers.js'
+
+describe('domain action', () => {
+  beforeEach(() => {
+    setupTestConfig()
+    mockApi([{
+      method: 'GET',
+      path: '/v2/resources',
+      response: {status: 'success', requestId: 'test', data: { ... }},
+    }])
+  })
+
+  afterEach(() => { restoreApi(); cleanupTestConfig() })
+
+  it('does the thing', async () => {
+    const {stdout} = await runCommand(['domain', 'action'], {root: process.cwd()})
+    expect(stdout).to.include('expected value')
+  })
+
+  it('outputs JSON with --json', async () => {
+    const {stdout} = await runCommand(['domain', 'action', '--json'], {root: process.cwd()})
+    const json = JSON.parse(stdout)
+    expect(json).to.have.property('expectedKey')
+  })
+})
+```
+
+## Rules
+
+- **File location mirrors source**: `src/commands/dataset/get.ts` → `test/commands/dataset/get.test.ts`
+- **Setup/teardown**: `setupTestConfig()` in `beforeEach`, `cleanupTestConfig()` + `restoreApi()` in `afterEach`. Always both.
+- **Mock envelope**: Full `{status: 'success', requestId: 'test', data: {...}}` — not just `{data}`.
+- **Realistic mocks**: Include all fields the command accesses, not empty objects.
+- **runCommand**: Always pass `{root: process.cwd()}`.
+- **Destructive commands**: Use `--force` to skip interactive prompts.
+- **Coverage per command**: At minimum one happy-path test + one `--json` test.
+- **Error paths**: New validation logic (`requireNumericId`, empty-body guard) needs tests asserting the exit code.
+- **Naming**: `describe('domain action')` matches the CLI invocation. `it('verbs behavior')`.