[Integrations] Build a connector framework and priority CRM, calendar, helpdesk, commerce, and automation connectors

[rahuliitk]

Parent epic: #81

Priority: P1 — product completion

Problem and current baseline

QuickVoice already supports HTTP tools, provider-specific integrations, secrets, and MCP catalog/custom connections. These mechanisms enable custom work but do not provide a uniform governed connector lifecycle: OAuth, field mapping, sync cursor, webhook ingestion, rate limits, schema drift, health, conflicts, and retries are handled inconsistently or left to each agent builder.

Outcome

An SMB can connect a CRM or calendar without custom code, an enterprise can govern credentials and data flow, and maintainers can add providers through reusable primitives rather than one-off credential forms and bespoke sync loops.

Requirements

• Define a provider-neutral connector SDK/runtime for manifest, auth, capabilities, objects/actions/triggers, schemas, field mappings, pagination/cursors, incremental/full sync, webhooks, rate limits, retries, health, diagnostics, and lifecycle hooks.
• Support OAuth authorization-code/PKCE, service-account/API-key/secret-reference methods where appropriate, token refresh/revoke, required scopes, sandbox/test account, and multi-instance connections.
• Separate read, create/update, delete, communication, financial, and administrative capabilities; apply per-connection/agent/workflow/environment allowlists and side-effect approval policy.
• Provide schema discovery and versioned snapshots; detect added/removed/type-changed fields, pause unsafe mappings, preview impact, and require review before destructive remapping.
• Add a mapping/transformation layer for canonical customer/contact, appointment, ticket, order/payment reference, conversation, campaign outcome, owner, and consent fields, including validation and preview.
• Implement incremental cursors, webhook plus reconciliation sync, deduplication, idempotency, tombstones/deletes, backfill, checkpoint/resume, bounded parallelism, and documented provider-specific ordering/conflict semantics.
• Support conflict strategies per field/object: source-of-truth, last-write with timestamp/provenance, merge, create-only, or manual review. Never silently overwrite ambiguous high-risk data.
• Add connection health, last success/cursor, token/scope status, latency, provider quota, error categories, affected mappings, pause/resume, test, resync/backfill, and disconnect/revoke with data-retention choices.
• Audit connection/auth/scope/mapping changes and each side-effect invocation/sync summary without logging secret values or protected payloads.
• Build priority connectors on the shared framework: HubSpot, Salesforce, Zoho; Google Calendar, Microsoft Outlook/Graph Calendar, Calendly; Zendesk, Intercom, ServiceNow; Shopify and Stripe; Zapier, Make, n8n; Slack and Microsoft Teams.
• Sequence delivery by reusable capability slices: first prove OAuth + contacts/appointments + webhooks + reconciliation with one CRM and one calendar; then expand providers and domains. Provider support claims require tested capability matrices, not logo-only entries.
• Allow custom HTTP/MCP integrations to coexist and clearly explain when a managed connector, MCP server, webhook, or HTTP tool is appropriate.

Interface and data implications

• Define ConnectorDefinition/Version, Connection, CredentialReference, CapabilityGrant, ExternalObjectReference, MappingVersion, SyncJob/Checkpoint, Conflict, WebhookRegistration, Health, and Invocation contracts.
• Expose public API resources and canonical connector events; internal provider adapters implement shared conformance interfaces.
• Store external IDs with organization, provider, connection, object type, mapping version, source timestamps, and provenance; credentials remain encrypted/write-only references.

UX and accessibility

Provide catalog categories/search, capability/scope disclosure, connect/test flow, mapping preview, sample-data validation, health/errors, sync history, conflict queue, and reconnect/revoke controls. Do not imply “verified,” “native,” or official partnership without evidence. Forms/tables/dialogs must be keyboard and screen-reader accessible; mappings/errors cannot rely on color.

Security, privacy, provider, and cost boundaries

Use least-privilege scopes, encrypted secret references, tenant isolation, SSRF/egress controls, webhook verification, redaction, consent/DNC enforcement, retention/deletion propagation, and audit logs. Provider terms, availability, residency, quotas, and paid plans are external dependencies. Stripe support must avoid storing card data and must not imply PCI scope reduction without review.

Failure modes and backward compatibility

Handle OAuth denial/expiry/revocation, missing scopes, provider 429/5xx, pagination bugs, webhook gaps/duplicates, schema drift, cursor loss, partial batch, deleted records, ambiguous identity, conflict, disconnect during sync, and provider API deprecation. Existing HTTP tools and MCP connections remain available; migrations to managed connectors are opt-in with preview and rollback.

Dependencies

• Stable API/scoped auth in [API] Stabilize public API v1 with scoped keys, idempotency, pagination, rate limits, and OpenAPI contracts #108 and events/webhooks in [Events] Add signed webhooks, retries, replay, dead-letter handling, and streaming subscriptions #110.
• MCP side-effect controls in [MCP] Harden MCP with OAuth, scopes, approvals, retries, auditability, and side-effect controls #115 where shared.
• Customer/contact/consent model and workflow tool contracts.
• Secret, audit, job, and enterprise environment/egress policy.

Out of scope

• Shipping all listed connectors in one release.
• Claiming full object coverage when only specific capabilities are supported.
• Arbitrary bidirectional synchronization without explicit mapping and conflict policy.

Acceptance criteria

• A documented connector manifest/adapter conformance suite covers auth, discovery, mappings, cursors, webhook/reconciliation, retries, health, and revoke.
• One CRM and one calendar connector prove the full reusable slice, including OAuth, refresh, incremental sync, webhook gap recovery, field mapping, and write idempotency.
• Each shipped provider has a tested capability/scope/limit matrix and no logo-only unsupported actions.
• Schema drift, lost cursor, partial sync, duplicate webhook, provider rate limit, and token revocation produce recoverable visible states.
• High-risk writes obey capability grants/approval policy, and retries cannot duplicate supported side effects.
• Disconnect/revoke stops future access and offers documented handling for previously synchronized data.
• Existing HTTP/MCP integrations continue working and are clearly differentiated from managed connectors.

Tests

• Provider-adapter conformance and contract fixtures for auth, schemas, pagination, cursor, webhooks, mappings, health, and errors.
• OAuth state/PKCE, refresh race, expiry/revoke, missing-scope, secret non-disclosure, and tenant-isolation tests.
• Incremental/full sync, checkpoint/resume, deduplication, tombstone, webhook gap/duplicate/order, and reconciliation tests.
• Idempotent create/update, conflict strategies, schema drift, mapping invalidation, partial batch, rate limit, and provider outage tests.
• Consent/DNC, retention/deletion, audit, egress/SSRF, payload redaction, and high-risk approval tests.
• Keyboard, screen-reader, focus, mapping preview/error, non-color health, and reconnect-flow checks.