Fix broken .vsix packaging: ship native deps + platform-specific builds - #4
Merged
Merged
Conversation
β¦uilds The extension is esbuild-bundled with native modules (@lancedb/lancedb, apache-arrow) marked `external` because they can't be inlined. But .vscodeignore blanket-excluded node_modules, so a packaged .vsix contained ZERO dependency files β the vector-store binary was missing and the extension broke on install. It only ever worked when run from source (node_modules on disk). Proven via `vsce ls`: 18 files, 0 from node_modules. Fix: - Reclassify dependencies by the bundling rule: a package belongs in `dependencies` iff it's in the esbuild `--external` list (must ship); everything esbuild bundles moves to `devDependencies`. Verified against the built bundle (externals appear as runtime require()s, bundled deps don't). Net: dependencies = @lancedb/lancedb, apache-arrow, @anthropic-ai/sdk, @google/generative-ai; diff/minimatch/openai/sql.js/ulid/zod/react/react-dom -> devDependencies. - .vscodeignore: stop blanket-ignoring node_modules (vsce already ships only `dependencies`); trim dependency cruft (*.ts/*.map/*.md, test/docs dirs); drop dev-only files (.vscode, scripts, .github). Verified: the native lancedb binary is now inside the produced .vsix. - Platform-specific packaging: each native binary is ~74 MB, so a single all-platforms .vsix would be ~370 MB. Added scripts/package-target.mjs (`npm run package:target`) and a Release workflow that builds one .vsix per OS/arch (linux x64/arm64, darwin x64/arm64, win32 x64), cross-fetching the right native package on the runner. - CI guard: fail the build if the @lancedb native binary ever falls out of the package again. - Remove stale `--external:vectordb` (package no longer used). - Add @vscode/vsce as a devDependency; docs in CONTRIBUTING + CHANGELOG. Verified locally: produced code-historian-linux-x64-1.0.0.vsix (28 MB) containing the native binary and dist/. Typecheck, lint (0 errors), 46 tests, and build all pass; `npm ci` lockfile in sync with all 5 platform packages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
The extension is esbuild-bundled, with native modules (
@lancedb/lancedb,apache-arrow) markedexternalbecause they can't be inlined into the JS bundle. But.vscodeignoreblanket-excludednode_modules, so a packaged.vsixshipped zero dependency files β the vector-store native binary was simply missing, and the extension would fail on install. It only ever worked when run from source (wherenode_modulesis on disk).Proven before the fix with
vsce ls:This predates the embedding work β it affects the core vector store today.
The fix
1. Reclassify dependencies by the bundling rule. A package belongs in
dependencies(and must ship) iff it's in the esbuild--externallist; everything esbuild bundles intodist/moves todevDependencies. Verified against the actual built bundle β externals appear as runtimerequire()s, bundled deps don't:2.
.vscodeignorestops blanket-ignoringnode_modules(vsce already includes onlydependencies), trims dependency cruft (*.ts/*.map/*.md, test/docs dirs), and drops dev-only files (.vscode,scripts,.github).3. Platform-specific packaging. Each native binary is ~74 MB, so one universal
.vsixwould be ~370 MB. Addedscripts/package-target.mjs(npm run package:target) and a Release workflow that builds one.vsixper target (linux x64/arm64, darwin x64/arm64, win32 x64), cross-fetching the correct native package on each runner.4. CI guard. The build now fails if the
@lancedbnative binary ever falls out of the package again.5. Cleanup. Removed a stale
--external:vectordbflag; added@vscode/vsceas a devDependency; documented the packaging model inCONTRIBUTING.md.Verified end-to-end (locally)
vsce lsafter the fix: ships@lancedb/lancedb+ platform binary +apache-arrow+ SDK closure; excludes react/diff/openai.code-historian-linux-x64-1.0.0.vsix(28 MB) and confirmedlancedb.linux-x64-gnu.nodeanddist/extension.jsare inside it.npm cilockfile in sync, with all 5 platform packages present for cross-builds.Note
This is independent of PR #3 (neural embeddings) and fixes the more urgent, already-merged
@lancedb/lancedbcase. Because the fix is general (ship thedependenciesclosure), when PR #3 merges its@xenova/transformers/onnxruntime-nodeoptional deps are handled automatically β though onnxruntime ships all-platform binaries in one package, so a follow-up.vscodeignoretrim per target would further shrink those builds.https://claude.ai/code/session_014bNJaULcYHnDkqUP6HcemQ
Generated by Claude Code