[BUG] npm install deletes an existing symlinked node_modules directory

[GrumpyLittleTed]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

If a package contains a node_modules that is symlinked then an npm install will issue the message
npm WARN reify Removing non-directory ....
and delete the node_modules directory.

Expected Behavior

A symlinked node_modules directory should be used as is and not recreated - many people symlink node_modules to a local filesystem so that it does not get synced to cloud storage (like Dropbox).

I suspect that the npm code is just checking if any existing file object named node_modules is a directory and does not also check to see if the non-directory file object is actually a symlinked directory.

Steps To Reproduce

cd /tmp
mkdir symlinked
mkdir z
cd z
npm init -y
ln -s /tmp/symlinked node_modules .
npm install lodash

Environment

• OS: Ubuntu 20.04.1
• Node: 16.7
• npm: 7.21.0

[vicary]

This behavior is introduced between npm 7.15 and 7.21.

The new npm install behavior is to always replace a symlinked node_modules with an actual directory, which breaks the usage of node_modules.nosync and a few existing npm modules that favors this behavior, e.g. nosync-icloud

If this breaking change is intended, it should be introduced with a major version upgrade, shouldn’t it?

---

Redirected to here from a long trip,

1. https://npm.community/ is archived and directs you to GitHub community
2. https://github.community/ doesn't have a hashtag/category for NPM
3. When through GitHub's support email and got redirected to https://github.com/npm/feedback/discussions
4. From npm should not forcibly replaces symlinked node_modules feedback#571 I got here.

Commenting in the hope for Google to index this and aid future lost souls to land here.

[dejanceltra]

This still works in 7.20.0, so it had to be introduced in later versions.

This also breaks when developing using Docker, where code is mounted from the host machine, but node_modules is symlinked to a different location, to improve performance.

[vicary]

Confirming a new lower bound of 7.20.6, the lower bound in my previous comment comes from a local cached version of the binary which is incorrect.

[dejanceltra]

Seems like this was added with 7.21.0 with upgrade of npm/arborist: npm/arborist@f2b0cee

Related: https://github.com/npm/arborist/issues/328

[vicary]

From the commit message it reads,

This is performed at every level, so that the symlink cannot be hidden
within a workspace, or created via some other creative means.

Sounds like an intentional incompatibility with yarn existing tools in the ecosystem.

If --force is set, then the safety measure is disabled.

Although skippable with --force, it piggybacks a lot of useful checks under the same flag.

In the spirit of semantic versioning it should be at least a major upgrade so the community can be prepared, shouldn't it?

[dejanceltra]

It works with --force, but this flags also does other things, which might not be desirable:

The -f or --force argument will force npm to fetch remote resources even if a local copy exists on disk.

[isaacs]

This is by design, and prevents a class of RCE security exploits. GHSA-gmw6-94gg-2rc2

[MartinMystikJonas]

Could there option to keep node_modules as symlink? It basically broke our CI becuse we used node_modules symlinked to shared folder to optimize build - now build that took 2-3 minutes takes about an hour and CI fails on timeout. Reason is it again and again needs to install/copy node_modules in each job.

[blaumeise20]

Please add an option to disable this! I used node_modules.nosync with iCloud, and the folder has a size of 300 MB. This uses all of my iCloud storage.

Using --force doesn't work either, so I can't really use my project until this is fixed.

[htorohn]

Has somebody find a solution or a workaround about this issue? I'm having the same issue and had to downgrade my node version on the server, but it is not the ideal solution.

[vicary]

@htorohn This is not about node, this is an npm design flaw. You may either downgrade, or switch away, from npm.