-
Type: Bug
-
Resolution: Won't Fix
-
Priority: Unknown
-
None
-
Affects Version/s: None
-
Component/s: None
-
-
- Describe the bug
-
[Mongodb](https://www.npmjs.com/package/mongodb) has dependencies that are not listed as peer dependencies. I tried to use `packageExtensions` to get around this but I am now getting dozens more errors from two libraries [Snappy](https://www.npmjs.com/package/snappy) and [@mongodb-js/zstd](https://www.npmjs.com/package/@mongodb-js/zstd).
```
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 89:28-70
➤ YN0000: Module not found: Error: Can't resolve './zstd.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/backend/lambda/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
```
```
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:622-662
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm64' in '/Users/sigex/workdir/backend/lambda/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
```
It looks like it's trying to require all these different binaries for different machines.
My `packageExtensions` is below and it's managed to get most of the warnings to shut up.
```yaml
packageExtensions:
"mongodb@*":
dependencies:
bson-ext: "*"
kerberos: "*"
"@mongodb-js/zstd": "*"
aws4: "*"
mongodb-core: "*"
bson: "*"
node-gyp: "*"
"mongodb-client-encryption": "*"
peerDependenciesMeta:
"@mongodb-js/zstd":
optional: true
snappy:
optional: true
bson-ext:
optional: true
kerberos:
optional: true
```
-
-
- To reproduce
-
I created a repo here, see branch `mongodb-broken-pnp`.
https://github.com/gruckion/yarn-berry-serverless-workspaces/tree/mongodb-broken-pnp
Run `yarn; yarn package` to reproduce the errors.
```
❯ yarn package
➤ YN0000:
➤ YN0000: Packaging lambdas-hello for stage testing (us-east-1)
➤ YN0000:
➤ YN0000: Warning: in ../../.yarn/_virtual_/mongodb-virtual-904dd890e4/0/cache/mongodb-npm-4.10.0-99903c2c4a-4847fe69b6.zip/node_modules/mongodb/lib/utils.js 993:38-93
➤ YN0000: Critical dependency: the request of a dependency is an expression
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 17:28-65
➤ YN0000: Module not found: Error: Can't resolve './zstd.win32-x64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 19:28-70
➤ YN0000: Module not found: Error: Can't resolve '@mongodb-js/zstd-win32-x64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 35:28-61
➤ YN0000: Module not found: Error: Can't resolve './zstd.darwin-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 37:28-66
➤ YN0000: Module not found: Error: Can't resolve '@mongodb-js/zstd-darwin-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 47:28-63
➤ YN0000: Module not found: Error: Can't resolve './zstd.darwin-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 65:28-64
➤ YN0000: Module not found: Error: Can't resolve './zstd.linux-x64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 67:28-69
➤ YN0000: Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-x64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 77:28-66
➤ YN0000: Module not found: Error: Can't resolve './zstd.linux-arm64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 79:28-71
➤ YN0000: Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-arm64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 89:28-70
➤ YN0000: Module not found: Error: Can't resolve './zstd.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd/index.js 91:28-75
➤ YN0000: Module not found: Error: Can't resolve '@mongodb-js/zstd-linux-arm-gnueabihf' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/@mongodb-js-zstd-npm-1.0.0-1751370982-73e82a30be.zip/node_modules/@mongodb-js/zstd'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:569-607
➤ YN0000: Module not found: Error: Can't resolve './snappy.android-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:622-662
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:811-852
➤ YN0000: Module not found: Error: Can't resolve './snappy.android-arm-eabi.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:867-910
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-android-arm-eabi' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1159-1198
➤ YN0000: Module not found: Error: Can't resolve './snappy.win32-x64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1213-1254
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-win32-x64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1403-1443
➤ YN0000: Module not found: Error: Can't resolve './snappy.win32-ia32-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1458-1500
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-win32-ia32-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1651-1692
➤ YN0000: Module not found: Error: Can't resolve './snappy.win32-arm64-msvc.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1707-1750
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-win32-arm64-msvc' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:1997-2032
➤ YN0000: Module not found: Error: Can't resolve './snappy.darwin-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2047-2084
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-darwin-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2231-2268
➤ YN0000: Module not found: Error: Can't resolve './snappy.darwin-arm64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2626-2662
➤ YN0000: Module not found: Error: Can't resolve './snappy.freebsd-x64.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2677-2715
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-freebsd-x64' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2900-2939
➤ YN0000: Module not found: Error: Can't resolve './snappy.linux-x64-musl.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:2954-2995
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-linux-x64-musl' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3131-3169
➤ YN0000: Module not found: Error: Can't resolve './snappy.linux-x64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3184-3224
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-linux-x64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3389-3430
➤ YN0000: Module not found: Error: Can't resolve './snappy.linux-arm64-musl.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3445-3488
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm64-musl' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3626-3666
➤ YN0000: Module not found: Error: Can't resolve './snappy.linux-arm64-gnu.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3681-3723
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm64-gnu' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3876-3920
➤ YN0000: Module not found: Error: Can't resolve './snappy.linux-arm-gnueabihf.node' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000:
➤ YN0000: in ../../.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy/index.js 1:3935-3981
➤ YN0000: Module not found: Error: Can't resolve '@napi-rs/snappy-linux-arm-gnueabihf' in '/Users/sigex/workdir/serverless-workspaces-pnp/.yarn/cache/snappy-npm-7.1.2-2cf48206bf-7381485fdd.zip/node_modules/snappy'
➤ YN0000: Environment: darwin, node 16.13.2, framework 3.22.0 (local), plugin 6.2.2, SDK 4.3.2
➤ YN0000: Docs: docs.serverless.com
➤ YN0000: Support: forum.serverless.com
➤ YN0000: Bugs: github.com/serverless/serverless/issues
➤ YN0000:
➤ YN0000: Error:
➤ YN0000: Webpack compilation failed:
➤ YN0000:
➤ YN0000: in ../../.yarn/unplugged/@mongodb-js-zstd-darwin-arm64-npm-1.0.0-df74982a20/node_modules/@mongodb-js/zstd-darwin-arm64/zstd.darwin-arm64.node 1:0
➤ YN0000: Module parse failed: Unexpected character '�' (1:0)
➤ YN0000: You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
➤ YN0000: (Source code omitted for this binary file)
➤ YN0000:
➤ YN0000: in ../../.yarn/unplugged/@napi-rs-snappy-darwin-arm64-npm-7.1.2-605592bf69/node_modules/@napi-rs/snappy-darwin-arm64/snappy.darwin-arm64.node 1:0
➤ YN0000: Module parse failed: Unexpected character '�' (1:0)
➤ YN0000: You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
➤ YN0000: (Source code omitted for this binary file)
➤ YN0000:
➤ YN0000: 1 deprecation found: run 'serverless doctor' for more details
➤ YN0000: Done in 17s 69ms
```
-
-
- Environment
-
```shell
System:
OS: macOS 13.0
CPU: (10) arm64 Apple M1 Max
Binaries:
Node: 16.13.2 - /private/var/folders/mv/dtw39jks6ml64s3pd1pnv_640000gn/T/xfs-7dfef409/node
Yarn: 3.2.3 - /private/var/folders/mv/dtw39jks6ml64s3pd1pnv_640000gn/T/xfs-7dfef409/yarn
npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm
```
-
-
- Additional context
-
Node Mongodb github repo has created 3 PRs to "support PnP" (found by searching for "PnP", but have written no documentation on how to use Node MongoDb with Yarn PnP. This issue is not specific to MongoDb and is more about the practice of not declaring peerDependencies and using optionalRequire(). There are many posts on how to resolve these issues with `packageExtensions` but in this case it's unclear as these are MANY files for each and every operating machine / cpu architecture.
https://github.com/mongodb/node-mongodb-native/pulls?q=pnp
I also posted a comment on this PR as mongodb.
https://github.com/mongodb/node-mongodb-native/pull/2606
I am also using `pnp-webpack-plugin`
- related to
-
NODE-3996 Design: Automate Node.js native package releases
- Backlog