[COMPASS-3150] Update to Electron 3/Node 10.2.1 Created: 27/Sep/18  Updated: 18/Jan/19  Resolved: 12/Dec/18

Status: Closed
Project: Compass
Component/s: Build
Affects Version/s: None
Fix Version/s: 1.18.0

Type: Task Priority: Major - P3
Reporter: Durran Jordan Assignee: Lucas Hrabovsky (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Epic Link: COMPASS-3214
Story Points: 8
Sprint: Iteration Naan, Iteration Octopus, Iteration Paneer, Iteration Qbert, Iteration Ratatouille, Iteration Sriracha, Iteration Takaioko

 Description   

Currently blocked by: https://github.com/atom/node-keytar/issues/125



 Comments   
Comment by Githook User [ 19/Nov/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3113: Rewrite evergreen integration (#1569)

  • Fix bad yaml
  • checkpoint
  • checkpoint
  • checkpoint
  • checkpoint
  • checkpoint
  • now have 5 boxes
  • Fix compass_env path
  • fix tar filename
  • update notes
  • :tada: Hadron-build can now be cleaved from the rest of the build process
  • Fixing windows, document how to extend tests in the future
  • Cleaning up warnings
  • Fix bash error
  • Checkpoint waiting to verify Ubuntu e2e
  • Install xvfb because they're not on Linux distros by default
  • Fixing generated script paths
  • Debugging xvfb
  • Checking in bash scripts instead of inlining
  • Pre dinner checkpoint
  • fix path
  • Fix distribution expansion not being applied
  • MacOS cannot currently be paralelized

But, this will still be faster than before as we're only `npm install`-ing 1 time instead of 4.

  • Need to stop xvfb after tests run or job will just keep going
  • Now that ubuntu working, on to RHEL
  • Debugging RHEL
  • Checkpoint
  • Try `set -o allexport` to figure out why PATH mods getting dropped
  • set -o allexport everything
  • Use xvfb maybe for all tests
  • Need to source for osx signing
  • Missed a set -o allexport
  • Debugging macOS
  • Debugging
  • debugging env
  • Checkpoint

almost all working...

  • Locking in macOS as complete.

Func tests, builds, and package/publish all working. Now faster than master which is just doing builds/package/publish and not tests.

  • Fix include compass_env.sh in build bundle
  • Missed an explicit set shell: bash
  • Add more notes and cleanup evergreen config
  • COMPASS-3115: Fix windows functional tests

Upgrade to `hadron-spectron@5`. See mongodb-js/hadron-spectron#3 for more details.

  • COMPASS-3116: Trying to figure out how to make `xfvb` run on RHEL
  • Refresh package-lock
  • Update hadron-spectron to beta.2 to debug travis
  • COMPASS-3116: Add `set -e` everywhere to avoid silent failures
  • COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2

  • COMPASS-3116: Debugging
  • Debugging
  • debugging
  • try upgrading kerberos
  • electron-rebuild messing up kerberos?
  • pin electron
  • disable build cache?
  • COMPASS-3116: repro'd travis kerberos symbol error

Should fix

```
/home/travis/build/10gen/compass/node_modules/electron/dist/electron: symbol lookup error: /home/travis/build/10gen/compass/node_modules/kerberos/build/Release/kerberos.node: undefined symbol: _ZN2v816FunctionTemplate3NewEPNS_7IsolateEPFvRKNS_20FunctionCallbackInfoINS_5ValueEEEENS_5LocalIS4_EENSA_INS_9SignatureEEEiNS_19ConstructorBehaviorENS_14SideEffectTypeE
Failed with exit code: 1
Output:
npm ERR! Test failed. See above for more details.
```

  • turn on debug logging
  • add cross-env
  • Enable debug for ubuntu evg posttest debugging
  • rollback plugins upgrading react and enzyme
  • upload artifacts to s3 when npm fails
  • fix path
  • These deps come with respective plugins.
  • dont run test hook scripts to debug
  • refresh package-lock
  • quick script to test a list of important plugins
  • TODO for testing native module plugins in release build
  • evg test plugin script
  • COMPASS-3116: mongodb-runner cleanup fixes posttest hang
  • feat(ci): Travis tests against whatever current stable server version is
  • COMPASS-3116: reactivate test hooks w/ updated mdb runner
  • Trying kerberos workaround
  • Update runner to latest
  • Is it a cache problem?
  • Regenerate package-lock

Updates mongodb-runner to 4.7.0 which includes the actual fix in https://github.com/mongodb-js/mongodb-tools/commit/7559449ba68a0bf266eac1a50e13a9363ce2227c

Related to NODE-1749

  • Func tests should now work on travis
  • Turn down debug chatiness now that issues resolved
  • COMPASS-3150: Upgrade hadron-spectron for Linux functional tests
  • chore(ci): Switch travis back to non-debugging config
  • debugging
  • debugging evergreen npm problem again...
  • COMPASS-3116: RHEL and Ubuntu now have libsecret-dev by default

See BUILD-4243

  • chore: Update runner to get better error messages in ci
  • COMPASS-3116: Finally capture npm failure logs
  • COMPASS-3116: Confirmed xvfb on nix boxes

The `xvfb-maybe` command in `package.json` will handle finding the `xvfb-run` executable and environment variables.

See https://github.com/paulcbetts/xvfb-maybe

  • Fix attaching npm debug logs
  • chore: Refresh package-lock
  • Rolling back to fix weird npm issues

https://github.com/10gen/compass/commit/1dbbf7bbe52f1d697adf44260afffc34d7d88c54

  • debugging travis
  • remove evg debugging
  • Revert "debugging travis"

This reverts commit e30a0bc537f118c592e952de523372587b9591a2.

  • Revert "Rolling back to fix weird npm issues"

This reverts commit 182e756a1982073b0952d521bdefa22469cfa79f.

  • make project npmrc globally readable?
  • COMPASS-3116: Fix nix npmrc not overriding registry
  • COMPASS-3116: Only electron-rebuild keytar and kerberos

[hadron-compile-cache](https://github.com/mongodb-js/hadron-compile-cache) :arrow_right: depends on chokidar :arrow_right: depends on fsevents...

http://npm.im/fsevents is supposed to be macOS only but there is something super wacky going on with that project... and because `electron-rebuild` doesn't unstead that `fsevents` is a transitive optional dependency of `hadron-compile-cache` (a real production dependency), `electron-rebuild` throws a panic and dies because `fsevents` (macOS only) cannot be rebuilt on Windows (which is not macOS). :dizzy_face:

```
An unhandled error occurred inside electron-rebuild
Traceback (most recent call last):
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\gyp_main.py", line 16, in <module>
sys.exit(gyp.script_main())
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 545, in script_main
return main(sys.argv[1:])
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 538, in main
return gyp_main(args)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 523, in gyp_main
generator.GenerateOutput(flat_list, targets, data, params)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 2019, in GenerateOutput
sln_projects, project_objects, flat=msvs_version.FlatSolution())
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1778, in _GatherSolutionFolders
return _DictsToFolders('', root, flat)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1731, in _DictsToFolders
for folder, contents in bucket.iteritems():
AttributeError: 'MSVSProject' object has no attribute 'iteritems'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\lib\configure.js:345:16)
gyp ERR! stack at ChildProcess.emit (events.js:182:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
gyp ERR! System Windows_NT 10.0.14393
gyp ERR! command "C:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\.deps
node.exe" "Z:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\node_modules\\node-gyp\\bin
node-gyp.js" "rebuild" "-target=3.0.6" "arch=x64" "dist-url=https://atom.io/download/electron" "build-from-source" "module_name=fse" "module_path=Z:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\node_modules\\fsevents\\lib\\binding\\Release
electron-v3.0-win32-x64" "
remote_path=./v1.2.4/" "package_name=fse-v1.2.4-electron-v3.0-win32-x64.tar.gz" "-host=https://fsevents-binaries.s3-us-west-2.amazonaws.com"
gyp ERR! cwd Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\fsevents
gyp ERR! node -v v10.2.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Failed with exit code: 1
```

An unhandled error occurred inside electron-rebuild
Traceback (most recent call last):
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\gyp_main.py", line 16, in <module>
sys.exit(gyp.script_main())
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 545, in script_main
return main(sys.argv[1:])
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 538, in main
return gyp_main(args)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 523, in gyp_main
generator.GenerateOutput(flat_list, targets, data, params)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 2019, in GenerateOutput
sln_projects, project_objects, flat=msvs_version.FlatSolution())
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1778, in _GatherSolutionFolders
return _DictsToFolders('', root, flat)
File "Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1731, in _DictsToFolders
for folder, contents in bucket.iteritems():
AttributeError: 'MSVSProject' object has no attribute 'iteritems'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\node-gyp\lib\configure.js:345:16)
gyp ERR! stack at ChildProcess.emit (events.js:182:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
gyp ERR! System Windows_NT 10.0.14393
gyp ERR! command "C:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\.deps
node.exe" "Z:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\node_modules\\node-gyp\\bin
node-gyp.js" "rebuild" "-target=3.0.6" "arch=x64" "dist-url=https://atom.io/download/electron" "build-from-source" "module_name=fse" "module_path=Z:\\data\\mci\\a4c1d1b6e5075de7c6e444eac5bab731\\src\\node_modules\\fsevents\\lib\\binding\\Release
electron-v3.0-win32-x64" "
remote_path=./v1.2.4/" "package_name=fse-v1.2.4-electron-v3.0-win32-x64.tar.gz" "-host=https://fsevents-binaries.s3-us-west-2.amazonaws.com"
gyp ERR! cwd Z:\data\mci\a4c1d1b6e5075de7c6e444eac5bab731\src\node_modules\fsevents
gyp ERR! node -v v10.2.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Failed with exit code: 1

  • debug rhel packaging
  • fix build bundle variant race
  • Why are package and test not setting correct env vars?

Evergreens CWD is mutable so we need to regenerate ~/compass_env.sh before running vs just restoring from the tarball

  • Fix running signtool.exe hack
  • debugging
  • All platforms back to being able to sign and package and upload releases
  • Add hadron-spectron plugin test

to try and figure out why func tests higher up are failing.

quick notes: xvfb-maybe makes linux tests now pass w/o hanging

  • debugging xvfb and cygwin
  • Upgrade keytar to 4.3.0

They've switched mainline to `libsecret` (was gnome-keyring specific before). Adding `libsecret` support is why I forked originally so that fork can just go away now.

Mainline `keytar` now also supports prebuilds for `electron@3`

cc @durran

  • Revert "TODO for testing native module plugins in release build"

This reverts commit 7ccd5167c67ad41d0925920c543001712fa5376c.

  • functional tests may not work but unit tests should
  • Adding debugging for uncaught exceptions
Comment by Githook User [ 07/Nov/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3150: Upgrade hadron-spectron for Linux functional tests
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/ee6b62dce9cbd674e12ad96e05d049ffb340de6b

Comment by Githook User [ 06/Nov/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/fe0e00041166586bb448319c725d7ef7a000a965

Comment by Githook User [ 05/Nov/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/82af4d53ceba897b0cd4ef1cdcd9893695cff181

Comment by Githook User [ 01/Nov/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/f97e677781bb81f5900bab96d1f1c4621a6c8bb3

Comment by Githook User [ 31/Oct/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/6b042270d794c32e0fa3a972d5aed80864e2c476

Comment by Githook User [ 31/Oct/18 ]

Author:

{'name': 'Lucas Hrabovsky', 'email': 'hrabovsky.lucas@gmail.com', 'username': 'imlucas'}

Message: COMPASS-3116: Update node versions

COMPASS-3150 published `hadron-build@20` which upgrades electron to 3.0

electron@3.0 maps to node@10.2
Branch: COMPASS-3113_refactor-evergreen
https://github.com/10gen/compass/commit/be57576f086aa74d1c913ee0344e25d982989632

Generated at Wed Feb 07 22:32:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.