[COMPASS-6720] Cleanup gatherFields() and mongodb-schema Created: 13/Apr/23  Updated: 10/Jan/24  Resolved: 22/May/23

Status: Closed
Project: Compass
Component/s: Import/Export, Tech debt
Affects Version/s: None
Fix Version/s: 1.37.0

Type: Task Priority: Major - P3
Reporter: Le Roux Bodenstein Assignee: Rhys Howell
Resolution: Done Votes: 0
Labels: milestone-5
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by COMPASS-6807 Consider replacing getSchema helper i... Needs Triage
Epic Link: COMPASS-5576
Story Points: 3
Documentation Changes: Not Needed
Sprint: Iteration Ankylosaurus, Iteration Brontosaurus

 Description   

We should just export all the types from mongodb-schema and update them. Or alternatively move schemaToPaths() there so we don't need all the types in compass.

finalizeSchema() inside schema analyzer replaces the fields object internally with something of a different type. We should fix that by just returning a different final result object that matches the type we want.



 Comments   
Comment by Githook User [ 30/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: redux-typescript-query-history-store
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 25/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: COMPASS-6740-scroll-to-a-card
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 24/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: COMPASS-6593-add-performance-tab-indicator
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 23/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: 6802-dev
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 23/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: compass-6825-explain-plan-modal-in-agg-builder
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 22/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: COMPASS-6799-OIDC-connection-form
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 22/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: 1.37-releases
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 22/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: chore(import-export, schema, field-store): update mongodb-schema, use schema paths helper COMPASS-6720 (#4401)

This updates our mongodb-schema dependency in Compass.

Updating to the new version involves a few changes:

  • Schema field paths now use string arrays instead of dot separated strings. This will make it easier for us to handle cases like dots in field names (previously we would join and split on dots).
  • Better schema type definitions are now exported from mongodb-schema. This involved some changes in compass-schema and compass-import-export as we were supporting our own types.
  • Removed the getSchemaPaths function from compass-import-export and updated to use the one from the mongodb-schema package,.
  • Removed the unused dimensionality and nestedFields properties and calculations in compass-field-store. (could be separated to another pr if preferred).
  • In compass-field-store renamed the aceFields flattened map used for editor field autocompletion to autocompleteFields to be more generic as we are using this in code-mirror and no longer have ace. (could be separated to another pr if preferred).
  • Updated schema property variable names, has_duplicates, average_length, and total_count, are now camelCase.
    When integrating changes in compass-schema there were updates needed in the telemetry methods, schemaContainsGeoData and calculateSchemaDepth. I ended up refactoring both and hopefully simplifying calculateSchemaDepth a bit, it looks like it might fix a recent error we had. Moved them off the store. Updated tests to be easier to maintain. We could consider moving the calculateSchemaDepth into mongodb-schema, but that's not included in these changes. I'm thinking this should fix COMPASS-6809
    Branch: main
    https://github.com/mongodb-js/compass/commit/1f2d29fb2e9529fa6fa8d80f28567a7216fbf975
Comment by Githook User [ 18/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: Merge branch 'main' into COMPASS-6720-update-mongodb-schema-remove-schema-paths
Branch: COMPASS-6720-update-mongodb-schema-remove-schema-paths
https://github.com/mongodb-js/compass/commit/9e6f5b109c047dd933785e69bc1dc736fe966707

Comment by Githook User [ 15/May/23 ]

Author:

{'name': 'Rhys', 'email': 'Anemy@users.noreply.github.com', 'username': 'Anemy'}

Message: feat: export schema types, remove lodash dependency, speed up analysis COMPASS-6720 (#189)
Branch: main
https://github.com/mongodb-js/mongodb-schema/commit/b2fc41b6e8f26af34ed7324d6d021b18fe1b8770

Generated at Wed Feb 07 22:44:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.