How frequently does the bug occur?

Always

Description

In our app we’ve seen and been able to consistently reproduce some data loss that happens during a flexible sync schema update, and are hoping to gain a little insight into what is causing the issue and determine what we should be doing differently to fix the issue.

We’re working on a minimal reproduction, but in the meantime, here is the general setup:

1. Our app is running Realm JS
2. The relevant portion of the schema creates a document tree, essentially:

schema = {
  name: 'Node',
  primaryKey: '_id',
  properties: {
    _id: 'uuid',
    text: 'string',
    children: 'Node[]'
  }
}

1. Users in the client app can create new Nodes. Creating a Node both creates a new document and adds that document to another Node’s children property.

The issue we’re seeing comes up when we deploy a schema update of our production app that adds a field to the Node schema. Specifically we have seen issues adding embedded object and array fields. Here is the sequence of events:

1. An authenticated user has the app open
2. The new schema is deployed, adding a field to the Node object
3. The user creates new Nodes in app
4. Upon restarting app, new Nodes created after schema change are no longer visible in client

A couple important things we have noted as we’ve been debugging this issue:

• We’ve noted (via realm.objects().addListener()) a large quantity of collection changes in the client app that begin a few minutes after the schema is deployed (they appear to be swapping out data locally). They happen in batches of a dozen or so, and can take a few minutes to complete, depending on number of documents the user has.
• Records created by the client during the collection changes do not seem to appear in the database until after the collection changes complete.
• Newly created records are written to the database, but the updates to the children field of the parent in which they were created are not. This leaves them orphaned and not visible to the client.

My best guess as to what is happening is that when the client determines that its own schema does not match the server schema, it begins to replace the local data with data from the server. Any local changes to existing data, such as adding a new object to the children array of an existing Node, are overwritten with the data that exists on the server.

Is this what is happening? And if so, what steps can we take to ensure that updates to documents during a schema change are persisted? If not, does anyone have thoughts on why we’re seeing this type of data loss?

Stacktrace & log output

No response

Can you reproduce the bug?

Always. See this repository for a complete reproduction.

Reproduction Steps

No response

Version

11.3.0

What services are you using?

Atlas Device Sync

Are you using encryption?

No

Platform OS and version(s)

MacOS 12.6.7

Build environment

No response

Cocoapods version

No response