[SERVER-35105] Applying renameCollection not supported in initial sync Created: 20/May/18  Updated: 28/Jun/18  Resolved: 21/May/18

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.6.3, 3.6.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Cezary Bartosiak Assignee: Ramon Fernandez Marina
Resolution: Duplicate Votes: 0
Labels: initialSync, renameCollection
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-4941 collection rename may not replicate /... Closed
Related
related to SERVER-35140 Add a startupWarning if running in th... Closed
Operating System: ALL
Participants:

 Description   

We have 5 instances in our replica set:

1. ARBITER, Mongo 3.6.4, Ubuntu 16.04
2. PRIMARY, Mongo 3.6.4, Ubuntu 16.04
3. SECONDARY, Mongo 3.6.4, Ubuntu 16.04
4. SECONDARY, Mongo 3.6.3, Debian 8.1
5. NEW MEMBER, Mongo 3.6.4, Ubuntu 16.04

The 4th member was a primary in the past and its data were cloned using a snapshot (we are in a cloud) to create members 2 and 3. What's important it had Mongo 3.4, but we had upgraded it before proceeding with replication. Now we are trying to attach another member, but we want it to be synchronized via an initial sync because we want it to have a different data structure (separate folders for databases). Unfortunately the initial sync was running for more than 40 hours and restarted because of the following error (colored by me):

2018-05-20T18:11:27.888+0000 I STORAGE [repl writer worker 10] createCollection: <anonymized>.tmp.agg_out.253 with no UUID.
2018-05-20T18:11:27.903+0000 E REPL [repl writer worker 4] Error applying operation: OplogOperationUnsupported: Applying renameCollection not supported in initial sync: { ts: Timestamp(1526688414, 116), t: 10, h: 8574474825826084483, v: 2, op: "c", ns: "<anonymized>.$cmd", wall: new Date(1526688414589), o: { renameCollection: "<anonymized>.tmp.agg_out.253", to: "<anonymized>.mime_types", stayTemp: false, dropTarget: true } } ({ ts: Timestamp(1526688414, 116), t: 10, h: 8574474825826084483, v: 2, op: "c", ns: "<anonymized>.$cmd", wall: new Date(1526688414589), o: { renameCollection: "<anonymized>.tmp.agg_out.253", to: "<anonymized>.mime_types", stayTemp: false, dropTarget: true } })
2018-05-20T18:11:27.904+0000 E REPL [replication-515] Failed to apply batch due to 'OplogOperationUnsupported: error applying batch :: caused by :: Applying renameCollection not supported in initial sync: { ts: Timestamp(1526688414, 116), t: 10, h: 8574474825826084483, v: 2, op: "c", ns: "<anonymized>.$cmd", wall: new Date(1526688414589), o: { renameCollection: "<anonymized>.tmp.agg_out.253", to: "<anonymized>.mime_types", stayTemp: false, dropTarget: true } }'

As far as I understand this kind of error was a bug in earlier versions of Mongo and should not happen in our case. However I can see createCollection [...] with no UUID which looks strange according to what I've read so far about this issue.

Unfortunately we cannot stop aggregations, so we will surely have more these $out commands which makes me not very optimistic in terms of next attempts...



 Comments   
Comment by Ramon Fernandez Marina [ 22/May/18 ]

cbartosiak, we've filed SERVER-35140 to investigate adding the startup warning you suggest – feel free to watch that ticket for updates.

Regards,
Ramón.

Comment by Cezary Bartosiak [ 21/May/18 ]

Thank you for confirming my assumption. The current attempt is at 50 % now and I can see messages like: createCollection: [...] with provided UUID [...], so it looks like it should work now. Maybe it is a good idea to add a warning at startup (like for XFS), isn't it? It is an important setting and it's so easy to forget about it.

Comment by Ramon Fernandez Marina [ 21/May/18 ]

cbartosiak, you're indeed hitting SERVER-4941, which was fixed in 3.6 – but for it to work, your sync source has to produce collections with UUIDs, which means the sync source has to be fully upgraded to 3.6 as well.

As you've found out, if your other nodes had featureCompatibilityVersion set to 3.4 the upgrade to 3.6 is not complete yet. Fully upgrading to 3.6 should take care of this issue, so I'm going to close this ticket as a duplicate of SERVER-4941.

Thanks,
Ramón.

Comment by Cezary Bartosiak [ 20/May/18 ]

My coloring has gone, but it's clear what's going on. I also pasted a link to another, closed issue (gone as well): https://jira.mongodb.org/browse/SERVER-4941

EDIT: I checked featureCompatibilityVersion setting and... it shows 3.4! It looks like a mistake done during an upgrade of the old instance. I will update this comment after another attempt is finished to let you know if changing it to 3.6 helps.

Generated at Thu Feb 08 04:38:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.