When trying to do a complete backup with mongodump, it always fails with an "i/o timeout", but not always on the exact same collection. Here's a typical output :
[...] 2015-05-19T13:45:19.202+0200 done dumping example12_analytics.tmp_d_carrier_country_date_device_error_flow_landing_op1_os_owner_provider_site 2015-05-19T13:45:19.203+0200 writing example12_analytics.d_date_owner to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_owner.bson 2015-05-19T13:45:19.203+0200 done dumping example12_analytics.d_date_landing_owner 2015-05-19T13:45:19.204+0200 writing example12_analytics.d_date_owner_provider to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_owner_provider.bson 2015-05-19T13:45:19.628+0200 writing example12_analytics.d_date_error_owner metadata to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_error_owner.metadata.json 2015-05-19T13:45:20.115+0200 done dumping example12_analytics.d_date_error_owner 2015-05-19T13:45:20.115+0200 writing example12_analytics.client_context_mobile to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/client_context_mobile.bson 2015-05-19T13:45:21.919+0200 [###########.............] example12_analytics.client_context_example 5749116/12446330 (46.2%) 2015-05-19T13:45:21.919+0200 [#######.................] example12_analytics.billing_provider_repository 6231634/19349001 (32.2%) 2015-05-19T13:45:21.919+0200 [######################..] example12_analytics.d_carrier_country_date_device_error_op1_os_owner_provider_regdate_site 6487164/6773612 (95.8%) 2015-05-19T13:45:21.919+0200 [####....................] example12_analytics.d_carrier_country_date_device_flow_landing_op1_os_owner_site 8024276/40391310 (19.9%) 2015-05-19T13:45:21.919+0200 [#############...........] example12_analytics.client_context_abcde 69672/126395 (55.1%) 2015-05-19T13:45:21.919+0200 [#####...................] example12_analytics.d_date_owner 21491/91050 (23.6%) 2015-05-19T13:45:21.919+0200 [#####...................] example12_analytics.d_date_owner_provider 18660/82934 (22.5%) 2015-05-19T13:45:21.919+0200 [........................] example12_analytics.client_context_mobile 101/67141 (0.2%) 2015-05-19T13:45:21.919+0200 2015-05-19T13:45:24.919+0200 [###########.............] example12_analytics.client_context_example 5769348/12446330 (46.4%) 2015-05-19T13:45:24.919+0200 [#######.................] example12_analytics.billing_provider_repository 6243221/19349001 (32.3%) 2015-05-19T13:45:24.919+0200 [#######################.] example12_analytics.d_carrier_country_date_device_error_op1_os_owner_provider_regdate_site 6500964/6773612 (96.0%) 2015-05-19T13:45:24.919+0200 [####....................] example12_analytics.d_carrier_country_date_device_flow_landing_op1_os_owner_site 8039685/40391310 (19.9%) 2015-05-19T13:45:24.919+0200 [#################.......] example12_analytics.client_context_abcde 89666/126395 (70.9%) 2015-05-19T13:45:24.919+0200 [################........] example12_analytics.d_date_owner 64371/91050 (70.7%) 2015-05-19T13:45:24.919+0200 [##############..........] example12_analytics.d_date_owner_provider 48384/82934 (58.3%) 2015-05-19T13:45:24.919+0200 [####....................] example12_analytics.client_context_mobile 12508/67141 (18.6%) 2015-05-19T13:45:24.919+0200 2015-05-19T13:45:26.195+0200 writing example12_analytics.d_date_owner_provider metadata to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_owner_provider.metadata.json 2015-05-19T13:45:27.016+0200 done dumping example12_analytics.d_date_owner_provider 2015-05-19T13:45:27.017+0200 writing example12_analytics.d_date_billing_owner to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_billing_owner.bson 2015-05-19T13:45:27.036+0200 writing example12_analytics.client_context_abcde metadata to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/client_context_abcde.metadata.json 2015-05-19T13:45:27.250+0200 writing example12_analytics.d_date_owner metadata to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_owner.metadata.json 2015-05-19T13:45:27.589+0200 done dumping example12_analytics.client_context_abcde 2015-05-19T13:45:27.589+0200 writing example12_analytics.d_date_owner_product to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/d_date_owner_product.bson 2015-05-19T13:45:27.632+0200 writing example12_analytics.client_context_mobile metadata to /var/lib/mongodb-backup/mongodb_20150519-132021/example12_analytics/client_context_mobile.metadata.json 2015-05-19T13:45:27.919+0200 [###########.............] example12_analytics.client_context_example 5799805/12446330 (46.6%) 2015-05-19T13:45:27.919+0200 [#######.................] example12_analytics.billing_provider_repository 6289711/19349001 (32.5%) 2015-05-19T13:45:27.919+0200 [#######################.] example12_analytics.d_carrier_country_date_device_error_op1_os_owner_provider_regdate_site 6528578/6773612 (96.4%) 2015-05-19T13:45:27.919+0200 [####....................] example12_analytics.d_carrier_country_date_device_flow_landing_op1_os_owner_site 8085866/40391310 (20.0%) 2015-05-19T13:45:27.919+0200 [........................] example12_analytics.d_date_billing_owner 101/28106 (0.4%) 2015-05-19T13:45:27.919+0200 2015-05-19T13:45:30.289+0200 Failed: error running `listIndexes`. Collection: `example12_analytics.d_date_owner` Err: read tcp 127.0.0.1:27017: i/o timeout
It happens after running for a while. The /var/lib/mongodb directory is 21GB in size (wiredTiger+snappy, it used to be about 100GB with mmapv1).
I am using the tools from the mongodb-linux-x86_64-rhel70-3.0.3.tgz binary download, since the tools are now written in Go and no longer included in the main sources. I have quickly tried to use a 2.6.10 mongodump, but it didn't work with :
connected to: 127.0.0.1 2015-05-20T09:45:29.900+0200 all dbs output of listDatabases isn't what we expected, no 'databases' field: { note: "from execCommand", ok: 0.0, errmsg: "not master" }
(I need to run it against a SECONDARY)
The command I have been using is this one, both with and without --oplog :
mongodump -u admin -p xxxxxxxxxxxx --authenticationDatabase=admin --oplog
- is duplicated by
-
TOOLS-755 Mongodump fails with message: read tcp 127.0.0.1:27017: i/o timeout (part 2)
-
- Closed
-