-
Type:
Question
-
Status: Closed
-
Priority:
Major - P3
-
Resolution: Works as Designed
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: Logging
-
Labels:None
I am using mongo-connector (Latest repo- https://github.com/10gen-labs/mongo-connector.git) to index data in ElasticSearch (1.1.1).
I am indexing ~50000 new documents daily.
Mongo-connector fails unexpectedly with logs-
1. ERROR - OplogManager: Last entry no longer in oplog cannot recover! .
This usually happens when the indexing becomes slow and the delta between oplog and elasticsearch becomes ~> 5000 documents.
2. WARNING - DELETE /####index_name/string/53d870d76e2f9017db0b6bda?refresh=true [status:404 request:0.004s]
INFO - > None
ReplicaSet info-
SECONDARY> rs.printReplicationInfo()
configured oplog size: 3751.087890625MB
log length start to end: 39595secs (11hrs)
oplog first event time: Fri Aug 01 2014 01:24:12 GMT+0530 (IST)
oplog last event time: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)
now: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)
I am starting mongo-connector with below parameters-
mongo-connector -m localhost:27017 -t es-ip:9200 --continue-on-error --auto-commit-interval=0 -d doc_managers/elastic_doc_manager.py -n index_foo -i name,age.....,ph --batch-size=1000 --continue-on-error -o foo-config.txt -w /log/foo.log &