Mongo Connector Failing Unexpectedly

XMLWordPrintableJSON

    • Type: Question
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Logging
    • None
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      I am using mongo-connector (Latest repo- https://github.com/10gen-labs/mongo-connector.git) to index data in ElasticSearch (1.1.1).
      I am indexing ~50000 new documents daily.

      Mongo-connector fails unexpectedly with logs-
      1. ERROR - OplogManager: Last entry no longer in oplog cannot recover! .
      This usually happens when the indexing becomes slow and the delta between oplog and elasticsearch becomes ~> 5000 documents.

      2. WARNING - DELETE /####index_name/string/53d870d76e2f9017db0b6bda?refresh=true [status:404 request:0.004s]
      INFO - > None

      ReplicaSet info-
      SECONDARY> rs.printReplicationInfo()
      configured oplog size: 3751.087890625MB
      log length start to end: 39595secs (11hrs)
      oplog first event time: Fri Aug 01 2014 01:24:12 GMT+0530 (IST)
      oplog last event time: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)
      now: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)

      I am starting mongo-connector with below parameters-
      mongo-connector -m localhost:27017 -t es-ip:9200 --continue-on-error --auto-commit-interval=0 -d doc_managers/elastic_doc_manager.py -n index_foo -i name,age.....,ph --batch-size=1000 --continue-on-error -o foo-config.txt -w /log/foo.log &

            Assignee:
            Unassigned
            Reporter:
            karan agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: