Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-14760

Mongo Connector Failing Unexpectedly

    XMLWordPrintable

    Details

    • Type: Question
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Works as Designed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None

      Description

      I am using mongo-connector (Latest repo- https://github.com/10gen-labs/mongo-connector.git) to index data in ElasticSearch (1.1.1).
      I am indexing ~50000 new documents daily.

      Mongo-connector fails unexpectedly with logs-
      1. ERROR - OplogManager: Last entry no longer in oplog cannot recover! .
      This usually happens when the indexing becomes slow and the delta between oplog and elasticsearch becomes ~> 5000 documents.

      2. WARNING - DELETE /####index_name/string/53d870d76e2f9017db0b6bda?refresh=true [status:404 request:0.004s]
      INFO - > None

      ReplicaSet info-
      SECONDARY> rs.printReplicationInfo()
      configured oplog size: 3751.087890625MB
      log length start to end: 39595secs (11hrs)
      oplog first event time: Fri Aug 01 2014 01:24:12 GMT+0530 (IST)
      oplog last event time: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)
      now: Fri Aug 01 2014 12:24:07 GMT+0530 (IST)

      I am starting mongo-connector with below parameters-
      mongo-connector -m localhost:27017 -t es-ip:9200 --continue-on-error --auto-commit-interval=0 -d doc_managers/elastic_doc_manager.py -n index_foo -i name,age.....,ph --batch-size=1000 --continue-on-error -o foo-config.txt -w /log/foo.log &

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: