Uploaded image for project: 'Hadoop Integration'
  1. Hadoop Integration
  2. HADOOP-224

Hive table creation return FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: mongo-hadoop-1.4
    • Fix Version/s: debugging-with-submitter
    • Component/s: hive
    • Labels:
      None
    • Environment:
      HDP 2.3, Hive 1.2
    • # Replies:
      10
    • Last comment by Customer:
      true

      Description

      It looks like the MongoDB Hive SerDe is not tested with newer versions of Hive looking at the github page https://github.com/mongodb/mongo-hadoop/wiki/Hive-Usage.

      When attempting to create an external Hive table to MongoDB 2.6.11 or 3.0.5 the following error results the same in both cases:

      create external table auditlogs_mongo (time_stamp timestamp, audit_event string, ip_address string, agent_id string, contract_id string, type string)
      stored by 'com.mongodb.hadoop.hive.MongoStorageHandler'
      tblproperties ('mongo.uri'='mongodb://localhost:27017/test.auditlogs');
      

      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable
      

      When trying with a mapping included:

      create external table auditlogs_mongo (time_stamp timestamp, audit_event string, ip_address string, agent_id string, contract_id string, type string)
      stored by 'com.mongodb.hadoop.hive.MongoStorageHandler'
      with serdeproperties('mongo.columns.mapping'='{"time_stamp":"TIME_STAMP", "audit_event":"AUDIT_EVENT", "ip_address":"IP_ADDRESS", "agent_id":"AGENT_ID", "contract_id":"CONTRACT_ID", "type":"TYPE"}')
      tblproperties ('mongo.uri'='mongodb://localhost:27017/test.auditlogs');
      

      the following error results:

      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON
      

      The returned errors are very non-intuitive as to what the underlying problem could be.

      Does the MongoDB Hive SerDe requires updates for newer versions of Hive to work?

        Attachments

          Activity

            People

            • Assignee:
              luke.lovett Luke Lovett (Inactive)
              Reporter:
              harisekhon Hari Sekhon
              Participants:
              Last commenter:
              Rathi Gnanasekaran
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                2 years, 6 weeks ago
                Date of 1st Reply: