[SERVER-6170] Mongoimport unable to load Date before epoch Created: 22/Jun/12  Updated: 15/Feb/13  Resolved: 27/Nov/12

Status: Closed
Project: Core Server
Component/s: Tools
Affects Version/s: 2.0.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Niels Basjes Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: insert
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

The test system runs CentOS 6.2 64Bit with mongo-10gen-2.0.6-mongodb_1.x86_64 and mongo-10gen-server-2.0.6-mongodb_1.x86_64 from the 10gen RPM repository.


Issue Links:
Depends
Duplicate
duplicates TOOLS-56 mongoimport fails when record contain... Closed
Related
related to TOOLS-56 mongoimport fails when record contain... Closed
Operating System: ALL
Participants:

 Description   

I have some data which we transformed into json so it can be imported into mongodb.

For some documents we need to have a date field that lies before 1970.

Here I document this issue simply with 1 millisecond before and after epoch.

We tested the documents by essentially doing this:

[nbasjes@localhost ~]$ echo 'db.datetest.save({"date": new Date(1)});' | mongo nielsbasjes
MongoDB shell version: 2.0.6
connecting to: nielsbasjes
bye

[nbasjes@localhost ~]$ echo 'db.datetest.save({"date": new Date(-1)});' | mongo nielsbasjes
MongoDB shell version: 2.0.6
connecting to: nielsbasjes
bye

and two documents have been inserted.

However when we try to load the same documents using mongoimport we get this:

[nbasjes@localhost ~]$ echo '{"date": new Date(1)}' | mongoimport -vv -d nielsbasjes -c datetest
Thu Jun 21 17:43:19 creating new connection to:127.0.0.1
Thu Jun 21 17:43:19 BackgroundJob starting: ConnectBG
Thu Jun 21 17:43:19 connected connection!
connected to: 127.0.0.1
Thu Jun 21 17:43:19 ns: nielsbasjes.datetest
Thu Jun 21 17:43:19 filesize: 0
Thu Jun 21 17:43:19 got line:{"date": new Date(1)}
Thu Jun 21 17:43:19 got line:
imported 1 objects

So far so good ...

[nbasjes@localhost ~]$ echo '{"date": new Date(-1)}' | mongoimport -vv -d nielsbasjes -c datetest
Thu Jun 21 17:43:24 creating new connection to:127.0.0.1
Thu Jun 21 17:43:24 BackgroundJob starting: ConnectBG
Thu Jun 21 17:43:24 connected connection!
connected to: 127.0.0.1
Thu Jun 21 17:43:24 ns: nielsbasjes.datetest
Thu Jun 21 17:43:24 filesize: 0
Thu Jun 21 17:43:24 got line:{"date": new Date(-1)}
Thu Jun 21 17:43:24 Assertion: 10340:Failure parsing JSON string near: "date": ne
0x581a82 0x528610 0xaa3ff3 0xaa5e63 0xa9cccd 0xaa0932 0x3bbb81ecdd 0x4fed69
 mongoimport(_ZN5mongo11msgassertedEiPKc+0x112) [0x581a82]
 mongoimport(_ZN5mongo8fromjsonEPKcPi+0x3d0) [0x528610]
 mongoimport(_ZN6Import8parseRowEPSiRN5mongo7BSONObjERi+0x8b3) [0xaa3ff3]
 mongoimport(_ZN6Import3runEv+0x16e3) [0xaa5e63]
 mongoimport(_ZN5mongo4Tool4mainEiPPc+0x169d) [0xa9cccd]
 mongoimport(main+0x32) [0xaa0932]
 /lib64/libc.so.6(__libc_start_main+0xfd) [0x3bbb81ecdd]
 mongoimport(__gxx_personality_v0+0x3c9) [0x4fed69]
exception:Failure parsing JSON string near: "date": ne
 
Thu Jun 21 17:43:24 got line:
imported 0 objects
encountered 1 error

We have tried several syntaxes for the same date and we get the same effect with all these forms:

  • {"date": new Date(-2174520376)}
  • {"date": new Date(1901,2,3,22,33,44,000)}
  • {"date": new Date("February 3, 1901 22:33:44")


 Comments   
Comment by Jeremy Mikola [ 22/Oct/12 ]

Niels: just to be clear, pre-epoch dates are currently not supported by mongoimport. Per SERVER-961, the functionality is planned but it has yet to be scheduled for a particular future release.

Comment by Jeremy Mikola [ 22/Oct/12 ]

IMO, this should receive the same planning as SERVER-961. Both tickets refer to a particular aspect of pre-epoch date imports (class function in this ticket and reserved words in object notation for the other).

Comment by Jeremy Mikola [ 22/Oct/12 ]

In 2.2, the assertion messages are a bit different (it alludes to BSON being too large):

$ echo '{"date": new Date(-1)}' | mongoimport -d test -c dates -vv
Mon Oct 22 12:12:51 creating new connection to:127.0.0.1:27017
Mon Oct 22 12:12:51 BackgroundJob starting: ConnectBG
Mon Oct 22 12:12:51 connected connection!
connected to: 127.0.0.1
Mon Oct 22 12:12:51 ns: test.dates
Mon Oct 22 12:12:51 filesize: 0
Mon Oct 22 12:12:51 got line:{"date": new Date(-1)}
Mon Oct 22 12:12:51 Assertion: 10340:Failure parsing JSON string near: "date": ne
0xaccb91 0x6796fb 0x67987c 0xa7840d 0x56121d 0x562d04 0xb3ea62 0x5542cc 0x7f8e46f0c76d 0x554159 
 mongoimport(_ZN5mongo15printStackTraceERSo+0x21) [0xaccb91]
 mongoimport(_ZN5mongo11msgassertedEiPKc+0x9b) [0x6796fb]
 mongoimport() [0x67987c]
 mongoimport(_ZN5mongo8fromjsonEPKcPi+0x56d) [0xa7840d]
 mongoimport(_ZN6Import8parseRowEPSiRN5mongo7BSONObjERi+0xa2d) [0x56121d]
 mongoimport(_ZN6Import3runEv+0x1314) [0x562d04]
 mongoimport(_ZN5mongo4Tool4mainEiPPc+0x1712) [0xb3ea62]
 mongoimport(main+0x2c) [0x5542cc]
 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f8e46f0c76d]
 mongoimport(__gxx_personality_v0+0x421) [0x554159]
Mon Oct 22 12:12:51 User Assertion: 13504:BSON representation of supplied JSON is too large: Failure parsing JSON string near: "date": ne
Mon Oct 22 12:12:51 exception:BSON representation of supplied JSON is too large: Failure parsing JSON string near: "date": ne
Mon Oct 22 12:12:51 
Mon Oct 22 12:12:51 imported 0 objects
Mon Oct 22 12:12:51 ERROR: encountered 1 error

Generated at Thu Feb 08 03:10:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.