Add source logic to validate timestamps monotonically increasing

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • Sprint 67
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      When in processing time mode:

      • Add debug assert in SourceOperator(s) to validate timestamps are monotonically increasing
      • After a checkpoint restore, if due to clock skew the new process Date_t::now() sees timestamps behind the last watermark, sleep until the wallclock time has made it to the watermark
      • Add our own wrapper around curTimeMillis64() function that guarantees monotonically increasing timestamps. This wrapper can look something like this:
      uint64_t curTimeMillis64_monotonic() 
      {
         static uint64_t val = 0;      <== TODO make this robust for multiple SPs in the same process
         uint64_t newVal = curTimeMillis64(); 
         if (newVal > val) { 
           val = newVal;
         } 
        return val; 
      }
      

            Assignee:
            Jada Lilleboe
            Reporter:
            Matthew Normyle
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: