ExportXMLWordPrintableJSON

    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      django-tasks-db is the ORM-backed backend for Django Tasks, the queue that ships with Django 6.0. It doesn't work on MongoDB, and the more serious of the two reasons fails silently.

      A worker claims a task by picking the next ready one and marking it running, holding a row lock across both steps with select_for_update(skip_locked=True) inside exclusive_transaction(). That assumes the database locks rows. MongoDB reports has_select_for_update = False, so Django drops the lock rather than refusing it, and transaction.atomic is a no-op there. Eight workers draining 60 tasks claimed 258 times, with 59 of the 60 claimed more than once and nothing raised. Snowflake fails the same way, 48 claims for 12 tasks, and it has transactions, so the feature gap is in row locking rather than on being non-relational.

      Index creation is the second reason, tracked separately as INTPYTHON-1082.

      A spike resolved claiming by adding an override point to DatabaseBackend in django-tasks-db, with the MongoDB implementation subclassing it from django-mongodb-extensions. It leaves django-tasks-db free of MongoDB-specific code, claims each task exactly once under contention, and leaves SQLite unchanged at 97 passing tests with its original 16 skips. The prototype, the reproductions and a draft upstream issue are in this gist.

      Definition of done

      This ticket is the decision. Implementation follows in separate tickets.

      • Agree whether to propose the claim hook upstream or carry the changes in django-mongodb-extensions.
      • Agree where two backend-agnostic portability fixes belong: return-value normalisation has no downstream hook, and datetime precision keys on supports_microsecond_precision, which Django does not define.
      • Agree the upstream issue text before anything is posted publicly.
      • Open follow-up tickets for whichever path is chosen.

      Pitfalls

      • Nothing goes upstream before the team agrees. The draft names a defect in a third-party project with a single maintainer, who would be accepting a non-relational code path.
      • If the maintainer declines, this work changes shape entirely, from integrating to forking or dropping. Don't commit to a definition of done that assumes agreement.
      • The claim bug is silent, so any test must assert that a task is claimed once, not that an error is raised.
      • Don't branch on has_select_for_update. SQLite reports False too and relies on BEGIN EXCLUSIVE, so keying on that flag routes SQLite down the wrong path.
      • migrate crashes on MongoDB until INTPYTHON-1082 lands, or until django-mongodb-extensions carries the interim index patch from the gist.

            Assignee:
            Rishabh Bisht
            Reporter:
            Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: