Release Notes - Python Driver - Version 4.0 - HTML format

Spec Change

  • [PYTHON-2820] - Test serialization of BSON with embedded null bytes in strings
  • [PYTHON-2832] - Provide options to limit number of mongos servers used in connecting to sharded clusters
  • [PYTHON-2856] - Tests for client-side errors with snapshot reads use incorrect syntax for test.expectEvents
  • [PYTHON-2883] - Regex decoding error tests in top.json have unexpected, invalid syntax
  • [PYTHON-2971] - PossiblePrimary must not be checked for wire version compatibility of MaxStalenessSeconds

Bug

  • [PYTHON-1272] - Potential deadlock in an exhaust Cursor destructor
  • [PYTHON-1450] - PyMongo should error on username or password with unescaped '%' characters
  • [PYTHON-1915] - Prohibit copying ClientSession objects
  • [PYTHON-1965] - The bson package should not depend on the pymongo package
  • [PYTHON-2234] - When mongocryptd spawn fails, the driver does not indicate what it tried to spawn
  • [PYTHON-2397] - MongoClient(ssl=True, tls=False) fails with an AttributeError
  • [PYTHON-2460] - Client can create more than minPoolSize background connections
  • [PYTHON-2518] - SON class should be compatible with Python 3's OrderedDict API
  • [PYTHON-2628] - Fix Typo using nonexistent encryption algorithm in PyMongo.encryption.create_data_key docs
  • [PYTHON-2634] - Client Connection Regression with PSA + minPoolSize/maxPoolSize
  • [PYTHON-2719] - RawBatchCursor should raise StopIteration on receiving an empty batch instead of returning empty bytes
  • [PYTHON-2743] - Incompatibility with gevent.Timeout
  • [PYTHON-2761] - Pinned connections are sometimes returned to the pool twice
  • [PYTHON-2790] - raw_bson doctest failures
  • [PYTHON-2795] - Improve error messages when parsing hostnames from URIs
  • [PYTHON-2799] - killCursors must use the namespace returned the initial command response when closing cursors
  • [PYTHON-2915] - Session persistence failure under transaction callback with large Bulk ops
  • [PYTHON-2966] - MongoClient.event_listeners does not work as documented
  • [PYTHON-2981] - Stop using MongoClient.address for hashing and equality
  • [PYTHON-2984] - Client does not retry retryable writeConcernError in bulk write on 4.4+
  • [PYTHON-3017] - Potential hang in CSFLE when reading a KMS network response
  • [PYTHON-3020] - Client does not mark server unknown after "not master" error without a code
  • [PYTHON-3021] - Client sends incorrect read preference to standalones

Epic

New Feature

  • [PYTHON-1636] - Support Exhaust Cursor in OP_MSG
  • [PYTHON-2057] - Make 'name' a required argument for DriverInfo class
  • [PYTHON-2389] - Add session support to find_raw_batches and aggregate_raw_batches
  • [PYTHON-2480] - Add MongoClient helper to access the current TopologyDescription
  • [PYTHON-2533] - Accept a filter expression as an alternative to slowMS/sampleRate
  • [PYTHON-2539] - Support AWS authentication with temporary credentials in CSFLE
  • [PYTHON-2671] - Support loadBalanced URI option
  • [PYTHON-2672] - SDAM and connection establishment changes for load balancers
  • [PYTHON-2673] - Connection pinning behavior for load balanced clusters
  • [PYTHON-2674] - SDAM error handling changes for load balanced clusters
  • [PYTHON-2675] - Monitoring API changes for load balanced clusters
  • [PYTHON-2767] - Support snapshot reads on secondaries
  • [PYTHON-2823] - Allow custom service names with srvServiceName URI option

Task

  • [PYTHON-1300] - Stop shipping .egg files
  • [PYTHON-1301] - Remove support for cursor managers
  • [PYTHON-1302] - Remove MongoClient.kill_cursors
  • [PYTHON-1307] - Remove support for SON manipulators
  • [PYTHON-1309] - Remove helpers for getLastError and friends
  • [PYTHON-1312] - Remove user management helpers
  • [PYTHON-1314] - Remove Database.authenticate and Database.logout
  • [PYTHON-1316] - Remove Database.eval and database.SystemJS
  • [PYTHON-1318] - Remove Collection.initialize_unordered_bulk_op and Collection.initialize_ordered_bulk_op
  • [PYTHON-1319] - Remove Collection.ensure_index and Collection.reindex
  • [PYTHON-1320] - Remove write methods deprecated in PyMongo 3.0
  • [PYTHON-1321] - Remove MongoReplicaSetClient
  • [PYTHON-1323] - Remove Collection.group
  • [PYTHON-1326] - Remove the "useCursor" aggregate option
  • [PYTHON-1410] - Remove "safe" legacy messages.
  • [PYTHON-1579] - Allow unencoded subdelimiters in usernames and passwords
  • [PYTHON-1587] - Remove database_names and collection_names
  • [PYTHON-1588] - Remove MongoClient.close_cursor
  • [PYTHON-1592] - Remove Collection.parallel_scan
  • [PYTHON-1690] - Fix error message when insert_many is given a single document instead of a list
  • [PYTHON-1840] - Remove waitQueueMultiple option
  • [PYTHON-1853] - Empty projections should return the entire document not just the _id
  • [PYTHON-1860] - Use OP_MSG not OP_GET_MORE in find_raw_batches and aggregate_raw_batches
  • [PYTHON-1878] - Add mongodb+srv URIs to Atlas Connectivity tests
  • [PYTHON-1908] - Drop Jython
  • [PYTHON-1949] - CodecOptions and JSONOptions should have the same default value for tz_aware
  • [PYTHON-2008] - Test mod_wsgi with modern Pythons
  • [PYTHON-2033] - Unified test format
  • [PYTHON-2038] - Stop exporting CertificateError from pymongo.errors
  • [PYTHON-2046] - Change default JSONMode from LEGACY to RELAXED
  • [PYTHON-2162] - Remove support for ssl* URI options
  • [PYTHON-2164] - Rethink MongoClient read only configuration attributes
  • [PYTHON-2235] - Remove deprecated pymongo.GEOHAYSTACK
  • [PYTHON-2288] - Remove IsMaster Class
  • [PYTHON-2310] - Remove the fsync and unlock MongoClient helpers and the is_locked attribute
  • [PYTHON-2341] - Migrate RHEL 6.2 testing to Amazon1-2018
  • [PYTHON-2355] - Canonical JSON Options should use UNSPECIFIED UuidRepresentation
  • [PYTHON-2363] - Rate limit new connection creations (maxConnecting)
  • [PYTHON-2364] - Fix DeprecationWarning: please use dns.resolver.resolve() instead
  • [PYTHON-2366] - Test OCSP+FLE with Python 3.9 on Windows
  • [PYTHON-2388] - Create PyMongo 4.0 upgrade guide
  • [PYTHON-2393] - Document support for unicode_decode_error_handler argument for MongoClient
  • [PYTHON-2395] - Consider server load during server selection
  • [PYTHON-2430] - Remove pymongo.message module
  • [PYTHON-2431] - Test Failure - MONGODB-AWS Auth macOS 10.14 Python
  • [PYTHON-2436] - Test failure - test.test_legacy_api.TestLegacyBulk.test_large_inserts_ordered
  • [PYTHON-2437] - Rely on venv instead of virtualenv
  • [PYTHON-2441] - Test failure - test_client.TestClient.test_continuous_network_errors
  • [PYTHON-2445] - Test Failure - MONGODB-AWS Auth Ubuntu 18.04
  • [PYTHON-2447] - Test Failure - test_cmap.TestCMAP.test_cmap_pool_checkout_maxConnecting_is_enforced_maxConnecting_is_enforced
  • [PYTHON-2455] - Update release checklist to include step to file docs ticket outlining compatibility changes/lack thereof
  • [PYTHON-2457] - Test that clients wait 500ms between failed isMaster checks
  • [PYTHON-2462] - Implement and test connection pool paused state
  • [PYTHON-2463] - Do not allow a MongoClient to be reused after it is closed
  • [PYTHON-2474] - Test failure - test_monitor_waits_after_server_check_error
  • [PYTHON-2482] - Test Versioned API with a server started with acceptAPIVersion2
  • [PYTHON-2487] - Benchmark Max Connecting
  • [PYTHON-2489] - Test failures related to Unified tests and Versioned API
  • [PYTHON-2501] - Remove iteritems from son.SON
  • [PYTHON-2502] - Remove Python 2.7 from release scripts
  • [PYTHON-2503] - Use time.monotonic always
  • [PYTHON-2505] - Always use match_hostname from the ssl module
  • [PYTHON-2507] - Workaround pip install failure: RuntimeError: Python 3.5 or later is required
  • [PYTHON-2512] - Update Astrolabe's Workload Executor to use the unified test runner
  • [PYTHON-2516] - Coverage failing: Couldn't combine from non-existent path 'coverage/coverage.*'
  • [PYTHON-2525] - Remove Collection.count and Cursor.count
  • [PYTHON-2526] - Test failure - test_server_selection_in_window.TestProse.test_load_balancing
  • [PYTHON-2528] - Remove Database.current_op
  • [PYTHON-2529] - Remove UUIDLegacy class
  • [PYTHON-2530] - Test failure - MONGODB-AWS Auth Windows: prepare_mongodb_aws.sh: line 2: python3: command not found
  • [PYTHON-2532] - Remove the modifiers option for find, find_one, and Cursor
  • [PYTHON-2534] - Test failure - test_pool_paused_error_is_retryable
  • [PYTHON-2536] - Document Versioned API Usage in Drivers (with Code Samples)
  • [PYTHON-2537] - Fix benchmark failures on Python 3
  • [PYTHON-2571] - Remove NotMasterError
  • [PYTHON-2575] - Test failure - PyPy cryptography install fails with: error: Can not find Rust compiler
  • [PYTHON-2578] - KMS requests should use ssl.CERT_REQUIRED instead of None for clarity
  • [PYTHON-2579] - Fix Atlas planned maintenance test runner
  • [PYTHON-2580] - Provide explicit guidance on handling command errors that occur before the handshake completes during operation execution
  • [PYTHON-2583] - Bump minimum pymongocrypt version to 1.1
  • [PYTHON-2584] - Investigate if the default localThresholdMS=15 is the culprit behind various flaky tests
  • [PYTHON-2586] - Changes to support Python 3.10
  • [PYTHON-2590] - RHEL 7.2 (zSeries) does not have python3 installed
  • [PYTHON-2591] - Ubuntu 16.04 (x86-64, ARM64, POWER8) tests are failing
  • [PYTHON-2605] - Improve mongodb+srv:// error message when dnspython is not installed
  • [PYTHON-2615] - Test Failure - test_client.TestClient.test_socket_timeout with eventlet and TLS
  • [PYTHON-2616] - Test failure - test_client.TestClient.test_network_error_message
  • [PYTHON-2630] - Statically initialize Py_buffer to avoid false positives in Coverity
  • [PYTHON-2631] - Add missing error message when raising InvalidBSON in C extensions
  • [PYTHON-2647] - Test failure - test_ssl.TestClientSSL.test_use_openssl_when_available Ubuntu 18.04 (ARM64)
  • [PYTHON-2667] - Test failure - SRV support broken when running with eventlet
  • [PYTHON-2676] - Unified Test Runner changes for Load Balancer Support
  • [PYTHON-2678] - Resync SRV spec tests
  • [PYTHON-2679] - Auto discover replica sets by default
  • [PYTHON-2702] - Removed deprecated names for renamed URI options
  • [PYTHON-2711] - Remove profile command helpers
  • [PYTHON-2720] - Add mockupDB tests for using OP_MSG for raw batches and exhaust cursors
  • [PYTHON-2721] - Test failure - invalid flag in regex options on v5.0.0-alpha0-442-g0520668
  • [PYTHON-2724] - Add FAQ to PyMongo documentation pointing users to PyMongoArrow
  • [PYTHON-2726] - Document behavior quirk of primaryPreferred/secondaryPreferred readPreferences immediately after MongoClient initialization
  • [PYTHON-2729] - test_index_filter fails on 5.0 dues to missing indexName response field
  • [PYTHON-2730] - Test failures due unnecessary getMore and killCursors calls in 5.0
  • [PYTHON-2731] - Run load balancer test suite with PyPy
  • [PYTHON-2734] - Test failure - test_cursor.TestRawBatchCursor.test_read_concern
  • [PYTHON-2744] - Load Balancer spec tests need to run against non-load balanced clusters too
  • [PYTHON-2748] - UUID example code doesn't work
  • [PYTHON-2750] - Connection pools must be created and eventually marked ready for any server if a direct connection is used
  • [PYTHON-2757] - Specify 5.0 cursor behaviour
  • [PYTHON-2762] - Unified tests should automatically get run when testing against a load balancer
  • [PYTHON-2764] - Test failure- test_Unacknowledged_write_using_dollar-prefixed_or_dotted_keys_may_be_silently_rejected_on_pre-5_0_server
  • [PYTHON-2765] - Test Failure - PyPy test_exhaust
  • [PYTHON-2768] - Add SDAM spec tests for load balancers
  • [PYTHON-2769] - Test redaction of replies to security-sensitive commands
  • [PYTHON-2773] - Mockupdb test failures
  • [PYTHON-2775] - Add documentation example for snapshot reads
  • [PYTHON-2779] - Fix topologies field in snapshot-sessions-not-supported-server-error tests
  • [PYTHON-2797] - Update changelog for 3.12 release
  • [PYTHON-2798] - Test failure - Windows FLE tests failing with oauth2.googleapis.com:443: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
  • [PYTHON-2800] - Add load balanced serverless connectivity tests
  • [PYTHON-2802] - Add link to public Time Series documentation
  • [PYTHON-2803] - Modernize Terminology in 4.0
  • [PYTHON-2806] - Test Failure - test_aggregate_raw_transaction on sharded clusters
  • [PYTHON-2808] - Remove Windows certifi workaround added in PYTHON-2798
  • [PYTHON-2816] - Ensure manylinux release task generates manylinux wheels compatible with older pip versions
  • [PYTHON-2817] - Add .readthedocs.yaml config file
  • [PYTHON-2827] - Stable API Strict Migration Example for Docs
  • [PYTHON-2833] - Set minWireVersion to 6 (MongoDB 3.6)
  • [PYTHON-2838] - Test failure - test_collection.TestCollection.test_manual_last_error
  • [PYTHON-2842] - Integration tests for observeSensitiveCommands
  • [PYTHON-2848] - Update documentation to prefer ping command over ismaster
  • [PYTHON-2850] - Remove map_reduce/inline_map_reduce methods
  • [PYTHON-2854] - Bump maxWireVersion in SDAM monitoring tests
  • [PYTHON-2855] - Update KMS TLS tests for mock responder file name changes
  • [PYTHON-2857] - Remove socketkeepalive option and other deprecated options for MongoClient
  • [PYTHON-2862] - Remove outdated versionchanged info from docs
  • [PYTHON-2868] - Test Serverless behind a load balancer to prevent test breakage
  • [PYTHON-2873] - Test failure - virtualenv creation fails with Python 3.10 on Windows
  • [PYTHON-2875] - Raise error when using min/max without hint
  • [PYTHON-2879] - get_ssl_context ValueError: too many values to unpack (expected 7)
  • [PYTHON-2898] - Update max staleness tests to report a 3.6+ wire version
  • [PYTHON-2899] - Remove code for MongoDB < 3.6 support
  • [PYTHON-2902] - Support dnspython 2
  • [PYTHON-2904] - Use Allowlist instead of Whitelist
  • [PYTHON-2905] - Test Failure - EncryptionError: data_key _id must be a UUID
  • [PYTHON-2911] - Bump maxWireVersion to 14 for MongoDB 5.1
  • [PYTHON-2914] - MongoClient should raise an error when given multiple URIs
  • [PYTHON-2918] - Bump minimum pymongocrypt version to 1.2.0
  • [PYTHON-2919] - Remove MongoDB 2.6-3.4 from performance testing
  • [PYTHON-2921] - Fix eventlet patching detection under Python 3.10
  • [PYTHON-2923] - Add Python 3.10 to release tasks
  • [PYTHON-2926] - Unskip test_readConcern_available_with_out_stage on 5.1+
  • [PYTHON-2927] - Test failure - test_continuous_network_errors
  • [PYTHON-2928] - Test failures with mod_wsgi
  • [PYTHON-2935] - Test Failure - test_use_after_close
  • [PYTHON-2937] - Test failure - test_transaction_starts_with_batched_write PyPy
  • [PYTHON-2938] - Fix race condition caused by MongoClient._process_periodic_tasks(client)
  • [PYTHON-2940] - Fix spec tests that require DNSPython
  • [PYTHON-2941] - Add a CMAP test that verifies the background thread hands over connections to threads doing checkout
  • [PYTHON-2948] - Add prose test for SRV polling with a custom service name
  • [PYTHON-2972] - Test Failure - test_srv_max_hosts_kwarg when dnspython is not installed
  • [PYTHON-2973] - Test Failure - MONGODB-AWS Auth InvalidURI: Bad database name
  • [PYTHON-2987] - Test failure - test_errors on PyPy 7.3.7 due to a unicode str repr change
  • [PYTHON-2990] - Github no longer supports "git://" for git clone
  • [PYTHON-2992] - Implement unified test format loop operation
  • [PYTHON-2998] - Remove md5 checksums from gridfs and remove disable_md5
  • [PYTHON-2999] - Remove unused and internal only CRAM-MD5 auth mechanism
  • [PYTHON-3003] - Add tlsOptions for KMS providers
  • [PYTHON-3004] - Implement FLE KMIP tests
  • [PYTHON-3005] - Make maxConnecting configurable
  • [PYTHON-3014] - Update how events are added to entity map to match specification
  • [PYTHON-3015] - SSL Handshake with Mongo atlas fails with Python 3.10
  • [PYTHON-3018] - Add docs for removed methods from Collection
  • [PYTHON-3019] - Doc test failure
  • [PYTHON-3024] - Update estimatedDocumentCount test for Atlas Data Lake

Improvement

  • [PYTHON-808] - Prevent use of Database and Collection in boolean expressions
  • [PYTHON-1337] - Add __slots__ to commonly used bson classes
  • [PYTHON-1359] - RawBSONDocument example
  • [PYTHON-1363] - Remove unused "retrieved" parameter from CommandCursor
  • [PYTHON-1489] - Merge ajdavis/pymongo-mockup-tests into pymongo
  • [PYTHON-1601] - Test enterprise auth without using deprecated "authenticate" and "logout" methods
  • [PYTHON-1880] - Raise a warning when no_cursor_timeout is used with an implicit session
  • [PYTHON-2096] - Validate that mongocryptd is not spawned if bypassAutoEncryption=true
  • [PYTHON-2318] - Testing Data Lake with Drivers
  • [PYTHON-2396] - Deprecate ssl_keyfile and ssl_certfile URI options in favor of tls* equivalents
  • [PYTHON-2434] - Evergreen task to collect release wheels
  • [PYTHON-2449] - Ensure cryptography can be installed across system configurations
  • [PYTHON-2453] - Versioned MongoDB API for Drivers
  • [PYTHON-2466] - Make MongoClient, Database and Collection objects hashable
  • [PYTHON-2472] - Sharing a MongoClient for metadata lookup can lead to deadlock in drivers using automatic encryption
  • [PYTHON-2475] - Implement Atlas Data Lake prose tests
  • [PYTHON-2506] - Test failure because $listLocalSessions is not supported in API version 1
  • [PYTHON-2508] - Clarify PyOpenSSL requirements for loading system CA certs
  • [PYTHON-2543] - Define error handling behavior of writeErrors and writeConcernError on Mongos
  • [PYTHON-2544] - Drivers should not inspect error messages for "not master" or "node is recovering"
  • [PYTHON-2548] - Implement change stream oplog parsing code for delta oplog entries
  • [PYTHON-2570] - Add test to ensure drivers expect operations to succeed when neither expectError nor expectResult are specified
  • [PYTHON-2596] - Connection errors should always include the host in the error message
  • [PYTHON-2599] - Versioned API database aggregation test only works on 4.9+ servers
  • [PYTHON-2600] - Add Versioned API test for abortTransaction
  • [PYTHON-2602] - Add test that ensures the error returned from being evicted from the WaitQueue is retryable
  • [PYTHON-2603] - Support Removal of Several Platform Builds
  • [PYTHON-2604] - Clarify behavior for runOn.minServerVersion for latest servers
  • [PYTHON-2608] - Test that KMS TLS connections verify peer certificates
  • [PYTHON-2624] - Update serverSelectionTimeoutMS of MongoClient to mongocryptd
  • [PYTHON-2629] - Use "hello" command when API Version is declared
  • [PYTHON-2635] - Test that session is unpinned when executing a non-transaction operation
  • [PYTHON-2641] - Replace usages of 'acceptAPIVersion2' with 'acceptApiVersion2'
  • [PYTHON-2677] - Better wait queue timeout errors for load balanced clusters
  • [PYTHON-2680] - Driver Handling of DBRefs
  • [PYTHON-2684] - Require passing Stable API options to getMore and transaction-continuing commands
  • [PYTHON-2697] - Fix race condition in the pool-checkout-no-idle test
  • [PYTHON-2699] - A pool must emit PoolReadyEvent before resuming the background thread
  • [PYTHON-2710] - Stable API Connection Examples for Docs
  • [PYTHON-2717] - Clarify whether maxPoolSize allows 0 option or no
  • [PYTHON-2718] - Add test for security-sensitive command monitoring event redaction
  • [PYTHON-2740] - Bump maxWireVersion for MongoDB 5.0
  • [PYTHON-2741] - Support 'let' option for aggregate command
  • [PYTHON-2766] - Improve documentation on cursor slices
  • [PYTHON-2776] - Disable writes on snapshot sessions
  • [PYTHON-2777] - Raise client error for snapshot sessions on <5.0 servers
  • [PYTHON-2794] - Remove insertedCount from insert result assertions and improve usage of $$unsetOrMatches
  • [PYTHON-2824] - Make GridOut implement full io.IOBase spec
  • [PYTHON-2859] - Add BSON Binary subtype 7
  • [PYTHON-2870] - Support building the docs in non-html formats

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.