-
Type: Bug
-
Resolution: Duplicate
-
Priority: Critical - P2
-
None
-
Affects Version/s: 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4
-
Component/s: None
-
Labels:
-
(copied to CRM)
-
Empty show more show less
on 3.5.x driver with unifiedTopology performs poorly with many simultaneous requests.
Execution time degrades rapidly with number of requests and it severely affects our back-end servers introducing unexpected spikes of response time.
Cause of issue is new Connection_Pool::withConnection - when Server::query (and other request methods) use it to send commands to server, it occupies connection until response from server received. It basically turns mongo's async protocol into synchronous one.
Legacy topology has no such behavior and Pool::write properly sends queries as soon as they are ready and then matches response from server by requestId - so connections never occupied.
here is simple script to reproduce issue
'use strict'; const MongoClient = require('mongodb').MongoClient; const URL = '<MONGO URL>'; // topology does not matter const MONGO_OPTS = {poolSize: 5, useUnifiedTopology: true}; const NUM_QUERIES = 512; (async function() { const connection = await MongoClient.connect(URL, MONGO_OPTS); const collection = connection.db('ec').collection('groups'); const promises = []; let queriesLeft = NUM_QUERIES; while (queriesLeft-- > 0) promises.push(makeQuery(collection)); const start = Date.now(); const results = await Promise.all(promises); results.sort((a, b) => a - b); const sumTime = results.reduce((r, a) => r + a, 0); const averageTime = Math.round(sumTime / results.length); const medianTime = results[Math.round(results.length / 2)]; const minTime = results[0]; const maxTime = results[results.length -1]; console.log(`Queries: MIN: ${minTime}ms, MAX: ${maxTime}ms, AVG: ${averageTime}ms, MEDIAN: ${medianTime}ms, SUM: ${sumTime}ms`); console.log(`Script: ${Date.now() - start}ms`); connection.close(); }()); async function makeQuery(collection) { const start = Date.now(); await collection.findOne({_id: `a${randomInt(1, 10000)}`}); return Date.now() - start; } function randomInt(min, max) { return ((Math.random() * (max - min)) | 0) + min; }
And here is some results:
UNIFIED 64 Queries: MIN: 121ms, MAX: 524ms, AVG: 315ms, MEDIAN: 317ms, SUM: 20148ms Script: 526ms UNIFIED 256 Queries: MIN: 151ms, MAX: 1851ms, AVG: 998ms, MEDIAN: 1000ms, SUM: 255467ms Script: 1853ms UNIFIED 1024 Queries: MIN: 292ms, MAX: 7006ms, AVG: 3653ms, MEDIAN: 3660ms, SUM: 3740181ms Script: 7015ms LEGACY 64 Queries: MIN: 65ms, MAX: 347ms, AVG: 320ms, MEDIAN: 343ms, SUM: 20507ms Script: 345ms LEGACY 256 Queries: MIN: 117ms, MAX: 423ms, AVG: 395ms, MEDIAN: 402ms, SUM: 101085ms Script: 387ms LEGACY 1024 Queries: MIN: 336ms, MAX: 963ms, AVG: 798ms, MEDIAN: 802ms, SUM: 817160ms Script: 705ms
On 1024 requests unified topology is 10 times slower.
3.4.x driver perform properly and have pretty same results for unifiedTopology and legacy one.