Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Critical - P2
-
Resolution: Duplicate
-
Affects Version/s: 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Case:
Description
on 3.5.x driver with unifiedTopology performs poorly with many simultaneous requests.
Execution time degrades rapidly with number of requests and it severely affects our back-end servers introducing unexpected spikes of response time.
Cause of issue is new Connection_Pool::withConnection - when Server::query (and other request methods) use it to send commands to server, it occupies connection until response from server received. It basically turns mongo's async protocol into synchronous one.
Legacy topology has no such behavior and Pool::write properly sends queries as soon as they are ready and then matches response from server by requestId - so connections never occupied.
here is simple script to reproduce issue
|
Show all |
'use strict'; |
|
|
const MongoClient = require('mongodb').MongoClient; |
|
|
const URL = '<MONGO URL>'; // topology does not matter |
const MONGO_OPTS = {poolSize: 5, useUnifiedTopology: true}; |
const NUM_QUERIES = 512;
|
|
|
(async function() { |
const connection = await MongoClient.connect(URL, MONGO_OPTS);
|
const collection = connection.db('ec').collection('groups'); |
|
|
const promises = [];
|
|
|
let queriesLeft = NUM_QUERIES;
|
while (queriesLeft-- > 0) promises.push(makeQuery(collection)); |
|
|
const start = Date.now();
|
const results = await Promise.all(promises);
|
results.sort((a, b) => a - b);
|
|
|
const sumTime = results.reduce((r, a) => r + a, 0);
|
const averageTime = Math.round(sumTime / results.length);
|
const medianTime = results[Math.round(results.length / 2)];
|
const minTime = results[0];
|
const maxTime = results[results.length -1];
|
|
|
console.log(`Queries: MIN: ${minTime}ms, MAX: ${maxTime}ms, AVG: ${averageTime}ms, MEDIAN: ${medianTime}ms, SUM: ${sumTime}ms`);
|
console.log(`Script: ${Date.now() - start}ms`);
|
|
|
connection.close();
|
}());
|
|
|
async function makeQuery(collection) { |
const start = Date.now();
|
await collection.findOne({_id: `a${randomInt(1, 10000)}`});
|
return Date.now() - start; |
}
|
|
|
function randomInt(min, max) { |
return ((Math.random() * (max - min)) | 0) + min; |
}
|
And here is some results:
UNIFIED 64
|
Queries: MIN: 121ms, MAX: 524ms, AVG: 315ms, MEDIAN: 317ms, SUM: 20148ms
|
Script: 526ms
|
|
|
UNIFIED 256
|
Queries: MIN: 151ms, MAX: 1851ms, AVG: 998ms, MEDIAN: 1000ms, SUM: 255467ms
|
Script: 1853ms
|
|
|
UNIFIED 1024
|
Queries: MIN: 292ms, MAX: 7006ms, AVG: 3653ms, MEDIAN: 3660ms, SUM: 3740181ms
|
Script: 7015ms
|
|
|
|
|
LEGACY 64
|
Queries: MIN: 65ms, MAX: 347ms, AVG: 320ms, MEDIAN: 343ms, SUM: 20507ms
|
Script: 345ms
|
|
|
LEGACY 256
|
Queries: MIN: 117ms, MAX: 423ms, AVG: 395ms, MEDIAN: 402ms, SUM: 101085ms
|
Script: 387ms
|
|
|
LEGACY 1024
|
Queries: MIN: 336ms, MAX: 963ms, AVG: 798ms, MEDIAN: 802ms, SUM: 817160ms
|
Script: 705ms
|
On 1024 requests unified topology is 10 times slower.
3.4.x driver perform properly and have pretty same results for unifiedTopology and legacy one.
