mongoClient.connect hangs sometimes and don't call the callback

We are using MongoDB version 3.6.8 with replica set. And using mongodb node package version 3.1.10 to connect.

Our code to connect is,

let client = new MongoClient(getUrl(), getOptions());

//getUrl() format is 'mongodb://<>'

client.connect(function (err) {

    // This callback is not executed

    if (err) {

        log('Error in connecting to MongoDb', err);

        return callback(err);

    }

    log('Successfully Connected...');

    callback();

});

We observed from our logs that the callback is not called sometimes. Is this a known issue? I could find some similar issues reported for different versions but they are closed now.
https://jira.mongodb.org/browse/NODE-1039
https://jira.mongodb.org/browse/NODE-1395
https://jira.mongodb.org/browse/NODE-1402

We are not able to reproduce this issue always. The issue happens for the initial connection after fail-over. Wanted to check if this has to be handled at our application level?(like trying to connect again if the callback is not called for long duration).