[SERVER-24168] Increase tolerance in SampleFromRandomCursorBasics::MimicNonOptimized Created: 16/May/16  Updated: 20/Nov/16  Resolved: 16/May/16

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 3.2.11, 3.3.8

Type: Task Priority: Major - P3
Reporter: Charlie Swanson Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: test-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Backport Completed:
Sprint: Query 14 (05/13/16)
Participants:
Linked BF Score: 0

 Description   

In build failures, one of these assertions sometimes fails, with values just outside the range deemed acceptable:

    // The average random meta value of the first document should be about 0.75.
    ASSERT_GTE(firstTotal / nTrials, 0.74);
    ASSERT_LTE(firstTotal / nTrials, 0.76);
 
    // The average random meta value of the second document should be about 0.5.
    ASSERT_GTE(secondTotal / nTrials, 0.49);
    ASSERT_LTE(secondTotal / nTrials, 0.51);

After a deep dive into probability theory, Max and I were able to figure out that (assuming a perfect PRNG) the first set of assertions should succeed at least 99.9999% of the time (six 9's). If we double the range, we should get to at least thirteen 9's.

A similar analysis can be applied to the second set of assertions, but it gets messy since it involves taking the convolution of two Beta distributions, which is not fun.



 Comments   
Comment by Charlie Swanson [ 05/Oct/16 ]

Backported this to the 3.2 branch after seeing BF-3704 crop up on the 3.2 branch.

cc ramon.fernandez, david.storch

Comment by Githook User [ 05/Oct/16 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'cswanson310@gmail.com'}

Message: SERVER-24168 Increase tolerance of random sampling tests.

SampleRandomCursorBasics::MimicNonOptimized has failed a couple times
with values that were close to the acceptable cutoffs. This commit
increases the acceptable cutoffs.

(cherry picked from commit 461ee5376cbe09cb5c0c4b1db118a3d5a5973a7e)
Branch: v3.2
https://github.com/mongodb/mongo/commit/a90b989b87efa32206401cc6bef0429db59f2b64

Comment by Githook User [ 16/May/16 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'cswanson310@gmail.com'}

Message: SERVER-24168 Increase tolerance of random sampling tests.

SampleRandomCursorBasics::MimicNonOptimized has failed a couple times
with values that were close to the acceptable cutoffs. This commit
increases the acceptable cutoffs.
Branch: master
https://github.com/mongodb/mongo/commit/461ee5376cbe09cb5c0c4b1db118a3d5a5973a7e

Generated at Thu Feb 08 04:05:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.