Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33395

PPC64 little endian altivec optimizations are broken on newer gcc

    • Fully Compatible
    • ALL
    • v3.6, v3.4
    • Hide

      $
      $ cat variables.list
      VERBOSE=1
      VARIANT_DIR="fedora"
      $ cat build-options
      -j4 --use-system-tcmalloc --wiredtiger=on --nostrip --disable-warnings-as-errors --variables-files=variables.list --opt=off
      $ scons build/fedora/mongo/db/fts/fts_unicode_phrase_matcher_test build/fedora/mongo/db/fts/unicode/byte_vector_test build/fedora/mongo/db/fts/unicode/string_test $(cat build-options)
      $ build/fedora/mongo/db/fts/fts_unicode_phrase_matcher_test; build/fedora/mongo/db/fts/unicode/byte_vector_test; build/fedora/mongo/db/fts/unicode/string_test

      Show
      $ $ cat variables.list VERBOSE=1 VARIANT_DIR="fedora" $ cat build-options -j4 --use-system-tcmalloc --wiredtiger=on --nostrip --disable-warnings-as-errors --variables-files=variables.list --opt=off $ scons build/fedora/mongo/db/fts/fts_unicode_phrase_matcher_test build/fedora/mongo/db/fts/unicode/byte_vector_test build/fedora/mongo/db/fts/unicode/string_test $(cat build-options) $ build/fedora/mongo/db/fts/fts_unicode_phrase_matcher_test; build/fedora/mongo/db/fts/unicode/byte_vector_test; build/fedora/mongo/db/fts/unicode/string_test
    • Platforms 2018-03-26, Platforms 2018-05-07

      Hi,
      I'm building mongodb on ppc64le platform and I see error fails related to ByteVector altivec.

      $ build/fedora/mongo/db/fts/fts_unicode_phrase_matcher_test; build/fedora/mongo/db/fts/unicode/byte_vector_test; build/fedora/mongo/db/fts/unicode/string_test
      2018-02-20T14:16:24.384+0100 I -        [main] going to run suite: FtsUnicodePhraseMatcher
      2018-02-20T14:16:24.384+0100 I -        [main] 	 going to run test: CaseAndDiacriticInsensitive
      2018-02-20T14:16:24.384+0100 E -        [main] Throwing exception: Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:48
      2018-02-20T14:16:24.385+0100 I -        [main] FAIL: CaseAndDiacriticInsensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:48
      2018-02-20T14:16:24.385+0100 I -        [main] 	 going to run test: CaseSensitiveAndDiacriticInsensitive
      2018-02-20T14:16:24.385+0100 E -        [main] Throwing exception: Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:68
      2018-02-20T14:16:24.385+0100 I -        [main] FAIL: CaseSensitiveAndDiacriticInsensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:68
      2018-02-20T14:16:24.385+0100 I -        [main] 	 going to run test: CaseInsensitiveAndDiacriticSensitive
      2018-02-20T14:16:24.385+0100 E -        [main] Throwing exception: Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:88
      2018-02-20T14:16:24.385+0100 I -        [main] FAIL: CaseInsensitiveAndDiacriticSensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:88
      2018-02-20T14:16:24.385+0100 I -        [main] 	 going to run test: CaseAndDiacriticSensitive
      2018-02-20T14:16:24.385+0100 I -        [main] 	 going to run test: CaseAndDiacriticInsensitiveTurkish
      2018-02-20T14:16:24.385+0100 E -        [main] Throwing exception: Expected: phraseMatcher.phraseMatches(find2, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:129
      2018-02-20T14:16:24.385+0100 I -        [main] FAIL: CaseAndDiacriticInsensitiveTurkish	Expected: phraseMatcher.phraseMatches(find2, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:129
      2018-02-20T14:16:24.385+0100 I -        [main] 	 DONE running tests
      2018-02-20T14:16:24.385+0100 I -        [main] **************************************************
      2018-02-20T14:16:24.385+0100 I -        [main] FtsUnicodePhraseMatcher        | tests:    5 | fails:    4 | assert calls:          0 | time secs:  0.000
      	CaseAndDiacriticInsensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:48
      	CaseSensitiveAndDiacriticInsensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:68
      	CaseInsensitiveAndDiacriticSensitive	Expected: phraseMatcher.phraseMatches(find1, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:88
      	CaseAndDiacriticInsensitiveTurkish	Expected: phraseMatcher.phraseMatches(find2, str, options) @src/mongo/db/fts/fts_unicode_phrase_matcher_test.cpp:129
      2018-02-20T14:16:24.385+0100 I -        [main] TOTALS                         | tests:    5 | fails:    4 | assert calls:          0 | time secs:  0.000
      2018-02-20T14:16:24.385+0100 I -        [main] Failing tests:
      2018-02-20T14:16:24.385+0100 I -        [main] 	 FtsUnicodePhraseMatcher/CaseAndDiacriticInsensitive Failed
      2018-02-20T14:16:24.385+0100 I -        [main] 	 FtsUnicodePhraseMatcher/CaseSensitiveAndDiacriticInsensitive Failed
      2018-02-20T14:16:24.385+0100 I -        [main] 	 FtsUnicodePhraseMatcher/CaseInsensitiveAndDiacriticSensitive Failed
      2018-02-20T14:16:24.385+0100 I -        [main] 	 FtsUnicodePhraseMatcher/CaseAndDiacriticInsensitiveTurkish Failed
      2018-02-20T14:16:24.385+0100 I -        [main] FAILURE - 4 tests in 1 suites failed
      2018-02-20T14:16:24.394+0100 I -        [main] going to run suite: ByteVector
      2018-02-20T14:16:24.394+0100 I -        [main] 	 going to run test: LoadStoreUnaligned
      2018-02-20T14:16:24.394+0100 I -        [main] 	 going to run test: Splat
      2018-02-20T14:16:24.394+0100 I -        [main] 	 going to run test: MaskAny
      2018-02-20T14:16:24.394+0100 E -        [main] Throwing exception: Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:70
      2018-02-20T14:16:24.394+0100 I -        [main] FAIL: MaskAny	Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:70
      2018-02-20T14:16:24.394+0100 I -        [main] 	 going to run test: MaskHigh
      2018-02-20T14:16:24.394+0100 E -        [main] Throwing exception: Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:82
      2018-02-20T14:16:24.395+0100 I -        [main] FAIL: MaskHigh	Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:82
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: CompareEQ
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: CompareGT
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: CompareLT
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: BitOr
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: BitOrAssign
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: BitAnd
      2018-02-20T14:16:24.395+0100 I -        [main] 	 going to run test: BitAndAssign
      2018-02-20T14:16:24.395+0100 I -        [main] 	 DONE running tests
      2018-02-20T14:16:24.395+0100 I -        [main] **************************************************
      2018-02-20T14:16:24.395+0100 I -        [main] ByteVector                     | tests:   11 | fails:    2 | assert calls:          0 | time secs:  0.000
      	MaskAny	Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:70
      	MaskHigh	Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:82
      2018-02-20T14:16:24.395+0100 I -        [main] TOTALS                         | tests:   11 | fails:    2 | assert calls:          0 | time secs:  0.000
      2018-02-20T14:16:24.395+0100 I -        [main] Failing tests:
      2018-02-20T14:16:24.395+0100 I -        [main] 	 ByteVector/MaskAny Failed
      2018-02-20T14:16:24.395+0100 I -        [main] 	 ByteVector/MaskHigh Failed
      2018-02-20T14:16:24.395+0100 I -        [main] FAILURE - 2 tests in 1 suites failed
      2018-02-20T14:16:24.403+0100 I -        [main] going to run suite: UnicodeString
      2018-02-20T14:16:24.403+0100 I -        [main] 	 going to run test: SubstrTest
      2018-02-20T14:16:24.403+0100 I -        [main] 	 going to run test: RemoveDiacritics
      2018-02-20T14:16:24.404+0100 E -        [main] Throwing exception: Expected output + filler == String::caseFoldAndStripDiacritics(&buf, input + filler, kCaseSensitive, kNormal) (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx == ^xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) @src/mongo/db/fts/unicode/string_test.cpp:88
      2018-02-20T14:16:24.404+0100 I -        [main] FAIL: RemoveDiacritics	Expected output + filler == String::caseFoldAndStripDiacritics(&buf, input + filler, kCaseSensitive, kNormal) (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx == ^xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) @src/mongo/db/fts/unicode/string_test.cpp:88
      2018-02-20T14:16:24.404+0100 I -        [main] 	 going to run test: CaseFolding
      2018-02-20T14:16:24.405+0100 I -        [main] FAIL: CaseFolding	 std::exception: text contains invalid UTF-8 in test CaseFolding
      2018-02-20T14:16:24.406+0100 I -        [main] 	 going to run test: CaseFoldingTurkish
      2018-02-20T14:16:24.406+0100 E -        [main] Throwing exception: Expected "kac yasındasınız" == String::caseFoldAndStripDiacritics(&buf, test1, kDiacriticSensitive, kTurkish) (kac yasındasınız == kac yasindasiniz) @src/mongo/db/fts/unicode/string_test.cpp:138
      2018-02-20T14:16:24.406+0100 I -        [main] FAIL: CaseFoldingTurkish	Expected "kac yasındasınız" == String::caseFoldAndStripDiacritics(&buf, test1, kDiacriticSensitive, kTurkish) (kac yasındasınız == kac yasindasiniz) @src/mongo/db/fts/unicode/string_test.cpp:138
      2018-02-20T14:16:24.406+0100 I -        [main] 	 going to run test: CaseFoldingAndRemoveDiacritics
      2018-02-20T14:16:24.407+0100 E -        [main] Throwing exception: Expected "ποσο χρονων εισαι?" == String::caseFoldAndStripDiacritics(&buf, test1, 0, kNormal) (ποσο χρονων εισαι? == Πόσο χρονών είσαι?) @src/mongo/db/fts/unicode/string_test.cpp:151
      2018-02-20T14:16:24.407+0100 I -        [main] FAIL: CaseFoldingAndRemoveDiacritics	Expected "ποσο χρονων εισαι?" == String::caseFoldAndStripDiacritics(&buf, test1, 0, kNormal) (ποσο χρονων εισαι? == Πόσο χρονών είσαι?) @src/mongo/db/fts/unicode/string_test.cpp:151
      2018-02-20T14:16:24.407+0100 I -        [main] 	 going to run test: SubstringMatch
      2018-02-20T14:16:24.407+0100 I -        [main] FAIL: SubstringMatch	 std::exception: text contains invalid UTF-8 in test SubstringMatch
      2018-02-20T14:16:24.407+0100 I -        [main] 	 going to run test: SubstringMatchTurkish
      2018-02-20T14:16:24.407+0100 E -        [main] Throwing exception: Expected: String::substrMatch(str, "yasındasınız", String::kNone, CaseFoldMode::kTurkish) @src/mongo/db/fts/unicode/string_test.cpp:182
      2018-02-20T14:16:24.407+0100 I -        [main] FAIL: SubstringMatchTurkish	Expected: String::substrMatch(str, "yasındasınız", String::kNone, CaseFoldMode::kTurkish) @src/mongo/db/fts/unicode/string_test.cpp:182
      2018-02-20T14:16:24.408+0100 I -        [main] 	 going to run test: BadUTF8
      2018-02-20T14:16:24.408+0100 I -        [main] 	 going to run test: UTF32ToUTF8
      2018-02-20T14:16:24.408+0100 I -        [main] 	 DONE running tests
      2018-02-20T14:16:24.408+0100 I -        [main] **************************************************
      2018-02-20T14:16:24.408+0100 I -        [main] UnicodeString                  | tests:    9 | fails:    6 | assert calls:          0 | time secs:  0.005
      	RemoveDiacritics	Expected output + filler == String::caseFoldAndStripDiacritics(&buf, input + filler, kCaseSensitive, kNormal) (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx == ^xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) @src/mongo/db/fts/unicode/string_test.cpp:88
      	CaseFolding	 std::exception: text contains invalid UTF-8 in test CaseFolding
      	CaseFoldingTurkish	Expected "kac yasındasınız" == String::caseFoldAndStripDiacritics(&buf, test1, kDiacriticSensitive, kTurkish) (kac yasındasınız == kac yasindasiniz) @src/mongo/db/fts/unicode/string_test.cpp:138
      	CaseFoldingAndRemoveDiacritics	Expected "ποσο χρονων εισαι?" == String::caseFoldAndStripDiacritics(&buf, test1, 0, kNormal) (ποσο χρονων εισαι? == Πόσο χρονών είσαι?) @src/mongo/db/fts/unicode/string_test.cpp:151
      	SubstringMatch	 std::exception: text contains invalid UTF-8 in test SubstringMatch
      	SubstringMatchTurkish	Expected: String::substrMatch(str, "yasındasınız", String::kNone, CaseFoldMode::kTurkish) @src/mongo/db/fts/unicode/string_test.cpp:182
      2018-02-20T14:16:24.409+0100 I -        [main] TOTALS                         | tests:    9 | fails:    6 | assert calls:          0 | time secs:  0.005
      2018-02-20T14:16:24.409+0100 I -        [main] Failing tests:
      2018-02-20T14:16:24.409+0100 I -        [main] 	 UnicodeString/RemoveDiacritics Failed
      2018-02-20T14:16:24.409+0100 I -        [main] 	 UnicodeString/CaseFolding Failed
      2018-02-20T14:16:24.410+0100 I -        [main] 	 UnicodeString/CaseFoldingTurkish Failed
      2018-02-20T14:16:24.410+0100 I -        [main] 	 UnicodeString/CaseFoldingAndRemoveDiacritics Failed
      2018-02-20T14:16:24.410+0100 I -        [main] 	 UnicodeString/SubstringMatch Failed
      2018-02-20T14:16:24.410+0100 I -        [main] 	 UnicodeString/SubstringMatchTurkish Failed
      2018-02-20T14:16:24.410+0100 I -        [main] FAILURE - 6 tests in 1 suites failed
      

      When I disable/remove altivec include in src/mongo/db/fts/unicode/byte_vector.h everything works.

      When I debuged byte_vector_test and error {{FAIL: MaskAny Expected ByteVector::countInitialZeros(mask) == offset (16 == 0) @src/mongo/db/fts/unicode/byte_vector_test.cpp:70}} I figured out that vec_extract(vec_vbpermq(_data, bits), 0) in ByteVector::maskHight() returns 0.

      I don't know if this is mongodb problem or problem in compiler,... Do you have any idea how to easily test it? (I don't know ppc64 altivec instructions well) What could be the minimal reproducer in case of compiler problem?

            Assignee:
            mira.carey@mongodb.com Mira Carey
            Reporter:
            mskalick Marek Skalický
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: