[SERVER-18479] Clang Address Sanitizer errors with RLP 7.11.1 on Linux x64 Created: 14/May/15  Updated: 11/Oct/18  Resolved: 11/Oct/18

Status: Closed
Project: Core Server
Component/s: Text Search
Affects Version/s: 3.1.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mark Benvenuto Assignee: DO NOT USE - Backlog - Platform Team
Resolution: Done Votes: 0
Labels: platforms-re-triaged
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-28498 Stop running the rlp* tasks on the "E... Closed
is related to SERVER-26130 RLP memory errors cause deadlock in u... Closed
Operating System: ALL
Participants:
Linked BF Score: 0

 Description   

This a issue for all the leaks in RLP for tracking purposes.

Version
Rosette Linguistics Platform (RLP) Version 7.11.1, Jul 8 2014 21:04:22
Compiler: Clang 3.5
Platform: Fedora 21, Linux 3.19.1-201.fc21.x86_64 #1 SMP Wed Mar 18
04:29:24 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Glibc 2.20

How to reproduce:
Using the C++ rlbl_sample, modify the code round line number 179, to say

BT_LanguageID langId = BT_LANGUAGE_ARABIC;

to ensure Arabic is used. Modify for other languages highlighted below.

Also, choose "European Language Analyzer2" in rlp-bl-context.xml instead of
"European Language Analyzer"

How to build
CXX="clang++"
$CXX -g -fsanitize=address -fno-omit-frame-pointer -m64 -c -DNDEBUG
-DBT_RELEASE_BUILD -D_GNU_SOURCE -DBT_LINUX -D_STL=std
-I../../../rlp/rlp/include -I../../../rlp/rlp/include/amd64-glibc25-gcc41
-I../../../rlp/utilities/include -pthread -pipe -ffor-scope
-fno-gnu-keywords -fPIC -Wall -W rlbl_sample.cpp -o
obj/amd64-glibc25-gcc41/rlbl_sample.o
$CXX -g -fsanitize=address -fno-omit-frame-pointer -m64
-Wl,-rpath='$ORIGIN/../../lib/amd64-glibc25-gcc41' -o
../../../rlp/bin/amd64-glibc25-gcc41/rlbl_sample
./obj/amd64-glibc25-gcc41/rlbl_sample.o
-L../../../rlp/lib/amd64-glibc25-gcc41 -lbtrlpcore
-L../../../rlp/lib/amd64-glibc25-gcc41 -lbtutils -lpthread -ldl -ldl

How to run
echo "hello world" > text_input.txt;
~/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample /home/mark/basis
text_input.txt bl text_output.txt

Errors from Address-Sanitizer
Arabic

31664==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new [] 
vs operator delete) on 0x602000c68070 
#0 0x4361ab in operator delete(void*) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x4361ab) 
#1 0x7f3ee623117a in 
BT_DisambiguationDecoderState::~BT_DisambiguationDecoderState() 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtutils.so.7.11+0x10a17a) 
#2 0x7f3ee6230990 in 
BT_PerceptronDecoder::Decode(BT_PerceptronSentence*) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtutils.so.7.11+0x109990) 
#3 0x7f3ed9f40b22 in 
bt::rlp::arbl::ProcessorContext::Run(BT_Blackboard*) const 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/bt_lp_arbl.so+0x2ab22) 
#4 0x7f3ee651960e in BT_Blackboard::Run() 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtrlpcore.so.7.11+0x2b60e) 
#5 0x7f3ee6520cb0 in BT_RLP_ContextImp::ProcessRun() 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtrlpcore.so.7.11+0x32cb0) 
#6 0x7f3ee65216fa in BT_RLP_ContextImp::ProcessUTF16Buffer(unsigned 
short const*, unsigned int, BT_LanguageID_enum) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtrlpcore.so.7.11+0x336fa) 
#7 0x4be592 in run(char const*, char const*, bool, char const*) 
/home/mark/basis/rlp/samples/cplusplus/rlbl_sample.cpp:193:10 
#8 0x4bb438 in main 
/home/mark/basis/rlp/samples/cplusplus/rlbl_sample.cpp:61:5 
#9 0x7f3ee4f33fdf in __libc_start_main 
/usr/src/debug/glibc-2.20/csu/libc-start.c:289 
#10 0x4bab7c in _start 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x4bab7c)

Urdu/Dari/Simplified Chinese (exact counts of leacked objects vary)

Direct leak of 12792 byte(s) in 13 object(s) allocated from: 
#0 0x435c6b in operator new(unsigned long) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x435c6b) 
#1 0x7f0d674b222a in bt_bl2::BT_Bl1ConfigParser::EndElement(char 
const*) (/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/bt_lp_bl2.so+0xf22a)
 
Indirect leak of 3534 byte(s) in 78 object(s) allocated from: 
#0 0x49d50b in malloc 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x49d50b) 
#1 0x7f0d674badf9 in btstlp_std::_String_base >::_M_allocate_block(unsigned long) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/bt_lp_bl2.so+0x17df9)
 
Indirect leak of 160 byte(s) in 5 object(s) allocated from: 
#0 0x49d50b in malloc 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x49d50b) 
#1 0x7f0d674bb5e5 in 
bt_bl2::ConfigEntry::ConfigEntry(bt_bl2::ConfigEntry const&) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/bt_lp_bl2.so+0x185e5)
 
SUMMARY: AddressSanitizer: 16486 byte(s) leaked in 96 allocation(s).

English

... TRIMMED.... 
Indirect leak of 24 byte(s) in 1 object(s) allocated from: 
#0 0x435c6b in operator new(unsigned long) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x435c6b) 
#1 0x7fa520f4fc30 in BT_FileMap::GetPlatformFileMap(char const*, bool, 
unsigned int) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtutils.so.7.11+0xc3c30)
 
Indirect leak of 8 byte(s) in 1 object(s) allocated from: 
#0 0x49d50b in malloc 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x49d50b) 
#1 0x7fa515ad8471 in btstlp_std::vector 
>::_M_insert_overflow(bt_bl2::BL2UserDictionary**, 
bt_bl2::BL2UserDictionary* const&, btstlp_std::__true_type const&, unsigned 
long, bool) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/bt_lp_bl2.so+0x17471)
 
Indirect leak of 8 byte(s) in 1 object(s) allocated from: 
#0 0x49d50b in malloc 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x49d50b) 
#1 0x7fa520fa1358 in 
bt::fst::Matcher::Matcher(btstlp_std::vector > const&) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtutils.so.7.11+0x115358)
 
Indirect leak of 8 byte(s) in 1 object(s) allocated from: 
#0 0x49d50b in malloc 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/rlbl_sample+0x49d50b) 
#1 0x7fa520fadcbd in btstlp_std::vector 
>::_M_insert_overflow(bt::fst::Matcher**, bt::fst::Matcher* const&, 
btstlp_std::__true_type const&, unsigned long, bool) 
(/home/mark/basis/rlp/bin/amd64-glibc25-gcc41/../../lib/amd64-glibc25-gcc41/libbtutils.so.7.11+0x121cbd)
 
SUMMARY: AddressSanitizer: 5173804 byte(s) leaked in 4341 allocation(s).



 Comments   
Comment by Mark Benvenuto [ 11/Oct/18 ]

Since RLP has been removed, we are no longer tracking these upstream issues.

Generated at Thu Feb 08 03:47:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.