Particularly rxvt. This is like death by 1000 paper cuts: nothing major, but super annoying.

1) refreshing the line (on backspaces, moving the cursor left, etc.) messed up the prompt (resulting in ">> ").
2) long lines get messed up (this may be related to 1, haven't looked yet)
3) alt shortcuts not supported

#1 (and maybe #2) is because rxvt expects \xb1[1G to return to the start of a line and xterm expects \xb1[0G. I have a feeling that there are a million of these and there's a reason readline is huge. Working on the ones listed above, I'll submit fixes upstream.