Year: 2025
Type: conference-paper
Abstract: Detecting ambiguity is important for language understanding, including uncertainty estimation, humour detection, and processing garden path sentences.We assess language models' sensitivity to ambiguity by introducing an adversarial ambiguity dataset that includes syntactic, lexical, and phonological ambiguities along with adversarial variations (e.g., word-order changes, synonym replacements, and randombased alterations).Our findings show that direct prompting fails to robustly identify ambiguity, while linear probes trained on model representations can decode ambiguity with high accuracy, sometimes exceeding 90%.Our results offer insights into the prompting paradigm and how language models encode ambiguity at different layers.We release both our code and data: coastalcph/lm_ambiguity. spaces/lmsys/chatbot-arena-leaderboard a asa aulllllll_s lb___srssj_ayyylysnnnnydoooononnnnomyyyynmmm y|m T|||| hTTTT|ehhhhT eeeebh oebbbbyoooob yyyyso aysssswaaaa swww at hwttttehhhh t eeeegth eieggddrl eiieelwrrswttl eeocwwiccomthiittpttiia hhvavetn eeh ttw ehhwwwi eetdiiiht ttrgtthhh eetl shtl ltetehshssehes.eeeccke toomseseppp.yloeysn.g..soclasoscpsoe.p.e.p p t pa*aoooo bllbpp|ll jhayyynndomdonnamynm tne .peoepoblbppl_jjjjj_sjT_jj__j_____rrssrrsrsrrsssaayayayynennnddoodoooononnonnmymyyy__yym mm_r |__|||rrrs||||| rhhT|hT| T||ahT| yyh|hT|||| hyh hTwy TTyhhTTTmm hTeeehmhmeehhth e eeem m T ehe hhnh ne am s a eem me|| m a|mma|aa || a TT nTnanwaaTnnaaaa h n nhos snnnseess e aea sssa sssm wann t swm awawmmawaaaw aawwnat ttatattth hh n h htttnttttnwetAJMJehehss h oeehhh s i ls seeh ertth te eantaa ewweammdpbhoocrraaeh twn rnx awo uuowiocwaaiqdacsstrylmgbnkeft yaeotiai mmmdctwho yoppilinoxuueoinatrmlnuowsurtaaittahhowmmiieemrppcytryslhitcialhhagawndwtntcs eennnetish olylwo htiehne aoa attcitihreinwtpahwtlge hh t riniintn iitihrtrtwgksllwhde tieistntliiih t wie wi iti hhrtnviitm tnsvootiihhwrwtiiiethvihihwtwtti esmmti i cettwntwehhiiietiitech nttthti etii ihehwetnt naahto ththitieie wtetth t ehwheetnnihhpt htelehlewnt tteeltei c tt eihltett tti elhhtthteww hstthlithl wtethdewt .tseehhltlcpdbhtbs*tleltieih leects shitltseoloc elciuolorrl msmpgbcltcptetltoshehcsese ptosheltlhierxpaoycclsheolitseoptltoethhcocnatusssilpelttekes.eymolcpsohsnktllpch..
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412887880