prompt leakage
/ˈprɒmpt ˈliːkɪdʒ/
a model revealing its hidden system prompt when coaxed
prompt leakage in a sentence
“One innocuous-looking question caused prompt leakage, and the bot recited its confidential instructions.”
Origin of prompt leakage
Latin promptus ready + Old Norse leka to drip
Related Words
hallucination
generating plausible-sounding but factually incorrect or fabricated information
sycophancy
over-agreeing with users and telling them what they want to hear rather than the truth
confabulation
filling gaps in knowledge with plausible but invented details
instruction drift
gradually deviating from initial instructions over long conversations
mode collapse
converging to repetitive or generic outputs regardless of varied inputs
catastrophic forgetting
losing previously learned capabilities when trained on new data