context caching
/ˈkɒntekst ˌkæʃɪŋ/
saving the processed state of a prompt prefix to avoid recomputing it
“Context caching is ideal for chatting with long documents.”
Origin: Latin contextus + caching
/ˈkɒntekst ˌkæʃɪŋ/
saving the processed state of a prompt prefix to avoid recomputing it
“Context caching is ideal for chatting with long documents.”
Origin: Latin contextus + caching