dm.cs.tu-dortmund.de/mlbits/neural-nlp-rag/
Retrieval-Augmented Generation – Lecture Notes
the word probabilities based on the retrieved documents
\[p'(y_i|x,y_{1\ldots i-1}) = \sum_z p_\eta(z|x) p(y_i|x,z,y_{1\ldots i-1})\] (note: always only using one document \(z\) at a time).
fine-tuning [...] retrieve from a database (e.g., Wikipedia)
bot chooses to answer with retrieval, or not (Blenderbot 1, [ RDGJ21 ] )
can store summaries of earlier conversation (Blenderbot 2, [ XuSzWe22 ] )
can search the [...] 2011. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1 (2011), 117–128. DOI: 10.1109/TPAMI.2010.57
[LPPP20]
Lewis, P.S.H., Perez, E., Piktus, A., Petroni, …
