GenAI

In this context, “interaction” refers to the process of assessing how well a document matches a given search query by comparing their representations.

A dense retrieval model is a model that uses some type of neural network architecture to retrieve relevant documents for a search query.

Traditional methods for retrieval commonly use “no-interaction” retrieval models. In this case, the search query and documents are processed separately

Advantages of no-interaction retrieval models are primarily that they are fast and computationally efficient

These characteristics make full interaction models great for second-stage retrieval, like reranking a curated set of candidate documents

extremely computationally expensive

contextually rich

scalable and contextually rich

storage requirements - they require an embedding for each token, which requires a lot more storage for a complete set of vectors

Disadvantages of no-interaction retrieval models lie in the lack of interaction between the search query and the documents.

multimodal late interaction retrieval models

vision language models (VLMs) instead of text-only models

The primary evaluation metric used in τ-bench is pass^k, which measures the probability that all k independent task trials are successful for a given task, averaged across all tasks. Unlike the pass@k metric that is common for other LLM evaluations (which measures if at least one of k trials succeeds), pass^k evaluates consistency and reliability—critical qualities for customer service applications where consistent adherence to policies is essential.

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

Agents interact with their environment and typically consist of several important components

chain-of-thought

This is where planning comes in. Planning in LLM Agents involves breaking a given task up into actionable steps.

reasoning” and “thinking” a bit loosely as we can argue whether this is human-like thinking or merely breaking the answer down to structured steps.

without any examples (zero-shot prompting)

Providing examples (also called few-shot prompting7)

ReAct

Reflecting

These Multi-Agent systems usually consist of specialized Agents, each equipped with their own toolset and overseen by a supervisor.

three LLM roles

SELF-REFINE

To enable planning in LLM Agents, let’s first look at the foundation of this technique, namely reasoning.