More Graph, More Agents: Scaling Graph Reasoning with LLMs Graph reasoning tasks have proven to be a tough nut to crack for Large Language Models (LLMs).…

Sign in or join now to see posts like this one and more.

**!!!! Great Talk with Bradley Rees NVIDIA RAPIDS cuGraph lead at KDD 24 Conference !!** We had an excellent discussion about the cuGraph user experience in…

Knowledge Graphs (KGs) represent human-crafted factual knowledge in the form of triplets (head, relation, tail), which collectively form a…

An easy trick to improve your LLM results without fine-tuning. Many people know "Few-Shot prompting" or "Chain of Thought prompting". A new (better) method was… | 77 comments on LinkedIn

Can LLMs understand graphs? The results might surprise you. Graphs are everywhere, from social networks to biological pathways. As AI systems become more…

GNN-RAG Combines the language understanding abilities of LLMs with the reasoning abilities of GNNs in a RAG style. The GNN extracts useful and relevant…

Revolutionizing Document Reranking with G-RAG: A Graph-Based Approach ... Discover how a novel graph-based reranker is transforming the way we retrieve…

Learn more about what graph embeddings are and how they're used to accelerate real-time analytics. Find out about graph embedding algorithms and storage.

Graph Neural Prompting with Large Language Models

🌟GraphGPT🌟 (385 stars in GitHub) is accepted by 🌟SIGIR'24🌟 (only 20.1% acceptance rate)! Thank Yuhao Yang, wei wei, and other co-authors for their precious…

Large language models (LLMs) have achieved great success in many fields, and recent works have studied exploring LLMs for graph discriminative tasks such as node classification. However, the abilities of LLMs for graph generation remain unexplored in the literature. Graph generation requires the LLM to generate graphs with given properties, which has valuable real-world applications such as drug discovery, while tends to be more challenging. In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments. Specifically, we propose several tasks tailored with comprehensive experiments to address key questions regarding LLMs' understanding of different graph structure rules, their ability to capture structural type distributions, and their utilization of domain knowledge for property-based graph generation. Our evaluations demonstrate that LLMs, particularly GPT-4, exhibit preliminary abilities in graph generation tasks, including rule-based and distribution-based generation. We also observe that popular prompting methods, such as few-shot and chain-of-thought prompting, do not consistently enhance performance. Besides, LLMs show potential in generating molecules with specific properties. These findings may serve as foundations for designing good LLMs based models for graph generation and provide valuable insights and further research.

G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering

LLM frameworks like LangChain are moving towards a graph-based approach for handling their workflows. This represents the initial steps of a much larger… | 90 comments on LinkedIn

Over the past 5 years, news about AI has been filled with amazing research – at first focused on graph neural networks (GNNs) and more recently about large language models (LLMs). Understand that business tends to use connected data – networks, graphs – whether you’re untangling supply networks in Manufacturing, working on drug discovery for Pharma, or mitigating fraud in Finance. Starting from supplier agreements, bill of materials, internal process docs, sales contracts, etc., there’s a graph inside nearly every business process, one that is defined by language. This talk addresses how to leverage both natural language and graph technologies together for AI applications in industry. We’ll look at how LLMs get used to build and augment graphs, and conversely how graph data gets used to ground LLMs for generative AI use cases in industry – where a kind of “virtuous cycle” is emerging for feedback loops based on graph data. Our team has been engaged, on the one hand, with enterprise use cases in manufacturing. On the other hand we’ve worked as intermediaries between research teams funded by enterprise and open source projects needed by enterprise – particularly in the open source ecosystem for AI models. Also, there are caveats; this work is not simple. Translating from latest research into production-ready code is especially complex and expensive. Let’s examine caveats which other teams should understand, and look toward practical examples.

(This post & prompt technique was inspired by a conversation with Nicky Clarke and a recent post by Kurt Cagle, Rethinking Hypergraphs.) Building prompts for ultra-large context windows requires a complete reimagining of how prompts are created.

Discover how Graph of Thoughts aims to revolutionize prompt engineering, and LLMs more broadly, enabling more flexible and human-like problem-solving.

The Intersection of Graphs and Language Models 🔲 ⚫ Large language models (LLMs) have rapidly advanced, displaying impressive abilities in comprehending… | 27 comments on LinkedIn

Links * Python Examples * JS Examples * YouTube Last week we highlighted LangGraph - a new package (available in both Python and JS) to better enable creation of LLM workflows containing cycles, which are a critical component of most agent runtimes. As a part of the launch, we highlighted two simple runtimes:

⚡ Building language agents as graphs ⚡

Neuralizing Retrieval: Infusing Symbolic Reasoning into Language Models through Algorithmic Alignment ↗ Retrieval-augmented generation (RAG) has emerged as…

Python open source projects; natural language meets graph technologies; graph topological transformations; graph levels of detail (abstraction layers)

By request, here are the slides from our #neurips2023 presentation yesterday! We really enjoyed the opportunity to present the different aspects of the work… | 18 comments on LinkedIn

Graph plays a significant role in representing and analyzing complex relationships in real-world applications such as citation networks, social networks, and biological data. Recently, Large Language Models (LLMs), which have achieved tremendous success in various domains, have also been leveraged in graph-related tasks to surpass traditional Graph Neural Networks (GNNs) based methods and yield state-of-the-art performance. In this survey, we first present a comprehensive review and analysis of existing methods that integrate LLMs with graphs. First of all, we propose a new taxonomy, which organizes existing methods into three categories based on the role (i.e., enhancer, predictor, and alignment component) played by LLMs in graph-related tasks. Then we systematically survey the representative methods along the three categories of the taxonomy. Finally, we discuss the remaining limitations of existing studies and highlight promising avenues for future research. The relevant papers are summarized and will be consistently updated at: https://github.com/yhLeeee/Awesome-LLMs-in-Graph-tasks.

A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests

Where does current work on graph technologies intersect with large language models, for building AI apps?

We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information ("LLM thoughts") are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by 31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks.

Vectors need Graphs! Embedding vectors are a pivotal tool when using Generative AI. While vectors might initially seem an unlikely partner to graphs, their… | 61 comments on LinkedIn

Large Language Models (LLMs) have shown remarkable generalization capability with exceptional performance in various language modeling tasks. However, they still exhibit inherent limitations in precisely capturing and returning grounded knowledge. While existing work has explored utilizing knowledge graphs to enhance language modeling via joint training and customized model architectures, applying this to LLMs is problematic owing to their large number of parameters and high computational cost. In addition, how to leverage the pre-trained LLMs and avoid training a customized model from scratch remains an open question. In this work, we propose Graph Neural Prompting (GNP), a novel plug-and-play method to assist pre-trained LLMs in learning beneficial knowledge from KGs. GNP encompasses various designs, including a standard graph neural network encoder, a cross-modality pooling module, a domain projector, and a self-supervised link prediction objective. Extensive experiments on multiple datasets demonstrate the superiority of GNP on both commonsense and biomedical reasoning tasks across different LLM sizes and settings.