AI/ML

I'm sure OpenAI's implementation of "deep research" is great, but I can't afford that. Ai2’s ScholarQA tool is FREE and open source!! Allen AI’s Scholar QA: https://scholarqa.allen.ai/ Please Like and Subscribe to support the channel! @LearnMetaAnalysis Access state of the art LLMs all in one place with ChatLLM – My 3 month review of ChatLLM: https://youtu.be/_Z3nLKvTbGc Tutorials and how-to guides: Connect a LLM to your Zotero (or any other local folder): https://youtu.be/b2BSZfOtD_w Conventional meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkEbYpBIgikgE0y9QR7QIgzs Three-level meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkHwRmu_TJXa7fSb6-WBXXoJ Three-level meta-analysis with correlated and hierarchical effects and robust variance estimation: https://www.youtube.com/playlist?list=PLXa5cTEormkEGenfcnp9X5dQUhmm7f9Jp Want free point and click (no coding required) meta-analysis software? Check out Simple Meta-Analysis: https://learnmeta-analysis.com/pages/simple-meta-analysis-software Tired of manually extracting data for systematic review and meta-analysis? Check out AI-Assisted Data Extraction, a free package for R! https://youtu.be/HuWXbe7hgFc Free ebook on meta-analysis in R (no download required): https://noah-schroeder.github.io/reviewbook/ Visit our website at https://learnmeta-analysis.com/ 0:00 OpenAI’s Deep Research 0:36 ScholarQA 1:26 First Test 11:49 Second Test 21:15 Debrief

Granite Snack Cookbook -- easily consumable recipes (python notebooks) that showcase the capabilities of the Granite models - ibm-granite-community/granite-snack-cookbook

I tested my local, small LLM connected to my Zotero database in Open WebUI against NotebookLM. The results were really, truly surprising. Please Like and Subscribe to support the channel! @LearnMetaAnalysis I tested Granite 3.1-2b and Granite 3.1-8b vs NotebookLM. I am now more impressed than ever! If you find Granite hallucinating with the default temperature setting, I've had good results setting it at about .20. Results from the testing: https://docs.google.com/spreadsheets/d/1nW7B_A7fzayaMancYwU89ApOOfLG5WOefzstdL_fg-Y/edit?usp=sharing Learn how to set up your own local, private research assistant like Dwayne with no coding: https://youtu.be/b2BSZfOtD_w The original RAG template I adapted (my modified version is in the google sheet): https://medium.com/@kelvincampelo/how-ive-optimized-document-interactions-with-open-webui-and-rag-a-comprehensive-guide-65d1221729eb Tutorials and how-to guides: Connect a LLM to your Zotero (or any other local folder): https://youtu.be/b2BSZfOtD_w Conventional meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkEbYpBIgikgE0y9QR7QIgzs Three-level meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkHwRmu_TJXa7fSb6-WBXXoJ Three-level meta-analysis with correlated and hierarchical effects and robust variance estimation: https://www.youtube.com/playlist?list=PLXa5cTEormkEGenfcnp9X5dQUhmm7f9Jp Want free point and click (no coding required) meta-analysis software? Check out Simple Meta-Analysis: https://learnmeta-analysis.com/pages/simple-meta-analysis-software Tired of manually extracting data for systematic review and meta-analysis? Check out AI-Assisted Data Extraction, a free package for R! https://youtu.be/HuWXbe7hgFc Free ebook on meta-analysis in R (no download required): https://noah-schroeder.github.io/reviewbook/ Visit our website at https://learnmeta-analysis.com/ 0:40 Setup 4:05 Results

Learn how to make simple changes that help your LLM chat with Zotero like a pro! I’m getting well written, well-cited results from a 2b parameter LLM. Please Like and Subscribe to support the channel! @LearnMetaAnalysis Embedding result testing: https://docs.google.com/spreadsheets/d/1P3rOLEO_NtCUYxaFIVaVZfMv4BOkQb3w/edit?usp=sharing&ouid=111617079417577058774&rtpof=true&sd=true Granite 3.1 Dense is my favorite LLM for this setup right now, it's available in 2b and 8b versions for ollama - https://ollama.com/library/granite3.1-dense:2b Snowflake Arctic Embed 2 has performed well for me so far as an embedding model: https://ollama.com/library/snowflake-arctic-embed2 MTEB leaderboard to see what embedding models perform well at different tasks: https://huggingface.co/spaces/mteb/leaderboard How to connect a LLM to Zotero - https://youtu.be/b2BSZfOtD_w I generally prefer local, private LLMs, but if you need large SOTA models like ChatGPT, Claude, Deepseek, Gemini, or Grok, check out ChatLLM - My 3 month review of ChatLLM: https://youtu.be/_Z3nLKvTbGc Tutorials and how-to guides: Conventional meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkEbYpBIgikgE0y9QR7QIgzs Three-level meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkHwRmu_TJXa7fSb6-WBXXoJ Three-level meta-analysis with correlated and hierarchical effects and robust variance estimation: https://www.youtube.com/playlist?list=PLXa5cTEormkEGenfcnp9X5dQUhmm7f9Jp Want free point and click (no coding required) meta-analysis software? Check out Simple Meta-Analysis: https://learnmeta-analysis.com/pages/simple-meta-analysis-software Tired of manually extracting data for systematic review and meta-analysis? Check out AI-Assisted Data Extraction, a free package for R! https://youtu.be/HuWXbe7hgFc Free ebook on meta-analysis in R (no download required): https://noah-schroeder.github.io/reviewbook/ Visit our website at https://learnmeta-analysis.com/ 0:15 Knowledge 0:59 Help make this better 1:32 Modify ‘knowledge’ settings 5:46 Demo of results 7:22 Top K 11:25 Testing Different embeddings 13:25 Use # not models 14:45 Impatient people (like me!) start here 21:38 Example Results

Learn how to use chat with your Zotero database using a private, local LLM with no coding required, Llama, Deepseek, any LLM you want! Please like and subscribe to help support the channel. @LearnMetaAnalysis Ollama - https://ollama.com/ Docker - https://www.docker.com/ Open WebUI Quickstart - https://docs.openwebui.com/getting-started/quick-start Zotero - https://www.zotero.org/ Zotero Directory Information - https://www.zotero.org/support/zotero_data Tutorials and how-to guides: Getting started with Open WebUI: https://youtu.be/gm_1VUg3L24 Conventional meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkEbYpBIgikgE0y9QR7QIgzs Three-level meta-analysis: https://www.youtube.com/playlist?list=PLXa5cTEormkHwRmu_TJXa7fSb6-WBXXoJ Three-level meta-analysis with correlated and hierarchical effects and robust variance estimation: https://www.youtube.com/playlist?list=PLXa5cTEormkEGenfcnp9X5dQUhmm7f9Jp Want free point and click (no coding required) meta-analysis software? Check out Simple Meta-Analysis: https://learnmeta-analysis.com/pages/simple-meta-analysis-software Tired of manually extracting data for systematic review and meta-analysis? Check out AI-Assisted Data Extraction, a free package for R! https://youtu.be/HuWXbe7hgFc Free ebook on meta-analysis in R (no download required): https://noah-schroeder.github.io/reviewbook/ Visit our website at https://learnmeta-analysis.com/ 0:00 What we’re building 1:40 Requirements 7:05 Sync Zotero database 10:13 Custom model 12:13 It works! 17:26 Changing LLM 18:54 Updating knowledge database

Deepseek R1 - the latest and greatest open source reasoning LLM - has taken the world by storm and a lot of content creators are doing a great job covering its implications and strengths/weaknesses. What I haven’t seen a lot of though is actually using R1 in agentic workflows to truly leverage its power. So that’s what I’m showing you in this video - we’ll be using the power of R1 to make a simple but super effective agentic RAG setup. We’ll be using Smolagents by HuggingFace to create our agent - it’s the simplest agent framework out there and many of you have been asking me to try it out. This agentic RAG setup centers around the idea that reasoning LLMs like R1 are extremely powerful but quite slow. Because of this, a lot of people are starting to experiment with combining the raw power of a model like R1 with a more lightweight and fast LLM to drive the primary conversation/agent flow. Think of basically giving R1 as a tool for an agent to use when it needs more reasoning power at the cost of a slower response (and higher costs). That’s what we’ll be doing here - creating an agent that has an R1 driven RAG tool to extract in depth insights from a knowledgebase. The example in this video is meant to be an introduction to these kind of reasoning agentic flows. That’s why I keep it simple with Smolagents and a local knowledgebase. But I’m planning on expanding this much further soon with a much more robust but still similar flow built with Pydantic AI and LangGraph! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Community Voting period of the oTTomator Hackathon is open! Head on over to the Live Agent Studio now and test out the submissions and vote for your favorite agents. There are so many incredible projects to try out! https://studio.ottomator.ai All the code covered in this video + instructions to run it can be found here: https://github.com/coleam00/ottomator-agents/tree/main/r1-distill-rag SmolAgents: https://huggingface.co/docs/smolagents/en/index R1 on Ollama: https://ollama.com/library/deepseek-r1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 00:00 - Why R1 for Agentic RAG? 01:56 - Overview of our Agent 03:33 - SmolAgents - Our Ticket to Fast Agents 06:07 - Building our Agentic RAG Agent with R1 14:17 - Creating our Local Knowledgebase w/ Chroma DB 15:45 - Getting our Local LLMs Set Up with Ollama 19:15 - R1 Agentic RAG Demo 21:42 - Outro ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Join me as I push the limits of what is possible with AI. I'll be uploading videos at least two times a week - Sundays and Wednesdays at 7:00 PM CDT!

Last week I introduced you to Crawl4AI - an open source and LLM friendly web scraper that makes it super easy to crawl any website and format it for a RAG knowledgebase for your AI agent. I even created a full AI agent as a follow up video that leverages this knowledgebase I created with Crawl4AI. A TON of you asked me to do the same thing in n8n, so here it is! In this video I show you exactly how to deploy Crawl4AI super easily with Docker and leverage it within your n8n workflows to crawl website pages in seconds. We even build a simple AI agent that uses this knowledgebase to become an expert at the documentation for Pydantic AI - my favorite AI Agent framework right now! There are a lot of ways to crawl websites, but many of them are expensive, slow, and/or difficult to work with. Crawl4AI on the other hand is easy to use, fast, and completely free since it is open source. The only thing you have to pay for is the machine in the cloud to run your crawler, and that’s only if you aren’t just running it on your computer! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Check out TEN Agent now (completely open source!) and see how easy it is to get started building voice AI agents for free: GitHub repo: https://github.com/TEN-framework/TEN-Agent Playground: https://agent.theten.ai/ If you aren't aware, voice agents are one of the biggest needs businesses have right now, so if you're a developer looking to make money with AI, tools like TEN Agent are definitely worth learning and using! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is the n8n workflow I covered in this video! It’s in a folder along with all the other Crawl4AI stuff I’ve done on my channel recently with Python. https://github.com/coleam00/ottomator-agents/blob/main/crawl4AI-agent/n8n-version/Crawl4AI_Agent.json Register now for the oTTomator AI Agent Hackathon with a $6,000 prize pool! https://studio.ottomator.ai/hackathon/register Try the Pydantic AI expert out now on the Live Agent Studio! https://studio.ottomator.ai Crawl4AI: https://github.com/unclecode/crawl4ai ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 00:00 - Intro to Crawl4AI + n8n 01:45 - Showing off the n8n Workflow 02:31 - What We're Crawling (and Ethics) 04:36 - How to Deploy Crawl4AI for n8n 07:57 - Deploying Crawl4AI with Docker 13:06 - TEN Agent 15:27 - Building Crawl4AI into n8n 29:15 - n8n + Crawl4AI RAG Demo 32:43 - Outro ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Join me as I push the limits of what is possible with AI. I'll be uploading videos at least two times a week - Sundays and Wednesdays at 7:00 PM CDT!

🚀🤖 Crawl4AI, Open-source LLM-Friendly Web Crawler & Scraper

Buckle up - HUGE amount of value in this video for building RAG AI Agents that actually work. Honestly I could have made this video into an entire course but I wanted to give it away to you for free. :) RAG is the most common approach for providing external knowledge to an LLM. The problem is, once you have your own curated data in a vector database as a knowledgebase for your LLM, often times these RAG setups can be very underwhelming. The wrong text is returned from the search, the LLM ignores the context provided, etc. The logic of RAG makes sense in your head but it just doesn’t work in practice. And you certainly aren’t alone! That’s why there is a TON of research in the industry for how to essentially just do RAG better. There are a lot of strategies out there, but out of all the ones I’ve researched and tried myself, agentic RAG is the most obvious, works the best, and is what I’m going to introduce you to and show you exactly how to implement in this video. In the last video on my channel, I showed you how to use Crawl4AI, an open source LLM-friendly web crawler, to scrape entire websites for RAG SUPER fast. We used the entire documentation for my favorite agent framework, Pydantic AI, as an example. Now we’re taking this MUCH further by: 1. Putting all the documentation in a database for RAG 2. Creating an agentic RAG agent to use this knowledgebase with Pydantic AI 3. Building a frontend to chat with our agent using Streamlit I’ll explain exactly what Agentic RAG is, what makes it so powerful, and then this AI agent we’ll build in the video will be the perfect example! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Try GPUStack for free - it's open source and you can find their GitHub repo here: https://github.com/gpustack/gpustack I don't have the pleasure of being sponsored by open source projects often, so this was a treat! It's the best GPU cluster manager for LLM inference that I have seen and a very honest recommendation! Here is their main site as well: https://gpustack.ai/ Key features of GPUStack: 1. Heterogeneous GPU cluster management including Linux, Mac and Windows with Nvidia, and Apple Silicon. AMD coming soon! 2. Distributed inference with smart scheduling: GPUStack can distribute a big model to multiple heterogeneous workers. Automatically calculates and decide whether distributed inference is required and configure it automatically. 3. Rich model types support: GPUStack supports LLM, VLM, Image Generation, Embedding, Rerank, TTS&STT models. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Previous video with Crawl4AI: https://youtu.be/JWfNLF_g_V0 All code for this Agentic RAG Agent can be found here: https://github.com/coleam00/ottomator-agents/tree/main/crawl4AI-agent Try this agent yourself right now on the Live Agent Studio (called the "Pydantic AI Expert")! https://studio.ottomator.ai Diagram to follow along with the knowledgebase creation flow: https://claude.site/artifacts/f4dca1c3-f137-4b82-9254-dfa01ca43802 Weaviate Article on Agentic RAG: https://weaviate.io/blog/what-is-agentic-rag ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 00:00 - Agentic RAG - the Holy Grail of RAG 02:18 - What is Agentic RAG? 06:22 - Breaking our Agent Down Step by Step 08:33 - Try this Agent Now for Free 09:00 - Code Overview 09:58 - Crawl4AI Review 10:52 - Creating Our Knowledgebase for Supabase 21:38 - GPUStack 23:33 - Supabase Setup 26:08 - Getting Crawl4AI Data into Supabase 28:09 - Basic RAG AI Agent with Pydantic AI 33:44 - Testing our Basic RAG Agent 36:33 - Agentic RAG Implementation 40:40 - Demo of Our Agentic RAG Agent 41:37 - Streamlit UI 44:53 - Outro ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Join me as I push the limits of what is possible with AI. I'll be uploading videos at least two times a week - Sundays and Wednesdays at 7:00 PM CDT! Sundays and Wednesdays are for everything AI, focusing on providing insane and practical educational value. I will also post sometimes on Fridays at 7:00 PM CDT - specifically for platform showcases - sometimes sponsored, always creative in approach!

All the open source AI Agents hosted on the oTTomator Live Agent Studio platform! - coleam00/ottomator-agents

One of the biggest challenges we face with LLMs is their knowledge is too general and limited for anything new. That’s why RAG is such a huge topic when it comes to AI right now - it’s a method for providing an LLM with external knowledge you curate so it can become an expert at something it wasn’t before - a specific AI framework, your ecommerce store, you name it. The problem is, that “curate” step can be very difficult and slow. That is where Crawl4AI comes in! Crawl4AI is an open source web crawling framework specifically designed for scraping websites and formatting the output in the BEST possible way for an LLM to understand. The best part is it solves a LOT of problems we typically have with systems that crawl websites - usually they are slow, resource intensive, and complicated. But Crawl4AI is VERY fast, intuitive, easy to set up, and extremely memory efficient. In this video, I show you how to use Crawl4AI to super easily crawl websites for LLMs in just seconds, and at the end I even show you a RAG AI agent I’ve built to be a “Pydantic AI” framework expert using Crawl4AI to build the knowledgebase. And you could really take this and use it for any website you want. Next video I'll do a deep dive into this agent! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Register now for the oTTomator AI Agent Hackathon with a $6,000 prize pool! https://studio.ottomator.ai/hackathon/register All code for this Crawl4AI RAG Agent can be found here: https://github.com/coleam00/ottomator-agents/tree/main/crawl4AI-agent Crawl4AI GitHub: https://github.com/unclecode/crawl4ai ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 00:00 - The Beauty of Crawl4AI 02:16 - Why Crawl4AI? 05:25 - Basic Crawl4AI Example - Single Page Crawl 06:56 - Crawling Multiple Pages 08:58 - Ethics of Web Scraping 10:01 - Crawling Multiple Pages Continued 12:24 - FAST Parallel Page Crawling 15:19 - Crawl4AI RAG AI Agent 17:48 - Outro ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Join me as I push the limits of what is possible with AI. I'll be uploading videos at least two times a week - Sundays and Wednesdays at 7:00 PM CDT! Sundays and Wednesdays are for everything AI, focusing on providing insane and practical educational value. I will also post sometimes on Fridays at 7:00 PM CDT - specifically for platform showcases - sometimes sponsored, always creative in approach!

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper - unclecode/crawl4ai

An open-source RAG-based tool for chatting with your documents. - Cinnamon/kotaemon

An open-source RAG-based tool for chatting with your documents. - Cinnamon/kotaemon

RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lightweight, open-source alternative to LangSmith, focusing on ...

Roaming RAG offers a fresh take on Retrieval-Augmented Generation, letting LLMs navigate well-structured documents like a human—exploring outlines and diving into sections to find answers. Forget complex retrieval setups and vector databases; this streamlined approach delivers rich context and reliable answers with less hassle. It’s perfect for structured content like technical manuals, product guides, or the innovative llms.txt format designed to make websites LLM-friendly.

RAG that intelligently adapts to your use case, data, and queries - circlemind-ai/fast-graphrag

Structured extraction - where an LLM helps turn unstructured text (or image content) into structured data - remains one of the most directly useful applications of LLMs. NuExtract is a …

Last week I was helping a friend of mine to get one of his new apps off the ground. I can’t speak much about it at the moment, other than like most apps nowadays it has some AI sprinkled over …

Audio Overview is a fun new feature of Google’s NotebookLM which is getting a lot of attention right now. It generates a one-off custom podcast against content you provide, where …

The ultimate AI productivity app that protects your privacy. Bring all your apps and data into one AI-powered search and assistant. Get it for you and for your teams today.

Summarize and query from a lot of heterogeneous documents. Any LLM provider, any filetype, scalable, under developpement - thiswillbeyourgithub/WDoc

Empowering RAG with a memory-based data interface for all-purpose applications! - qhjqhj00/MemoRAG

Here's an interesting new embedding/RAG technique, described by Anthropic but it should work for any embedding model against any other LLM. One of the big challenges in implementing semantic search …

New version of my `files-to-prompt` CLI tool for turning a bunch of files into a prompt suitable for piping to an LLM, [described here previously](https://simonwillison.net/2024/Apr/8/files-to-prompt/). It now has a `-c/--cxml` …

Interesting tips here from Anthropic's documentation about how to best prompt Claude to work with longer documents. **Put longform data at the top**: Place your long documents and inputs …

Build a retrieval system with semantic, full-text, and fuzzy search in Postgres to be used as a backbone in RAG pipelines.

This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and cont...

Uncovering the design space of RAG systems that use unstructured data and future work directions that are important.

Retrieval Augmented Generation (RAG) is a technique for adding extra “knowledge” to systems built on LLMs, allowing them to answer questions against custom information not included in their training data. …