Weaviate - Podcast Details

Shows

Weaviate PodcastMUVERA with Rajesh Jayaram and Roberto Esposito - Weaviate Podcast #123!Multi-vector retrieval offers richer, more nuanced search, but often comes with a significant cost in storage and computational overhead. How can we harness the power of multi-vector representations without breaking the bank? Rajesh Jayaram, the first author of the groundbreaking MUVERA algorithm from Google, and Roberto Esposito from Weaviate, who spearheaded its implementation, reveal how MUVERA tackles this critical challenge.Dive deep into MUVERA, a novel compression technique specifically designed for multi-vector retrieval. Rajesh and Roberto explain how it leverages contextualized token embeddings and innovative fixed dimensional encodings to dramatically reduce storage requirements while maintaining high retrieval...2025-05-281h 13

Weaviate PodcastPatronus AI with Anand Kannappan - Weaviate Podcast #122!AI agents are getting more complex and harder to debug. How do you know what's happening when your agent makes 20+ function calls? What if you have a Multi-Agent System orchestrating several Agents? Anand Kannappan, co-founder of Patronus AI, reveals how their groundbreaking tool Percival transforms agent debugging and evaluation. Percival can instantly analyze complex agent traces, it pinpoints failures across 60 different modes, and it automatically suggests prompt fixes to improve performance. Anand unpacks several of these common failure modes. This includes the critical challenges of "context explosion" where agents process millions of tokens. He also explains domain adaptation for...2025-05-151h 01

Weaviate PodcastStructured Outputs with Will Kurt and Cameron Pfiffer - Weaviate Podcast #119!Hey everyone! Thanks so much for watching another episode of the Weaviate Podcast! Dive into the fascinating world of structured outputs with Will Kurt and Cameron Pfeiffer, the brilliant minds behind Outlines, the revolutionary open-source library from .txt.ai that's changing how we interact with LLMs. In this episode, we explore how constrained decoding enables predictable, reliable outputs from language models—unlocking everything from perfect JSON generation to guided reasoning processes.Will and Cameron share their journey to founding .txt.ai, explain the technical magic behind Outlines (hint: it involves finite state machines!), and debunk misconceptions around structured generation pe...2025-04-091h 10

Weaviate PodcastSynthetic Data with David Berenstein and Ben Burtenshaw - Weaviate Podcast #118!Synthetic Data: The Building Bocks of AI's Future! Hey everyone! I am SUPER EXCITED to publish the 118th episode of the Weaviate Podcast featuring David Berenstein and Ben Burtenshaw from HuggingFace! This podcast explores the intricacies of synthetic data generation, detailing methodologies such as data augmentation, distillation, and instruction refinement. The conversation delves into persona-driven synthetic data, highlighting applications like Persona Hub, and discusses algorithms to enhance diversity, complexity, and quality of generated data. Additionally, they cover integration with Hugging Face’s ecosystem, including Argilla for annotation, AutoTrain for fine-tuning, and advanced data exploration tools like the Data Studio an...2025-03-251h 02

Weaviate PodcastLetta AI with Sarah Wooders - Weaviate Podcast #117!Hey everyone! Thank you so much for watching the 117th episode of the Weaviate podcast! In this episode, we dive deep into the cutting edge of AI agent development with Sarah Wooders, co-founder and CTO of Letta AI. Emerging from Berkeley's Sky Computing Lab, Sarah and her team have pioneered a revolutionary approach to stateful agents - AI systems that genuinely remember both you and themselves across extended conversations. The conversation explores how the groundbreaking MemGPT project evolved into Letta's comprehensive Agent Development Environment (ADE), which empowers developers to build truly persistent AI experiences. Sarah shares powerful insights on...2025-03-0357 min

Weaviate PodcastAgent Experience with Matt Biilmann, Sebastian Witalec, and Charles Pierse - Weaviate Podcast #116!Hey everyone! Thank you so much for watching another episode of the Weaviate Podcast! I am SUPER excited to welcome Matt Biilmann, Co-Founder and CEO of Netlify, as well as Sebastian Witalec and Charles Pierse from Weaviate to discuss Agent Experience! You have probably heard about how you can connect LLMs to external software tools. This supercharges the capabilities of AI systems and what they can do. So what does that mean for you as a software developer?This podcast explores different ideas around designing software user experiences for Agents as well as Humans. How do we write documentation...2025-02-2752 min

Weaviate PodcastOptimizing Retrieval Agents with Shirley Wu - Weaviate Podcast #115!Hey everyone! Thank you so much for watching the 115th episode of the Weaviate Podcast featuring Shirley Wu from Stanford University!We explore the innovative Avatar Optimizer—a novel framework that leverages contrastive reasoning to refine LLM agent prompts for optimal tool usage. Shirley explains how this self-improving system evolves through iterative feedback by contrasting positive and negative examples, enabling agents to handle complex tasks more effectively.We also dive into the STaRK Benchmark, a comprehensive testbed designed to evaluate retrieval systems on semi-structured knowledge bases. The discussion highlights the challenges of unifying textual and re...2025-02-191h 00

Weaviate PodcastContextual AI with Amanpreet Singh - Weaviate Podcast #114!Hey everyone! Thank you so much for watching the 114th episode of the Weaviate Podcast featuring Amanpreet Singh, Co-Founder and CTO of Contextual AI! Contextual AI is at the forefront of production-grade RAG agents! I learned so much from this conversation! We began by discussing the vision of RAG 2.0, jointly optimizing generative and retrieval models! This then lead us to discuss Agentic RAG and how the RAG 2.0 roadmap is evolving with emerging perspectives on tool use. Amanpreet continues to further motivate the importance of continual learning of the model and the prompt / few-shot examples -- discussing the limits of...2025-02-1257 min

Weaviate PodcastCartesia AI with Karan Goel - Weaviate Podcast #113!Hey everyone! Thank you so much for watching the 113th episode of the Weaviate Podcast with Karan Goel from Cartesia AI! Cartesia AI is leading the AI world in text-to-speech models! As exciting as these new applications in speech generation are, Cartesia is also building around an incredibly exciting new neural network architecture that cuts across all of AI -- State Space Models. State Space Models (SSMs) present a new approach to modeling long sequences circumventing the quadratic attention bottlenecks of transformers. In the podcast, we discuss Karan's perspectives around end-to-end modeling, long context and Multimodal processing, building and...2025-01-2853 min

Weaviate PodcastGoogle Vertex AI RAG Engine with Lewis Liu and Bob van Luijt - Weaviate Podcast #112!Hey everyone! Thank you so much for watching the 112th episode of the Weaviate Podcast! This is another super exciting one, diving into the release of the Vertex AI RAG Engine, its integration with Weaviate and thoughts on the future of connecting AI systems with knowledge sources! The podcast begins by reflecting on Bob's experience speaking at Google in 2016 on Knowledge Graphs! This transitions into discussing the evolution of knowledge representation perspectives and things like the semantic web, ontologies, search indexes, and data warehouses. This then leads to discussing how much knowledge is encoded in the prompts themselves and...2025-01-1558 min

Weaviate PodcastMorningstar Intelligence Engine with Aravind Kesiraju - Weaviate Podcast #111!Hey everyone! I am SUPER EXCITED to publish the 111th Weaviate Podcast with Aravind Kesiraju from Morningstar! Aravind is a Principal Software Engineer who has lead the development behind the Morningstar Intelligence Engine! There are so many interesting aspects to this, and if you are building Agentic systems that would benefit from a high-quality financial retrieval API, you should check this out right now! The podcast dives into all sorts of ingredients that went into building this system: from custom RAG data pipelines with content management system integrations and embedding task queues, to exploring new chunking strategies, tool marketplaces...2025-01-0853 min

Weaviate PodcastArctic Embed with Luke Merrick, Puxuan Yu, and Charles Pierse - Weaviate Podcast #110!Hey everyone! Thank you so much for watching the 110th episode of the Weaviate Podcast! Today we are diving into Snowflake’s Arctic Embedding model series and their newly released Arctic Embed 2.0 open-source model, additionally supporting multilingual text embeddings. The podcast covers the origin of Arctic Embed, Pre-training embedding models, Matryoshka Representation Learning (MRL), Fine-tuning embedding models, Synthetic Query Generation, Hard Negative Mining, and Single-Vector Embeddings Models in the cohort of Multi-Vector ColBERT, SPLADE, and Re-rankers.2024-12-181h 33

Weaviate PodcastAgentic RAG with Erika Cardenas - Weaviate Podcast #109!Hey everyone! Thank you so much for watching the 109th episode of the Weaviate Podcast with Erika Cardenas! Erika, in collaboration with Leonie Monigatti, have recently published "What is Agentic RAG". This blog post that was even covered in VentureBeat with additional quotes from Weaviate Co-Founder and CEO Bob van Luijt! This podcast continues the discussion on all things Agentic RAG, covering the basics of Agents, how Agentic RAG changes the game compared to Vanilla RAG systems, Multi-Agent Systems and CrewAI / OpenAI Swarm, Letta, DSPy, and many more! The podcast also anchors by discussing Agentic Generative Feedback Loops and...2024-11-1334 min

Weaviate PodcastLet Me Speak Freely? with Zhi Rui Tam - Weaviate Podcast #108!JSON mode has been one of the biggest enablers for working with Large Language Models! JSON mode is even expanding into Multimodal Foundation models! But how exactly is JSON mode achieved? There are generally 3 paths to JSON mode: (1) constrained generation (such as Outlines), (2) begging the model for a JSON response in the prompt, and (3) A two stage process of generate-then-format. I am BEYOND EXCITED to publish the 108th Weaviate Podcast with Zhi Rui Tam, the lead author of Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language...2024-11-0740 min

Weaviate PodcastSWE-bench with John Yang and Carlos E. Jimenez - Weaviate Podcast #107!Hey everyone! Thank you so much for watching the 107th episode of the Weaviate Podcast! This one dives into SWE-bench, SWE-agent, and most recently SWE-bench Multimodal with John Yang from Stanford University and Carlos E. Jimenez from Princeton University! One of the most impactful applications of AI we have seen so far is in programming and software engineering! John, Carlos, and team are at the cutting-edge of developing and benchmarking these systems! I learned so much from the conversation and I really hope you find it interesting and useful as well!2024-10-3058 min

Weaviate PodcastAI in Education with Rose E. Wang - Weaviate Podcast #106!Hey everyone! I am SUPER excited to publish the 106th episode of the Weaviate Podcast featuring Rose E. Wang!! Rose is a Ph.D. student at Stanford University where she has lead incredible research at the cutting-edge of AI applications in Education. The podcast heavily discusses her recent work on Tutor CoPilot! Tutor CoPilot is one of the world's largest randomized control trials on the impact AI is having on education, testing 900 students and 1800 tutors in grades K-12. I think this is such an inspiring study and it is interesting to see the data coming in quantifying the impact...2024-10-2251 min

Weaviate PodcastCompound AI Systems with Philip Kiely - Weaviate Podcast #105!Hey everyone! Thank you so much for tuning into the 105th episode of the Weaviate Podcast! This one features Philip Kiely diving into all sorts of apsects related to Compound AI Systems! We are now seeing far better results with AI models by breaking up tasks into multiple stages and inferences. Philip explains the work they are doing at Baseten to optimize and scale deployments of these emerging systems and all sorts of aspects about them from Structured Generation to their distinction with Agents! I hope you find it useful!2024-10-1756 min

Weaviate PodcastAI Agents That Matter with Sayash Kapoor and Benedikt Stroebl - Weaviate Podcast #104!AI Researchers have overfit to maximizing state-of-the-art accuracy at the expense of the cost to run these AI systems! We need to account for cost during optimization. Even if a chatbot can produce an amazing answer, it isn't that valuable if it costs, say $5 per response! I am beyond excited to present the 104th Weaviate Podcast with Sayash Kapoor and Benedikt Stroebl from Princeton Language and Intelligence! Sayash and Benedikt are co-first authors of "AI Agents That Matter"! This is one of my favorite papers I've studied recently which introduces Pareto Optimal optimization to DSPy and really tames the...2024-09-181h 00

Weaviate PodcastAI-Native Development with Guy Podjarny and Bob van Luijt - Weaviate Podcast #102!AI is completely transforming how we build software! But how exactly? What does it mean for a software application to be AI-Native versus AI-Enabled? How many other aspects of software development and creativity are impacted by AI? I am super excited to publish our 102nd Weaviate Podcast with Guy Podjarny and Bob van Luijt on AI-Native Development! Guy Podjarny is a co-founder of Snyk, a remarkably successful Cybersecurity company. He is now back on the founder journey, diving into AI-Native Development with Tessl! Guy...2024-08-1452 min

Weaviate PodcastScaling Pandas with Devin Petersohn - Weaviate Podcast #101!Hey everyone! Thank you so much for watching the 101st episode of the Weaviate Podcast with Devin Petersohn! Devin is the creator of Modin, one of the world's most advanced systems for scaling Pandas! Devin then went onto co-found Ponder, which was acquired by Snowflake in early 2023. This was one of my favorite podcasts of all time, I learned so much about the internals of Data Systems and I hope you do as well!2024-07-1747 min

Weaviate PodcastACORN with Liana Patel and Abdel Rodriguez - Weaviate Podcast #99!Liana Patel is a Ph.D. student at Stanford University who is the lead author of ACORN, a breakthrough in Approximate Nearest Neighbor Search with Filters! Also joining the podcast is Abdel Rodriguez, a Vector Index Researcher and Engineer at Weaviate. This podcast dives into all sorts of details behind ACORN. Starting with how Liana developed her interest in Approximate Nearest Neighbor Search algorithms and then transitioning into how ACORN differs from previous approaches, the Two-Hop Neighborhood Heuristic, Predicate Subgraphs, Experimental Details, and many more topics! Major thank you to Liana and Abdel for joining the podcast, this was...2024-06-2553 min

Weaviate PodcastThe Future of Search with Nils Reimers and Erika Cardenas - Weaviate Podcast #97!Hey everyone! I am SUPER excited to publish our 97th Weaviate Podcast on the state of AI-powered Search technology featuring Nils Reimers and Erika Cardenas! Erika and I have been super excited about Cohere's latest works to advance RAG and Search and it was amazing getting to pick Nils' brain about all these topics! We began with the development of Compass! Nils explains the current problem with embeddings as a soup!! For example, imagine embedding this video description, the first part is about the launch of a podcast, whereas this part is about an...2024-06-1159 min

Weaviate PodcastDeep Learning with Letitia Parcalabescu - Weaviate Podcast #96!Hey everyone! Thank you so much for watching the 96th episode of the Weaviate podcast featuring Letitia Parcalabescu! While completing her Ph.D. studies at the University of Heidelberg, Letitia started her YouTube channel: AI Coffee Break with Letitia! Her videos break down complex concepts in AI with a creative mix of technical expertise and visualizations unlike anyone else in the space!We began the podcast by discussing our shared background in creating content on YouTube from starting, to plans for the future, and everything else in between!We then discussed the evolution of Deep Learning over the last...2024-06-051h 35

Weaviate PodcastGoogle Cloud Marketplace with Dai Vu and Bob van Luijt - Weaviate Podcast #95!Hey everyone, thank you so much for watching the 95th Weaviate Podcast! We are beyond honored to feature Dai Vu from Google on this one, alongside Weaviate Co-Founder Bob van Luijt! This podcast dives into all things Google Cloud Marketplace and the state of AI. Beginning with the proliferation of Open-Source models and how Dai sees the evolving landscape with respect to things like Gemini Pro 1.5, Gemini Nano and Gemma, as well as the integration of 3rd party model providers such as Llama 3 on Google Cloud platforms such as Vertex AI. Bob and Dai continue to unpack the next...2024-05-0741 min

Weaviate PodcastRAGKit with Kyle Davis - Weaviate Podcast #93!Hey everyone! I am SUPER excited to publish our newest Weaviate podcast with Kyle Davis, the creator of RAGKit! At a high-level, the podcast covers our understanding of RAG systems through 4 key areas: (1) Ingest / ETL, (2) Search, (3) Generate / Agents, and (4) Evaluation. Discussing these lead to all sorts of topics from Knowledge Graph RAG, to Function Calling and Tool Selection, Re-ranking, Quantization, and many more! This discussion forced me to re-think many of my previously held beliefs about the current RAG stack, particularly the definition of “Agents”. I came in believing that the best way of viewing “Agents” is an abstraction on top o...2024-04-151h 27

Weaviate PodcastVetRec with David de Matheu - Weaviate Podcast #92!I've seen a lot of interest around RAG for X application domain, Legal, Accounting, Healthcare, .... David and Kevin are maybe the best example of this I have seen so far, pivoting from Neum AI to VetRec! We begin the podcast by discussing the decision to switch gears, the advice given by Y Combinator, and David's experience in learning a new application domain. We then continue to discuss technical opportunities around RAG for Veterinarians, such as SOAP notes and Differential Diagnosis! We conclude with David's thoughts on the ETL space, companies like Unstructured and...2024-03-2859 min

Weaviate PodcastTengyu Ma on Voyage AI - Weaviate Podcast #91!Voyage AI is the newest giant in the embedding, reranking, and search model game! I am SUPER excited to publish our latest Weaviate podcast with Tengyu Ma, Co-Founder of Voyage AI and Assistant Professor at Stanford University! We began the podcast with a deep dive into everything embedding model training and contrastive learning theory. Tengyu delivered a masterclass in everything from scaling laws to multi-vector representations, neural architectures, representation collapse, data augmentation, semantic similarity, and more! I am beyond impressed with Tengyu's extensive knowledge and explanations of all these topics. The next chapter...2024-03-201h 02

Weaviate PodcastSelf-Discover DSPy with Chris Dossman - Weaviate Podcast #90!One of the core values of DSPy is the ability to add “reasoning modules” such as Chain-of-Thought to your LLM programs! For example, Chain-of-Thought describes prompting the LLM with “Let’s think step by step …”. Interestingly, this meta-prompt around asking the LLM to think this way dramatically improves performance in tasks like question answering or document summarization. Self-Discover is a meta-prompting technique that searches for the optimal thinking primitives to integrate into your program! For example, you could “Let’s think out of the box to arrive at a creative solution” or “Please explain your answer in 4 levels o...2024-03-061h 02

Weaviate PodcastMatryoshka Embeddings with Aditya Kusupati, Zach Nussbaum, and Zain Hasan - Weaviate Podcast #89!Hey everyone! Thank you so much for watching the 89th Weaviate Podcast on Matryoshka Representation Learning! I am beyond grateful to be joined by the lead author of Matryoshka Representation Learning, Aditya Kusupati, Zach Nussbaum, a Machine Learning Engineer at Nomic AI bringing these embeddings to production, and my Weaviate colleague, Zain Hasan, who has done amazing research on Matryoshka Embeddings! We think this is a super powerful development for Vector Search! This podcast covers all sorts of details from generally what Matryoshka embeddings are, the challenges of training them, experiences building an embeddings API product from Nomic AI...2024-02-201h 12

Weaviate PodcastInstructor with Jason Liu - Weaviate Podcast #88!Jason Liu is the creator of Instructor, one of the world's leading LLM frameworks, particularly focused on structured output parsing with LLMs, or as Jason puts it "making LLMs more backwards compatible". It is hard to understand the impact of Instructor, this is truly leading us to the next era of LLM programming. It was such an honor chatting with Jason, his experience currently as an independent consultant and previously engineering at StitchFix and Meta makes him truly one of the most unique guests we have featured on the Weaviate podcast! I hope you enjoy the podcast!2024-02-1455 min

Weaviate PodcastXMC.dspy with Karel D'Oosterlinck - Weaviate Podcast #87!Hey everyone! Thank you so much for watching the 87th episode of the Weaviate Podcast! I am SUPER excited to welcome Karel D'Oosterlinck! Karel is the creator of IReRa (Infer-Retrieve-Rank)! IReRa is one of the most impressive systems that have been built for Extreme Multi-Label Classification, leveraging the emerging paradigm of DSPy compilation! This podcast dives into all things IReRa, XMC, DSPy compilation, and applications in Biomedical NLP and Recommendation! I hope you find this useful!2024-02-061h 08

Weaviate PodcastDSPy and ColBERT with Omar Khattab! - Weaviate Podcast #85Hey everyone! I am beyond excited to present our interview with Omar Khattab from Stanford University! Omar is one of the world's leading scientists on AI and NLP. I highly recommend you check out Omar's remarkable list of publications linked below! This interview completely transformed my understanding of building RAG and LLM applications! I believe that DSPy will be one of the most impactful software project in LLM development because of the abstractions around *program optimization*. Here is my TLDR of this concept of LLM programs and program optimization with DSPy, I of course encourage you to view the...2024-01-1531 min

Weaviate PodcastSelf-Driving Databases with Andy Pavlo: AI-Native Databases #1Hey everyone! Thank you so much for watching the first episode of AI-Native Databases with Andy Pavlo! This was an epic one! We began by explaining the "Self-Driving Database" and all the opportunities to optimize DBs with AI and ML at both the low-level, as well as how we query and interact with them. We also discussed new opportunities with DBs + LLMs, such as bringing the data to the model (such as ROME, MEMIT, GRACE), in addition to bringing the model to the data (such as RAG). We also discuss the subjective "opinion" of these models and many more! ...2023-12-181h 14

Weaviate PodcastWeaviate 1.23 Release Podcast with Etienne Dilocker!Hey everyone! Thank you so much for watching the Weaviate 1.23 Release Podcast with Weaviate Co-Founder and CTO Etienne Dilocker! Weaviate 1.23 is a massive step forward for managing multi-tenancy with vector databases. For most RAG and Vector DB applications, you will have an uneven distribution in the # of vectors per user. Some users have 10k docs, others 10M+! Weaviate now offers a flat index with binary quantization to efficiently balance when you need an HNSW graph for the 10M doc users and when brute force is all you need for the 10k doc users! Weaviate also comes with some other "...2023-12-1455 min

Weaviate PodcastRudy Lai on Tactic Generate - Weaviate Podcast #78!Hey everyone! Thank you so much for watching the 78th episode of the Weaviate podcast featuring Rudy Lai, the founder and CEO of Tactic Generate! Tactic Generate has developed a user experience around applying LLMs in parallel to multiple documents, or even folders / collections / databases. Rudy discussed the user research that lead the company to this direction and how he sees the opportunities in building AI products with new LLM and Vector Database technologies! I hope you enjoy the podcast, as always more than happy to answer any questions or discuss any ideas you have about the content in...2023-11-2956 min

Weaviate PodcastRAGAS with Jithin James, Shahul Es, and Erika Cardenas - Weaviate Podcast #77!Hey everyone, thank you so much for watching the 77th Weaviate Podcast on RAGAS, featuring Jithin James, Shahul ES, and Erika Cardenas! RAGAS is one of the hottest rising startups in Retrieval-Augmented Generation! RAGAS began it's journey with the RAGAS score, a matrix of evaluations for generation and retrieval. Generation evaluated on Faithfulness (is the response grounded in the context) as well as Relevancy (is the response useful). Retrieval is then evaluated on Precision (How many of the search results are relevant to the question?) and Recall (How many of the relevant search results are captured in the retrieved...2023-11-2049 min

Weaviate PodcastPatrick Lewis on Retrieval-Augmented Generation - Weaviate Podcast #76!Hey everyone, I am SUPER excited to present our 76th Weaviate Podcast featuring Patrick Lewis, an NLP Research Scientist at Cohere! Patrick has had an absolutely massive impact on Natural Language Processing with AI and Deep Learning! Especially notable for the current climate in AI and Weaviate is that Patrick is the lead author of the original "Retrieval-Augmented Generation" paper!! Patrick has contributed to many other profoundly impactful papers in the space as well such as DPR, Atlas, Task-Aware Retrieval with Instruction, and many many others! This was such an illuminating conversation, here is a quick overview of the...2023-11-1458 min

Weaviate PodcastTanmay Chopra on Emissary - Weaviate Podcast #75!Hey everyone! Thank you so much for watching the 75th Weaviate Podcast featuring Tanmay Chopra! The podcast details Tanmay's incredible career in Machine Learning from Tik Tok to Neeva and now building his own startup, Emissary! Tanmay shared some amazing insights into Search AI such as how to process Temporal Queries, how to think about diversity in Retrieval, and Query Recommendation products! We then dove into the opportunity Tanmay sees in fine-tuning LLMs and knowledge distillation that motivated Tanmay to build Emissary! I thought Tanmay's analogy of GPT-4 to 3D printers was really interesting, tons of great nuggets in...2023-11-0850 min

Weaviate PodcastSimba Khadder on FeatureForm - Weaviate Podcast #74!Hey everyone! Thank you so much for watching the 74th Weaviate Podcast feature Simba Khadder, the CEO and Co-Founder of FeatureForm! To begin, "features" broadly describe the inputs to machine learning models that they use to produce outputs, or predictions. Feature stores orchestrate the construction of features, whether that be transformations for tabular machine learning models such as XGBoost, to chunking for vector embedding inference, and now features for LLM inference in RAG. Right out of the gate, Simba really opened my eyes to the role that feature engineering plays in RAG. Further touching on this at the very...2023-11-0756 min

Weaviate PodcastCharles Packer on MemGPT - Weaviate Podcast #73!Hey everyone! I am SUPER excited to publish our 73rd Weaviate Podcast with Charles Packer, the lead author of MemGPT at UC Berkeley! MemGPT presents the "Operating System for LLMs", an incredibly exciting idea to explicitly prompt the LLM with the information that it has a limited context window and give it memory management tools to behave accordingly! This was such a fun discussion with Charles diving into all things related to the paper! I hope you enjoy the podcast!! Check out MemGPT here! https://memgpt.ai/ Chapters 0:00 Welcome Charles! 0:27 LLM Operating System 4:47 Memory Management Tools 6:50 Interrupts in LLM...2023-11-0651 min

Weaviate PodcastMadelon Hulsebos on Tabular Machine Learning - Weaviate Podcast #72!Hey everyone! Thank you so much for watching the 72nd episode of the Weaviate Podcast with Madelon Hulsebos!! Madelon is one of the world's experts on Machine Learning with Tables and Tabular-Structured Data, this was such an eye-opening conversation! We discussed all sorts of topics from the relationship of tabular data and embeddings, to searching through tables, semantic joins, more complex Text-to-SQL, using machine learning for query execution, using tabular data in search and recommendation reranking, and many more! This was easily one of the most knowledge packed episodes of the Weaviate podcast so far, please don't hesitate to...2023-11-0149 min

Weaviate PodcastVibs Abhishek on Alltius AI - Weaviate Podcast #71!Hey everyone! Thank you so much for watching the 71st Weaviate Podcast with Vibs Abhishek! Vibs is the CEO and Founder of Alltius AI, as well as a professor at UC Irvine business school! In order to tame the somewhat chaotic emerging landscape of RAG and LLM applications, Alltius has settled on 3 core pillars of Knowledge, Skills, and Deployment Channels! Vibs further explained how he sees the distinction between Assistants and Agents and many more topics important to Enterprise deployment of RAG applications such as reducing hallucinations and employing classifiers to route skills and knowledge sources! I learned so...2023-10-2655 min

Weaviate PodcastKevin Cohen on Neum AI - Weaviate Podcast #70!Hey everyone! Thank you so much for watching the 70th episode of the Weaviate podcast with Neum AI CTO and Co-Founder Kevin Cohen! I first met Kevin when he was debugging an issue with his distributed node utilization and have since learned so much from him about how he sees the space of Data Ingestion, also commonly referenced as ETL for LLMs! There are so many interesting parts to this from the general flow of data connectors, chunkers and metadata extractors, embedding inference, and the last leg of the mile of importing the vectors to a Vector DB such...2023-10-1855 min

Weaviate PodcastCharles Pierse on Tactic Generate - Weaviate Podcast #69!Hey everyone! Thank you so much for watching the 69th episode of the Weaviate Podcast featuring Charles Pierse from Tactic! Tactic has recently launched their new Tactic Generate project, an incredible UI for conducting research across multiple documents. I think there is a massive opportunity to pair these prompts and LLM workflows with User Interfaces and take more of a holistic User Experience perspective. Tactic Generate has done an incredible job of that, please take a look from the link below! I had such a fun conversation catching up with Charles (Charles was our 2nd Weaviate Podcast guest!), I...2023-10-041h 08

Weaviate PodcastWeights and Biases on Fine-Tuning LLMs - Weaviate Podcast #68!Hey everyone! Thank you so much for watching the 68th episode of the Weaviate Podcast! We are super excited to welcome Morgan McGuire, Darek Kleczek, and Thomas Capelle! This was such a fun discussion beginning with generally how see the space of fine-tuning from why you would want to do it, to the available tooling, intersection with RAG and more! Check out W&B Prompts! https://wandb.ai/site/prompts Check out the W&B Tiny Llama Report! https://wandb.ai/capecape/llamac/reports/Training-Tiny-Llamas-for-Fun-and-Science--Vmlldzo1MDM2MDg0 Chapters 0:00 Tiny Llamas! 1:53 Welcome! 2:22 LLM Fine-Tuning 5:25 Tooling for Fine-Tuning 7:55 Why Fine-Tune? 9:55 RAG...2023-09-2052 min

Weaviate PodcastFarshad Farahbakhshian and Etienne Dilocker on Weaviate and AWS - Weaviate Podcast #67!Hey everyone! Thank you so much for watching the 67th Weaviate Podcast, announcing Weaviate on the AWS Marketplace! This was one of my favorite podcasts to date with a deep dive on the details of running RAG applications in the cloud, our general understanding of LLM Fine-Tuning and RAG, as well as a really interesting discussion on VPCs and Hybrid SaaS! I hope you find the podcast useful, as always we are more than happy to answer any questions or discuss any ideas you have about the content presented in the podcast! Learn more here: https://aws.amazon.com...2023-09-131h 01

Weaviate PodcastHybrid SaaS in Weaviate Explained!Hey everyone! Here is a clip from our newest Weaviate podcast with Farshad Farahbakhshian, Gen AI specialist at AWS and Etienne Dilocker, CTO and Co-Founder of Weaviate! This podcast announces Weaviate on the AWS marketplace and is packed with info on running Weaviate in the cloud such as this clip explaining how Hybrid SaaS works! I hope you find the clip useful, we are more than happy to answer any questions you have about the content in this clip! Chapters 0:00 Quick Intro for Context 0:29 Etienne Dilocker on Hybrid SaaS 2023-09-1204 min

Weaviate PodcastDavid Garnitz on VectorFlow - Weaviate Podcast #66!Hey everyone! Thank you so much for watching the 66th Weaviate Podcast with David Garnitz, the creator of VectorFlow! VectorFlow (open-sourced on GH and linked below) is a new tool for ingesting data into Vector Databases such as Weaviate! There is quite an interesting End-to-End stack emerging at the ingestion layer, from retrieving data from misc. sources such as Slack, Salesforce, GitHub, Google Drive, Notion, ... to then Chunking the Text (maybe with the use of Visual Document Layout parsers like what Unstructured is imagining), extracting Metadata potentially (say the "age" of an NBA player as in the Evaporate-Code+ research...2023-09-071h 04

Weaviate PodcastOfir Press on AliBi and Self-Ask - Weaviate Podcast #65!Hey everyone! Thank you so much for watching the Weaviate Podcast! I am SUPER excited to publish my conversation with Ofir Press! Ofir has done incredible work pioneering AliBi attention and Self-Ask prompting and I learned so much from speaking with him! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! +Huge Congratulations on your Ph.D. Ofir! AliBi Attention: https://arxiv.org/abs/2108.12409 Self-Ask Prompting: https://arxiv.org/abs/2210.03350 Ofir Pres on YouTube: https://www.youtube.com/@ofirpress Chapters 0:00 Welcome Ofir Press 0:41 Large Context...2023-08-311h 07

Weaviate PodcastShishir Patil and Tianjun Zhang on Gorilla - Weaviate Podcast #64!Hey everyone! Thank you so much for watching the 64th Weaviate Podcast with Shishir Patil and Tianjun Zhang, co-authors of Gorilla: Large Language Models Connected with Massive APIs! I learned so much about Gorilla from Shishir and Tianjun, from the APIBench dataset to the continually evolving APIZoo, how the models are trained with Retrieval-Aware Training, Self-Instruct Training data and how the authors think of fine-tuning LLaMA-7B models for tasks such as this, and many more! I hope you enjoy the podcast! As always I am more than happy to answer any questions or discuss any ideas you have...2023-08-3049 min

Weaviate PodcastNils Reimers on Cohere Search AI - Weaviate Podcast #63!Hey everyone! Thank you so much for watching the 63rd Weaviate Podcast, I couldn't be more excited to welcome Nils Reimers back to the podcast!! Similar to our debut episode together, we began by describing the latest collaboration of Weaviate and Cohere (episode 1, new multilingual embedding models; episode 2, rerankers!), and then continued into some of the key questions around search technology. In this one, we discussed the importance of temporal queries and metadata extraction, long document representation, and future directions for Retrieval-Augmented Generation! I hope you enjoy the podcast, as always I am more than happy to answer any...2023-08-171h 05

Weaviate PodcastAtai Barkai on PodcastGPT - Weaviate Podcast #62!Hey everyone! Thank you so much for watching the 62nd Weaviate Podcast with Atai Barkai! We are stepping into the meta with this one for a podcast about podcasts! Podcasts are one of the biggest opportunities of new technologies, starting with Whisper's ability to transcribe audio to text and advances with speaker diarization, .. the question to be explored is, What Vector Database and LLM applications can we build with this data?! What is the future of podcasting with these new technologies?! I had so much fun discussing all these ideas with Atai! As always we are more than happy...2023-08-0955 min

Weaviate PodcastRohit Agarwal on Portkey - Weaviate Podcast #61!Hey everyone! Thank you so much for watching the 61st episode of the Weaviate Podcast! I am beyond excited to publish this one! I first met Rohit at the Cal Hacks event hosted by UC Berkeley where we had a debate about the impact of Semantic Caching! Rohit taught me a ton about the topic and I think it's going to be one of the most impactful early applications of Generative Feedback Loops! Rohit is building Portkey, a SUPER interesting LLM middleware that does things like load balancing between LLM APIs, and as discussed in the podcast there are...2023-08-0349 min

Weaviate PodcastPatrice Bourgougnon on WPSolr - Weaviate Podcast #60Hey everyone! Thank you so much for watching the 60th Weaviate podcast with Patrice Bourgougnon! Patrice is the creator of WPSolr, integrating AI search capabilities with Wordpress and Woocommerce. Patrice is one of the most active contributors to Weaviate, filing issues and poking holes in new releases! Patrice shared incredible feedback on Weaviate and how he sees the state of Vector Databases and Search! As always, we are more than happy to answer any questions or ideas you have about the content discussed in the podcast! Thanks for watching! Chapters 0:00 Introduction 0:45 Vector Databases and Wordpress 4:50 Weaviate Client Languages 10:00 Inference...2023-08-021h 25

Weaviate PodcastAndriy Mulyar on Nomic AI, Atlas, and GPT4All - Weaviate Podcast #58Hey everyone! Thank you so much for watching the 58th episode of the Weaviate Podcast! I am SUPER excited to welcome Andriy Muylar! Andriy is the Co-Founder of Nomic AI, a company fresh off a $17M series A raise! Nomic has created some incredible products such as Atlas and GPT4All! I was really impressed by Andriy's vision of the state and forecasted evolution of these topics! I hope you enjoy the podcast! As always, we are more than happy to answer any questions or discuss any ideas you have about the content discussed in the podcast! Integration Tutorial...2023-07-1858 min

Weaviate PodcastCharles Frye on Full Stack Deep Learning - Weaviate Podcast #57!Hey everyone! Thank you so much for watching the 57th Weaviate podcast with Charles Frye! Charles is an educator at Full Stack Deep Learning, one of the world's top courses on Deep Learning with lectures available on YouTube (link below)! This was one of the most thorough Weaviate podcasts published so far, covering all sorts of topics around the evolution of Deep Learning! Particularly we discussed the Retrieval-Augmented Generation stack with Vector Databases and Zero-Shot Large Language Models and how that compares to more conventional machine learning workflows and the MLOPs stack! I really enjoyed chatting with Charles and...2023-07-131h 39

Weaviate PodcastEtienne Dilocker on Weaviate 1.20 - Weaviate Podcast #56!Chapters 0:00 Weaviate 1.20!!! 0:40 Multi-Tenancy 35:36 PQ Rescoring 47:20 Re-Ranking, AutoCut, Rank Fusion 58:58 Cloud Monitoring Metrics 2023-07-121h 02

Weaviate PodcastAleksa Gordcic - Weaviate Podcast #55!Hey everyone! Thank you so much for watching the 55th episode of the Weaviate Podcast with Aleksa Gordcic! This episodes dives into Aleksa's incredible story from Deep Learning YouTube to DeepMind and now creating Ortus! We dived into all sorts of topics, I loved hearing about the latest updates on Ortus and how Aleksa is sees the current state of AI development! We are more than happy to answer any questions or discuss any ideas you might have about the content in the podcast! Thanks so much for watching! Check out Ortus here! - https://www.ortusbuddy.ai/welcome ...2023-07-051h 07

Weaviate PodcastYana Welinder on Kraftful - Weaviate Podcast #52!Hey everyone, thank you so much for watching the 52nd episode of the Weaviate Podcast with Yana Welinder! Yana is the Founder and CEO of Kratful (https://www.kraftful.com/). Kratful is an incredibly interesting "ChatGPT but for Product Research" -- curating specific skills for Product Managers into a collection of prompts. We discussed all sorts of things from the latest innovations in LLMs to the ChatGPT marketplace and product management, I really hope you enjoy the podcast!2023-06-1441 min

Weaviate PodcastGreg Kamradt and Colin Harmon on LLM Agents - Weaviate Podcast #51Hey everyone, thank you so much for watching the 51st episode of the Weaviate Podcast with Greg Kamradt and Colin Harmon! Greg and Colin are both entrepreneurs in the space of new AI tools powered by LLMs! This podcast is about keeping up with the evolution of LLM Agents from AutoGPT to connecting LLMs with Vector Databases or Wolfram Alpha, as well as the ChatGPT Marketplace, Personalized LLMs, Private LLMs, and many more! I think there are so many interesting nuggets from this podcast, thank you so much to Greg and Colin for joining, really enjoyed this one! Data...2023-06-0754 min

Weaviate PodcastRetrieving Texts based on Abstract Descriptions Explained!This video explores a new paper exploring the use of summarization chains to represent long texts and use (original text, summary) pairs for optimizing text embeddings models! Here are 3 main takeaways I think everyone working with Weaviate may get value from: 1. Understanding of Summary Indexing and the Prompts (as well as Prompt Chains) used to build them. 2. Continued development of LLM-generated data for search -- creating (full text, summary) pairs gives you (1) data to build a summary index with as mentioned, (2) data to compare different embedding models with, and (3) data to train your own embedding model. 3. Tournament style evaluation...2023-06-0228 min

Weaviate PodcastKapa AI with Emil Sorensen and Finn Bauer - Weaviate Podcast #50!Hey everyone, thank you so much for watching the 50th (!!!) Weaviate Podcast with Emil Sorensen and Finn Bauer from Kapa AI! Are you curious about taking either your, or your company's, specific information and putting into a Vector DB + LLM system? Emil and Finn are doing this at the highest level, taking the documentation of software companies like Weaviate and building these LLM-augmetnted assistant systems for them. This podcast takes a complete tour from Data Ingestion to Cleaning, Chunking, LLM latency, and emerging trends in LLMs such as cheap fine-tuning with LoRA or Long Context Windows such as GPT-4 32...2023-05-3135 min

Weaviate PodcastNeurosymbolic AI in Search with Professor Laura Dietz - Weaviate Podcast #49!Hey everyone, thank you so much for watching the 49th episode of the Weaviate Podcast!! This podcast features Professor Laura Dietz from the University of New Hampshire! I came across Dr. Dietz's tutorial at ECIR on Neuro-Symbolic Approaches for Information Retrieval and am so grateful that she was interested in joining the Weaviate Podcast! I learned so much about Neurosymbolic Search, especially around the role of Entity Linking and Entity Re-Ranking -- as well as the topic of Knowledge Graphs and Vector Search. We also discussed Prof. Dietz and collaborators latest perspectives paper on Large Language Models for Relevance...2023-05-251h 30

Weaviate PodcastUnstructured with Brian Raymond - Weaviate Podcast #48!Hey everyone, thank you so much for watching the 48th episode of the Weaviate Podcast!! This is a SUPER exciting one, welcoming Brian Raymond the CEO / Founder of Unstructured! Unstructured is a perfect complimenting technology for Weaviate, helping people get their Unstructured data into Weaviate! The podcast dives into the nuances of this task, but it generally revolves around Unstructured's abstraction of Partitioning, Cleaning, and Staging! Unstructured is making groundbreaking innovations on using Visual Document Layout models for Partitioning, for example saying that this part of the PDF is the header, body, image caption, and so on. Cleaning then...2023-05-2343 min

Weaviate PodcastChatArena with Yuxiang Wu - Weaviate Podcast #47!Hey everyone, thank you so much for watching the Weaviate podcast! I am so excited about this episode! ChatArena is a software framework for multi-agent chat games. There are quite a few interesting applications of this, firstly we can use this kind of system to evaluate the intelligence of an LLM based on how intelligent it sounds in conversation with another LLM! Another interesting idea is to have the LLM impersonate people such as Lex Fridman or Sam Altman and simulate conversations between these people -- retrieving from their digital content to facilitate the impersonation. I thought there was...2023-05-1751 min

Weaviate PodcastHyperDB with John Dagdelen, Bob van Luijt, and Etienne Dilocker - Weaviate Podcast #46!Hey everyone! Thank you so much for watching the Weaviate Podcast! This is pretty novel episode featuring both Weaviate Co-Founders Bob van Luijt and Etienne Dilocker! This is also extremely novel because we are featuring a competitor vector database, HyperDB! John Dagdelen is the founder of HyperDB which is a hyper-fast local vector database for use with LLM Agents. Now accepting SAFEs at $135M cap. HyperDB: https://github.com/jdagdelen/hyperDB More seriously, John has produced an incredible body of research - https://scholar.google.com/citations?user=TiCS5FEAAAAJ&hl=en&oi=ao. John's work on Scientific Literature...2023-05-101h 06

Weaviate PodcastWeaviate 1.19 Release with Etienne Dilocker - Weaviate Podcast #44!Hey everyone! Thank you so much for watching the Weaviate 1.19 release podcast! We have all sorts of cool new features, in addition to the database and module features, I really want to encourage readers to see the `groupBy` search discussed at 14:32, quite an interesting idea for improving search performance! Chapters 0:00 Welcome Etienne! 0:38 gRPC API 9:50 Generative Cohere 14:32 groupBy search 19:33 Bitmap or BM25 index tuning 22:20 Additional Tokenization Options 24:05 Tunable Consistency 2023-05-0426 min

Weaviate PodcastErika Cardenas, Roman Grebennikov, and Vsevolod Goloviznin on Recommendation and Metarank - Pod #43!Thank you so much for watching the 43rd episode of the Weaviate Podcast with Roman Grebennikov and Vesvolod Goloviznin from Metarank, as well as Erika Cardenas from Weaviate! This podcast is a masterclass on Ranking models, additionally touching on the connection between Search and Recommendation. Learning-to-rank is an exciting idea where we use models that produce more fine-grained relevance scores than the offline indexing techniques of vector search and bm25, however with the tradeoff of the speed of these inferences. Romand and Vsevolod touched on another extremely interesting part of these ranking models which is the estimation of features...2023-04-121h 00

Weaviate PodcastEthan Steininger on Mixpeek and the AI Landscape - Weaviate Podcast #42!Thank you so much for watching the 42nd episode of the Weaviate Podcast! Ethan Steininger is the founder of Mixpeek, an intelligence layer that sits on top of your S3 bucket, so you can search and analyze unstructured data at scale. Ethan has also created Collie with the headline of "Enter your website and Collie will fetch every asset, then give you an embedded search bar that wows your users". Ethan began the podcast by describing his background at MongoDB and integrating the database with full text search functionality. Ethan then presented the founding vision of Mixpeek and some...2023-04-051h 22

Weaviate PodcastWeaviate 1.18 Release Podcast - Weaviate Podcast #40!Chapters 0:00 Weaviate 1.18!!! 0:32 Bitmap Indexing! 11:40 HNSW PQ 25:33 Cursor API 30:03 Filters in Hybrid Search 32:55 WAND Scoring 40:35 Replication 49:10 Building a Database in Golang 1:00:55 Thank you!2023-03-071h 02

Weaviate PodcastLeo Boystov on Information Retrieval Science - Weaviate Podcast #38Hey everyone! Thank you so much for watching the 38th episode of the Weaviate podcast! This episode features Leo Boystov, an expert in Information Retrieval technology! We discussed a very wide range of topics from an overview of IR methods such as BM25, Neural Bi-Encoder and Cross-Encoder rankers, and a super exciting new work Leo has co-authored on using Large Language Models to generate training data for Neural Ranking models titled "InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers." We also discussed Leo's work on Non-Metric Space Search, the challenge of long document ranking, Robustness in Generalization Testing, and ended...2023-03-011h 28

Weaviate PodcastGPT Index and Weaviate with Jerry Liu and Bob van Luijt - Weaviate Podcast #37Hey everyone! Thank you so much for watching the 37th episode of the Weaviate podcast! This episode discusses some of the ideas behind GPT Index. GPT Index presents really exciting ideas about how we use LLMs to index our data and then traverse these data structures. We began the podcast by discussing the origins of the tool and the ideas behind the Tree Index. We then discussed generalizing these trees to graphs and whether we are headed to the Knowledge Graph 2.0. Another really interesting topic we covered is the inference cost of building and traversing LLM indices like this...2023-02-2251 min

Weaviate PodcastLangChain and Weaviate with Harrison Chase and Bob van Luijt - Weaviate Podcast #36Hey everyone! Thank you so much for watching the 36th episode of the Weaviate podcast! This episode continues on the marriage between LLMs and Semantic Search, welcoming back Weaviate CEO and Co-Founder Bob van Luijt! Enter LangChain and its creator, Harrison Chase, providing the glue between LLMs and tools, such as semantic search. LangChain provides a set of abstractions around chaining multiple language model calls with different prompts, strategies for overcoming the 4096 token limit, and connecting LLMs with their tools. LangChain Hub is a collection of these chains if you want to check it out yourself! Huge thank you...2023-02-1547 min

Weaviate PodcastBob van Luijt on Generative Search with Weaviate - Weaviate Podcast #35This podcast debuts a huge new release from Weaviate... the generate module! The generate module is a new API in Weaviate that facilitates passing YOUR data from the Weaviate database to ChatGPT. This enables ChatGPT to become knowledgeable about your particular business or interests! Here is a great snippet from Bob around the 43 minute mark that describes how this kind of LLM technology is changing the world of database technology, "Yeah so, what I’m really excited about and this is something that it’s just so funny right because if you see it, you have this huge epiphany. I’ve...2023-02-0743 min

Weaviate PodcastDmitry Kan on Neural Search Frameworks - Weaviate Podcast #34I am so excited to host Dmitry Kan on the Weaviate Podcast!! Dmitry is a world class expert on emerging trends in search technology! This podcast reflects on Dmitry's latest characterization of the field, the Neural Search Pyramid. This describes the different components involved with building a Deep Learning-powered Search experience from the Approximate Nearest Neighbor index algorithms, to Database functionality, LLM orchestration, Vectorization optimization, Data preprocessing, User Interface, and many more! We also concluded the podcast with an interesting debate around renaming "Vector Search" to something else that reaches a broader audience. I really hope you enjoy the p...2023-01-251h 47

Weaviate PodcastNils Reimers on Cohere Embedding ModelsWeaviate podcast #33. Thank you so much for watching the 33rd Weaviate Podcast! This episode features one of the heroes of Deep Learning for Search, Nils Reimers! Nils' work on SentenceBERT is one of the foundational works for applying Deep Representation Learning to text search. This is the idea that personally inspired me to work in this field. Having seen the successes of Contrastive Representation Learning for Computer Vision, I was mind-blown by the possibility of this for NLP and text search. In addition to the scientific foundation, the software development of the Sentence Transformers library and BEIR...2023-01-1155 min

Weaviate PodcastSam Bean, Zain Hasan, and John Trengrove on You.com and SparkWeaviate Podcast #32. Thank you so much for watching the Weaviate podcast! We are super excited to host Sam Bean from You.com! As well as welcome Zain Hasan and John Trengrove to the Weaviate podcast for the first time! Sam begins by describing You.com, and then we dive into the Weaviate Spark Connector that Sam played a massive role in creating. I thought this was such a masterclass in the Spark big data technology; John, Sam, and Zain are all data engineering pros and I've never learned more about a new technology from a podcast...2023-01-0951 min

Weaviate PodcastWeaviate 1.17 Release with Etienne Dilocker and Parker DuckworthWeaviate Podcast #31. Weaviate 1.17!! This is a massive release for Weaviate, debuting Replication, Hybrid Search, BM25, Faster Startup and Import Times, as well as other fixes! Replication and Hybrid Search are two massive features for Weaviate, and we really hope you enjoy the description of them from the podcast. Please also check out the Weaviate 1.17 release blog post for more information as well - https://weaviate.io/blog/2022/12/Weaviate-release-1-17.html! This is also a very special podcast as we welcome Parker Duckworth for the first time to the podcast! Parker gave an excellent explanation of Re...2022-12-2142 min

Weaviate PodcastBob van Luijt, Chris Dossman and Marco Bianco on the future of searchWeaviate Podcast #30. Chapters 0:00 The future of search! 0:42 Welcome Marco and Chris! 4:28 Solving Hallucination with External Memory LLMs 8:16 Bob van Luijt on Weaviate and LLMs, Collaborations 14:48 What we have is not yet what the technology is capable of 16:45 Everything is Search! 18:55 The Magic of Machine Learning 20:30 Asking follow up questions 22:28 Meaning in LLMs and RLHF 27:10 How ChatGPT is Evangelizing the Technology 29:45 What is the future of search from a user perspective? 34:38 Integration with Existing Businesses 35:20 Impact on Creativity 37:37 Data Visualization from Natural Language Q...2022-12-141h 05

Weaviate PodcastMatthijs Douze on Quantization and FAISSWeaviate Podcast #29. Hey everyone, thank you so much for watching another episode of the Weaviate podcast! This episode features Matthijs Douze, one of the most talented and accomplished scientists we've hosted on the Weaviate podcast! Matthijs has pioneered the use of Product Quantization to compress vector representations and enable even faster and more efficient approximate nearest neighbor vector search. Matthijs told an incredible story about the history of this research, from searching from SIFT vectors for Computer Vision Search applications like real-time CD Cover album search to the problems facing modern IVF-PQ systems and the use of PQ in...2022-11-301h 12

Weaviate PodcastMaarten Grootendorst on BERTopicWeaviate Podcast #28. Thank you so much for watching the 28th Weaviate Podcast! This episode features Maarten Grootendorst, developer of the BERTopic python library and an active evangelist of this exciting cluster analysis technology, (Maarten has written some incredible articles here - https://medium.com/@maartengrootendorst)! In this podcast, Maarten did an incredible job explaining how BERTopic works, with particular details such as k-Means clustering vs. HDBSCAN, Semi-Supervised topic modeling, Dynamic topic modeling, and many more! I was amazed at Maarten's expertise in the miscellaneous details of these algorithms! We are extremely excited about adding BERTopic to Weaviate, please see...2022-11-1753 min

Weaviate PodcastMichael Goin on Neural MagicWeaviate Podcast #27. Thank you so much for watching the 27th episode of the Weaviate Podcast! This is truly one of my favorite podcasts we have published so far, I think the way Weaviate and Neural Magic fit together is really exciting! Michael did an amazing job explaining the concepts behind how Neural Magic delivers and tests inference acceleration, as well as the vision for the future of Deep Learning with Sparsity and CPU inference. I really hope you enjoy the podcast, more than happy to answer any questions or entertain any ideas/discussion! Thanks again for watching! Weaviate users...2022-10-2644 min

Weaviate PodcastJonathan Frankle on MosaicML CloudWeaviate Podcast #26. Thank you so much for watching the 26th episode of the Weaviate Podcast! This is another really special episode! Jonathan Frankle is one of the world's experts in Deep Learning and is making incredible advances at MosaicML in efficient Deep Learning training. The headline event is the release of MosaicML Cloud and a set of new cost estimates for GPT language models at different scales (linked below). Jonathan explains that these numbers are a baseline and he predicts they could get to as low as $100K as they seek opportunities for efficiency optimizations. This story has already...2022-10-1944 min

Weaviate PodcastErik Bernhardsson and Etienne Dilocker on Vector Search in Production.Weaviate Podcast #25. Thank you so much for watching the 25th episode of the Weaviate Podcast! This is a really special episode with Erik Bernhardsson! Erik is one of the early thought leaders on Approximate Nearest Neighbor (ANN) Search, creating the ANNOY library at Spotify. Erik shared incredible insights about vector search at Spotify such as the role of Offline and Online Machine Learning inference and the role of multi-stage re-ranking pipelines. Erik has also done massively impactful work on benchmarking ANN algorithms! We really hope you enjoy the podcast and would be thrilled to answer any questions you have...2022-10-0644 min

Weaviate PodcastWeaviate v1.15 Release with Etienne Dilocker and Dirk KulawiakWeaviate Podcast #24. Weaviate v1.15 Release! Thank you so much for checking out the Weaviate podcast -- here is a summary of what is new in Weaviate 1.15: 1. Cloud-native backups – allows you to configure your environment to create backups – of selected classes or the whole database – straight into AWS S3, GCS or local filesystem 2. Reduced memory usage - we found new ways to optimize memory usage, reducing RAM usage by 10-30%. 3. Better control over Garbage Collector – with the introduction of GOMEMLIMIT we gained more control over the garbage collector, which significantly reduced the chances of OOM kills for your Weaviate setups. 4. Faster im...2022-09-081h 06

Weaviate PodcastOri Ram on Learning to Retrieve Passages without SupervisionWeaviate Podcast #23. Thank you so much for watching the 23rd episode of the Weaviate Podcast! This episode dives into a new technique for Self-Supervised retrieval in NLP with some incredible results!2022-08-311h 02

Weaviate PodcastYaoshiang Ho on Masterful AIWeaviate Podcast #22. Thank you so much for watching the 22nd Weaviate Podcast with Yaoshiang Ho! Yaoshiang is a Co-Founder of Masterful AI, a company doing incredible work in the Computer Vision model training and deployment space (https://www.masterfulai.com/). I really hope you enjoy this podcast, Yaoshiang and I went deep into some of the cutting edge Computer Vision algorithms such as Noisy Student, SimCLR, and Barlow Twins -- as well as the broader topic of Semi-Supervised Learning in which we have a small labeled dataset and a large unlabelled dataset. I am so excited about model training...2022-08-1057 min

Weaviate PodcastLaura Ham on Weaviate User ExperienceWeaviate Podcast #21. Thank you for watching the 21st Weaviate Podcast with Laura Ham! Laura Ham has worked on Weaviate at SeMI Technologies for a little over 5 years. She has had a heavy influence on all things from the GraphQL User Experience design to the Graph data model, and the creation of educational content! I really enjoyed this podcast, please see the list of topics under “chapters”! Here are some examples of recent coding tutorial videos Laura has made on “How to add custom modules to Weaviate” and integrations of Weaviate with Jina AI and Haystack.2022-07-2752 min

Weaviate PodcastTuana Celik on Question Answering with HaystackWeaviate Podcast #20. Tuana Celik, a Developer Advocate at Deepset, presented many exciting ideas around Question Answering! We began with her Game of Thrones Question Answering Demo on HuggingFace Spaces and continued to discuss all topics QA from Extractive to Abstractive, benefits of Retrieve-then-Read, and Zero-Shot Generalization, to give a quick preview. For our Weaviate users, please check out this demo from Laura Ham on how to use Haystack QA in tandem with the Weaviate Vector Search Database: https://www.youtube.com/watch?v=Bkoza.... I really hope you enjoy this podcast, please don't forget to check the Chapters to...2022-07-1346 min

Weaviate PodcastEtienne Dilocker on Weaviate v1.14 Release!Weaviate Podcast #19. SeMI Technologies Co-Founder and CTO Etienne Dilocker returns to the Weaviate podcast to describe what's new with Weaviate v1.14! Please see the chapter outlines if you would like to skip ahead to the update most relevant to you! Please also see this blog post lead by Sebastian Witalec describing the new changes to Weaviate! Weaviate v1.14 Blog Post: https://weaviate.io/blog/2022/07/Weav...2022-07-0847 min

Weaviate PodcastVincent D. Warmerdam on Applications of Nearest Neighbor SearchWeaviate Podcast #18. Thank you for watching the 18th Weaviate Podcast with Vincent D. Warmerdam! Vincent is an engineer at Spacy working on exciting tools such as Prodigy! Vincent describes how nearest neighbor search can aid in tasks such as Data De-Duplication and Data Labeling! Vincent shared many interesting ideas from representations of text, challenges with annotator disagreement, lessons from hosting data labeling workshops to train data scientists, and many more!2022-06-2857 min

Weaviate PodcastUNC research team on VL Adapter for Efficient CLIP TransferWeaviate Podcast #14. Thanks for watching the Weaviate podcast! Our 14th episode welcomes Yi-Lin Sung, Jaemin Cho, and Professor Mohit Bansal, a research team from UNC! Our guests present their work on VL Adapter, a technique to achieve full fine-tuning performance while only updating 4% of original parameters!! This is an incredibly interesting finding for the sake of cost-effective tuning of Vision and Language models based on CLIP. We additionally discussed topics around compression bottlenecks in neural architectures, V&L datasets, and the tricky question of compositional generalization. If you are curious about using CLIP in Weaviate, please check out this...2022-04-2628 min

Weaviate PodcastYury Malkov and Etienne Dilocker about HNSW in Vector Search and WeaviateWeaviate Podcast #10. A guided conversation about HNSW by Connor Shorten between Yury Malkov, Staff ML Engineer at Twitter and the co-inventor of HNSW, and Etienne Dilocker, the co-founder of Weaviate. Check the timestamps below!2022-03-101h 00

Weaviate PodcastArvind Neelakantan of OpenAI • Embeddings API in WeaviateWeaviate Podcast #7. Arvind Neelakantan, Research Lead at Open AI, talks with Connor Shorten about their newly released embeddings API, his work at Open AI, the integration into Weaviate, and more.2022-02-1147 min

Weaviate PodcastHow Zencastr Searches through their Podcast Transcriptions with WeaviateWeaviate Podcast #6. Alex Cannan, a Machine Learning engineer at Zencastr, talks with Connor Shorten about a really exciting use case of applying search to look through podcast transcription. Topics discussed are the need for fine-tuning, building your own vector database versus Weaviate, data privacy for Deep Learning applications, and many more!2022-02-0255 min

Weaviate PodcastHow The Knowledge Management Bot Katie leverages WeaviateWeaviate Podcast #5. Katie is a knowledge management bot, continuously improving, self-learning, and trained by humans. Under the hood, Katie is powered by the Weaviate vector search engine, during this podcast, Katie's Michael Wechner will talk about all things vector search and more!2022-01-211h 25

Weaviate PodcastOn Deepset's Haystack and how they leverage The Weaviate Vector Search EngineWeaviate Podcast #4. NLP frameworks like Deepset's Haystack are powerful tools to help data scientists and software engineers work with the latest and greatest in natural language processing. In this interview, Malte Pietsch will be talking about Haystack and how they leverage the Weaviate vector search engine as a persistent storage engine for their data and vector representations.2022-01-1157 min

Weaviate PodcastA Vision for The Future of Vector SearchWeaviate Podcast #3. Join ‪Connor Shorten and Bob van Luijt (SeMI Technologies) for the third Weaviate vector search engine Podcast. During the show, they will be discussing use cases, the GraphQL API, knowledge graphs, Weaviate as a product, vector search engine use cases, and a vision for the future of vector search.2021-12-201h 00

Weaviate PodcastHow Keenious uses Weaviate to Enable Semantic Search through 60M+ Academic PUBsWeaviate Podcast #2. Join ‪Connor Shorten (Henry AI Labs) and Charles Pierse (Keenious) for the second Weaviate vector search engine Podcast. During the show, they will be discussing how Keenious uses Weaviate and broader, all things NLP!2021-12-131h 23

Weaviate PodcastCommunity and Weaviate Core UpdateWeaviate Podcast #1. Join ‪Connor Shorten and Etienne Dilocker (SeMI Technologies) for the first Weaviate Podcast. During the show, they will be discussing Weaviate's horizontal scalability features in the v1.8.0 release and a wide variety of topics surrounding the Weaviate Slack channel.2021-12-0546 min