Vector databases are crucial for AI-driven applications, but selecting the best one can be tricky. This article breaks down key performance metrics such as recall, nDCG, and QPS, focusing on the HNSW algorithm and VectorDBBench for benchmarking. It also features Docker Compose setups for Milvus, Redis, Chroma, and PgVector to ensure fair and consistent performance testing.
As a small but mighty agency, we pride ourselves on our ability to innovate and adapt in the always evolving world of Software Development and Design. At Pixion, our secret weapon for constantly improving our workflow is teamwork and as a Product Designer, one of my favorite methods for working closely with our Engineers to create beautiful and reliable products is the Hot Potato Process. If you haven't heard of it before, let me introduce you to the approach that helps keep our team ahead of the game.
Optimize your LLM prompts for Retrieval-Augmented Generation (RAG) with the JSON-based, and few-shot techniques. The article explores strategies for reducing hallucinations, improving context alignment, and addressing formatting errors. Extensive testing demonstrates the benefits of structuring prompts with clear instructions, separating context chunks, and leveraging in-context learning examples.
Explore the RAG evaluation on a large scale to assess the effectiveness of different strategies. This article analyzes answer quality across various configurations, addressing challenges like answer coverage, comparison methods, and rate limits. It also provides insights into the cost and resource impact of running the evaluation pipeline.
Discover how various parameters affect performance in Retrieval-Augmented Generation (RAG). This article explores factors like chunk size, overlap, and context size, examining their impact on the large language model's ability to answer questions. It also delves into error analysis and the cost breakdown of running the full pipeline, including token usage and pricing considerations.
Explore the embedding process in Retrieval-Augmented Generation (RAG) strategies. This article provides insights into the chunking process, LLM-generated summaries and hypothetical questions, offering valuable analysis on embedding configurations and their impact on answer generation.
Dive into the practical process of synthetic test set generation with Ragas. This article showcases how AI models are applied to create diverse question types using U.S. Code corpus of data, highlighting real-world testing and cost analysis in RAG applications.
This short interview sums up my path from a summer JavaScript School participant to a full time Pixion software engineer explaining how and why that happened.
Are you interested in building a Retrieval-Augmented Generation (RAG) application? In previous articles, we introduced fundamental concepts such as retrieval strategies, database choices, vector search indices, and the Ragas evaluation framework. Now let's tie all this together into a comprehensive solution.
Exploring the integration of LangSmith, a monitoring platform for AI language systems, with Ragas, a framework for synthetic test set generation and RAG evaluation. LangSmith's detailed tracing capabilities are vital for tracking the dynamic state of a system in production, ensuring seamless performance and insightful analysis.
Ever wondered how a manual QA tester, with no prior experience or knowledge in programming, can ace automated tests? This blog covers the story how one becomes proficient in test automation.
Providing a comprehensive guide to RAG evaluation with Ragas, this article delves into metrics like faithfulness, context precision, and answer correctness. It emphasizes the importance of prompt engineering, classification, and feature extraction to enhance the accuracy and reliability of RAG systems.
Delving into RAG application testing with Ragas, this article explores manual vs. automated test set creation using LLMs, focusing on prompt design and the generation process for accurate and efficient testing.
How a background in translation can unexpectedly pave the way for a career in QA, demonstrating the potential of transferable skills and diverse career paths.
Explore advanced RAG testing with a comprehensive framework. From simple setup to synthetic test data generation, elevate your RAG pipeline evaluation with powerful tools and insights.
Explore the vital step of evaluating Retrieval-Augmented Generation (RAG) applications. Learn key metrics like precision, recall, faithfulness, and relevance, essential for crafting accurate and reliable AI-generated responses.
Are you navigating the complex world of vector databases in Postgres? Explore PGVector and its indexes: Flat, HNSW, and IVFFlat. Dive into the nuances of accuracy, speed, and scalability to make informed decisions.
Discovered by the sudden hype around retrieval-augmented generation (RAG), vector databases quickly gained traction. The article provides a guiding hand for navigating this new landscape.
Exploring Hypothetical Questions and HyDE strategies, designed to boost the accuracy of LLM-based applications by refining the match between user queries and data. It examines the generation of hypothetical questions and answers emphasizing the contrast in their approach.
Looking into the scalability problem of the RAG, the article explores the idea of Hierarchical Index Retrieval. A retrieval strategy that aims to deal with the problem through progressive narrowing of the search space.
Continuing on the RAG strategies series, the article explores how to use the chunk size to improve the retrieval process performance. It presents two popular strategies: Sentence Window Retrieval and Auto-Merging Retrieval.
Delving into the essentials of Retrieval-Augmented Generation (RAG) and Basic Index Retrieval, showcasing how they complement Language Large Models by enriching them with targeted, domain-specific knowledge for more nuanced, human-like responses.
Exploring the fusion of large language models with external data retrieval to transform AI applications. The series offers a deep dive into practical strategies and insights for implementing this innovative approach.
Short intro to the custom clock process on Heroku for greater control and scheduling visibility.
What can QA Engineer do to feel like a part of a team, and not as if they are working against everyone else when reporting issues?
