Beyond Relational: Vector Search and Semantic Queries

Introduction: Unlocking Semantic Understanding

Welcome back, intrepid data explorer! In our journey with Stoolap, we’ve seen how it masterfully handles traditional relational data with high performance, concurrency, and robust transactions. But the world of data is evolving, moving beyond simple keyword matching and exact joins. We’re entering an era where applications need to understand the meaning behind data. This is where vector search and semantic queries come into play, and Stoolap is perfectly positioned to deliver these capabilities right within your application.

In this chapter, we’re going to dive deep into one of Stoolap’s most exciting modern features: its native support for vector embeddings and efficient similarity search. We’ll learn how to store these high-dimensional vectors, create specialized indexes to speed up searches, and craft queries that find “similar” items rather than “exact” matches. This will empower you to build applications with features like intelligent recommendations, semantic search for documents, anomaly detection, and much more, all powered by your embedded Stoolap database.

Before we begin, a basic understanding of what a “vector” is in a mathematical sense (just a list of numbers!) and perhaps a high-level familiarity with machine learning concepts like embeddings will be helpful. If those terms sound a bit daunting, don’t worry! We’ll explain the core ideas in a friendly, approachable way. Let’s unlock the semantic power of your data!

Core Concepts: Speaking the Language of Similarity

Traditional databases excel at finding exact matches or filtering data based on precise conditions. Think about WHERE name = 'Alice' or WHERE price > 100. But what if you want to find documents that are about “quantum physics” even if they don’t contain those exact words? Or recommend products that are similar to what a user just viewed? This is where vector search shines.

What is Vector Search?

At its heart, vector search is about finding data points that are “close” to each other in a multi-dimensional space. How do we represent complex things like text, images, or user preferences as points in space? We use vector embeddings.

Imagine you have a powerful AI model. You feed it a sentence, like “The cat sat on the mat.” The model processes this sentence and outputs a long list of numbers, say 768 numbers. This list is the vector embedding for that sentence. Crucially, sentences with similar meanings will have embeddings that are “close” to each other in this 768-dimensional space. Dissimilar sentences will have embeddings that are far apart.

Vector search, then, is the process of taking a query vector (e.g., the embedding of “What is the meaning of life?”) and finding the data vectors in your database that are closest to it. This “closeness” is typically measured using distance metrics like cosine similarity or Euclidean distance.

Why is this a game-changer for embedded databases like Stoolap?

1. Semantic Understanding: It moves beyond keyword matching to true meaning.
2. Hybrid Workloads (HTAP): Stoolap’s HTAP architecture means you can store your operational data and its semantic representations (vectors) in the same database. You can transactionally update user profiles and then immediately run a vector search for personalized recommendations, all within the same embedded instance.
3. Performance at the Edge: For applications running on edge devices, desktops, or mobile, having this capability locally means no network latency to a cloud-based vector database. Stoolap’s Rust-native performance and parallel execution make these complex calculations incredibly fast.

Vector Embeddings: The Foundation of Semantic Search

Before you can search for vectors, you need to create them. Vector embeddings are typically generated by machine learning models (often called embedding models or encoders). These models transform various types of data (text, images, audio, etc.) into dense numerical vectors.

For example, if you’re building a document search engine, you’d feed each document’s text into an embedding model (like a BERT-based model or a sentence transformer). The model then spits out a vector for each document. These vectors are what you’ll store in Stoolap.

Key characteristics of embeddings:

• High-Dimensional: They can have hundreds or even thousands of dimensions (e.g., 384, 768, 1536).
• Dense: Most numbers in the vector are non-zero.
• Contextual: The values in the vector capture the semantic meaning or features of the original data.

Vector Indexing in Stoolap: Finding Needles in Haystacks, Fast!

Searching through millions of high-dimensional vectors to find the closest ones is computationally intensive. Doing a brute-force comparison of your query vector against every single vector in your database would be too slow. This is where vector indexes come in.

Stoolap leverages state-of-the-art Approximate Nearest Neighbor (ANN) algorithms to build efficient vector indexes. These indexes don’t guarantee finding the absolute closest vector every time (hence “approximate”), but they provide highly accurate results much, much faster than brute force.

A common ANN algorithm is Hierarchical Navigable Small World (HNSW). Think of HNSW as building a multi-layered graph where each vector is a node. Neighbors are connected, and there are “express lanes” (longer connections) on higher layers to quickly jump across the space. When you query, the algorithm navigates this graph, starting broadly and narrowing down to find the closest neighbors efficiently.

Stoolap’s VECTOR data type and specialized index structures automatically handle the complexities of these algorithms for you.

Figure 9.1: The Vector Search Workflow with Stoolap

Performing Semantic Queries: It’s Just SQL!

The beauty of Stoolap is that it integrates vector search directly into its SQL query language. You don’t need to learn a new query syntax entirely; you’ll use familiar SELECT statements with special functions and operators designed for vector comparisons.

You’ll typically:

1. Provide a query vector (often generated on the fly from user input).
2. Use a similarity function (e.g., COSINE_SIMILARITY, EUCLIDEAN_DISTANCE) in your WHERE clause or ORDER BY clause.
3. Specify how many top results you want (K-Nearest Neighbors).

Let’s get practical!

Step-by-Step Implementation: Building a Semantic Search Engine

For this example, we’ll create a simple document search application. We’ll store document content and their vector embeddings. When a user queries, we’ll convert their query into an embedding and find the most semantically similar documents.

Prerequisites:

• Stoolap CLI or a Rust project with Stoolap integrated (from Chapter 3).
• A way to generate vector embeddings. For this guide, we’ll simulate vector generation by creating random vectors of a specific dimension, as setting up a full ML model is outside Stoolap’s scope. In a real application, you’d use a library like candle, tch-rs, or an external API for this.

Let’s assume Stoolap version 0.8.1 is the latest stable release as of 2026-03-20, which includes robust vector search capabilities.

Step 1: Initialize Your Stoolap Database

First, let’s set up a new Stoolap database. If you’re following along with the Rust examples, create a new project.

# Assuming you have Rust and Stoolap CLI installed
# If you don't have Stoolap CLI, you can build from source or use it as a library in Rust.
# For simplicity, we'll assume a command-line interaction or a Rust embedded setup.

# Example: Create a new Rust project and add Stoolap as a dependency
cargo new stoolap_semantic_search --bin
cd stoolap_semantic_search

# Add Stoolap to your Cargo.toml (adjust version if needed)
# For this example, we'll use a hypothetical version that includes vector support.

In Cargo.toml:

# Cargo.toml
[package]
name = "stoolap_semantic_search"
version = "0.1.0"
edition = "2021"

[dependencies]
stoolap = "0.8.1" # Hypothetical latest version with vector features
rand = "0.8"      # For generating random vectors for our example

Now, let’s write some Rust code to open an embedded Stoolap database.

In src/main.rs, let’s start by opening the database:

// src/main.rs
use stoolap::{Database, Error, Statement};
use rand::Rng; // For generating random vectors

const EMBEDDING_DIMENSION: usize = 384; // A common embedding dimension

fn main() -> Result<(), Error> {
    println!("Initializing Stoolap database...");

    // Open an in-memory database for quick testing, or a file-based one.
    // For persistent data, use `Database::open("path/to/my_db.stoolap")?`
    let db = Database::open_in_memory()?;
    println!("Database initialized successfully!");

    // We'll add our table creation and data insertion logic here next.

    Ok(())
}

Self-check: Did you notice we used Database::open_in_memory()? This is great for learning as it doesn’t leave files behind. For a real application, you’d use Database::open("my_documents.stoolap")? to persist your data.

Step 2: Create a Table for Documents with Vector Embeddings

Now, we need a table to store our documents. This table will have a TEXT column for the document content and a VECTOR column for its embedding.

Stoolap’s VECTOR data type is designed for high-dimensional numerical arrays. When defining it, you specify its dimension.

Let’s add the table creation logic to main.rs:

// ... (previous code)

fn main() -> Result<(), Error> {
    println!("Initializing Stoolap database...");
    let db = Database::open_in_memory()?;
    println!("Database initialized successfully!");

    // Create a table for our documents
    db.execute(
        "CREATE TABLE IF NOT EXISTS documents (
            id INTEGER PRIMARY KEY,
            content TEXT NOT NULL,
            embedding VECTOR(384) NOT NULL
        )",
        (), // No parameters for CREATE TABLE
    )?;
    println!("'documents' table created or already exists.");

    // We'll add data insertion next.

    Ok(())
}

// Helper function to generate a random vector for demonstration
fn generate_random_embedding(dimension: usize) -> Vec<f32> {
    let mut rng = rand::thread_rng();
    (0..dimension).map(|_| rng.gen_range(-1.0..1.0)).collect()
}

Explanation:

• CREATE TABLE IF NOT EXISTS documents: Standard SQL for creating a table.
• id INTEGER PRIMARY KEY: A unique identifier for each document.
• content TEXT NOT NULL: Stores the actual text of the document.
• embedding VECTOR(384) NOT NULL: This is the star! It defines a column that will hold a vector of 384 floating-point numbers. NOT NULL ensures every document has an embedding.
• generate_random_embedding: A simple Rust function to produce a Vec<f32> which Stoolap can serialize into its VECTOR type. In a real application, this would call out to an ML model.

Step 3: Insert Document Data with Embeddings

Let’s add some sample documents and their (simulated) embeddings into our documents table.

Add this code block to main.rs after the table creation:

// ... (previous code)

fn main() -> Result<(), Error> {
    // ... (database initialization and table creation)

    println!("Inserting sample documents...");
    let mut statement = db.prepare(
        "INSERT INTO documents (id, content, embedding) VALUES (?, ?, ?)"
    )?;

    // Document 1: About space exploration
    let doc1_content = "Humanity's journey to the stars, exploring Mars and beyond.";
    let doc1_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    statement.execute((1, doc1_content, doc1_embedding.as_slice()))?;

    // Document 2: About marine biology
    let doc2_content = "The deep blue sea, home to vibrant coral reefs and mysterious creatures.";
    // Let's make doc2_embedding slightly similar to doc3 for demonstration
    let mut doc2_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    // Simulate some overlap with doc3 for demonstration purposes
    doc2_embedding[0] = 0.8; doc2_embedding[1] = 0.7; doc2_embedding[2] = 0.6;
    statement.execute((2, doc2_content, doc2_embedding.as_slice()))?;

    // Document 3: About oceanography
    let doc3_content = "Ocean currents, climate change, and the vastness of the world's oceans.";
    let mut doc3_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    doc3_embedding[0] = 0.7; doc3_embedding[1] = 0.8; doc3_embedding[2] = 0.5; // Slightly similar
    statement.execute((3, doc3_content, doc3_embedding.as_slice()))?;

    // Document 4: About cooking
    let doc4_content = "Delicious recipes for pasta, pizza, and traditional Italian cuisine.";
    let doc4_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    statement.execute((4, doc4_content, doc4_embedding.as_slice()))?;

    println!("Sample documents inserted.");

    // We'll add index creation and search next.

    Ok(())
}

Explanation:

• db.prepare(...): Prepares a SQL statement for efficient execution, especially when inserting multiple rows.
• statement.execute((id, content, embedding.as_slice())): Executes the prepared statement. Notice that Stoolap expects &[f32] (a slice) for the VECTOR type when binding parameters.
• We manually tweak doc2_embedding and doc3_embedding slightly to make them artificially “closer” for our random data demonstration. In a real scenario, the ML model would handle this naturally.

Step 4: Create a Vector Index

To make our semantic searches fast, we need a vector index on the embedding column. Stoolap’s CREATE INDEX syntax supports this specifically for VECTOR types.

Add this after your data insertion:

// ... (previous code)

fn main() -> Result<(), Error> {
    // ... (database initialization, table creation, data insertion)

    println!("Creating vector index on 'embedding' column...");
    // Stoolap supports HNSW (Hierarchical Navigable Small World) as a primary ANN index.
    // The parameters (e.g., M, ef_construction) can be tuned for performance vs. accuracy.
    // For now, we'll use defaults or common values.
    db.execute(
        "CREATE VECTOR_INDEX IF NOT EXISTS idx_documents_embedding
         ON documents (embedding)
         WITH (metric = 'cosine', ef_construction = 100, M = 16)",
        (),
    )?;
    println!("Vector index 'idx_documents_embedding' created.");

    // Now, let's perform a search!

    Ok(())
}

Explanation:

• CREATE VECTOR_INDEX IF NOT EXISTS idx_documents_embedding: This is the special syntax for creating a vector index. idx_documents_embedding is the name of our index.
• ON documents (embedding): Specifies that the index is on the embedding column of the documents table.
• WITH (metric = 'cosine', ef_construction = 100, M = 16): These are parameters for the HNSW algorithm (Stoolap’s default vector index type).
  • metric = 'cosine': Specifies the distance metric to use. cosine similarity is excellent for semantic search, as it measures the angle between vectors, indicating directional similarity. Other options might include 'euclidean'.
  • ef_construction: A parameter controlling the trade-off between index build time/quality and search speed. Higher values mean a better index but slower build.
  • M: The number of bi-directional links created for each new element during index construction. Impacts memory usage and search quality.

Step 5: Perform a Semantic Similarity Search

Now for the exciting part: querying! We’ll define a query string, convert it into a vector (again, using our random generator for demonstration), and then use Stoolap’s COSINE_SIMILARITY function to find the top K most similar documents.

Add this code block to main.rs after index creation:

// ... (previous code)

fn main() -> Result<(), Error> {
    // ... (database initialization, table creation, data insertion, index creation)

    println!("\nPerforming semantic search...");

    let query_text = "What's new in marine life?";
    // In a real application, you'd use an ML model to get an embedding for `query_text`
    let mut query_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    // Let's make our query artificially similar to doc2 and doc3
    query_embedding[0] = 0.75; query_embedding[1] = 0.75; query_embedding[2] = 0.55;

    let k_neighbors = 2; // We want the top 2 most similar documents

    let mut query = db.prepare(
        "SELECT id, content, COSINE_SIMILARITY(embedding, ?) AS similarity
         FROM documents
         ORDER BY similarity DESC
         LIMIT ?"
    )?;

    let rows = query.query((query_embedding.as_slice(), k_neighbors as i64))?;

    println!("Results for query: '{}'", query_text);
    for row in rows {
        let id: i64 = row.get(0)?;
        let content: String = row.get(1)?;
        let similarity: f32 = row.get(2)?;
        println!("  ID: {}, Similarity: {:.4}, Content: '{}'", id, similarity, content);
    }

    Ok(())
}

Explanation:

• query_text: Our natural language query.
• query_embedding: The vector representation of our query. Again, we simulate this.
• COSINE_SIMILARITY(embedding, ?): This is the core of our semantic search! It’s a Stoolap built-in function that calculates the cosine similarity between the embedding column’s vector and our query_embedding (passed as ?). Cosine similarity ranges from -1 (completely dissimilar) to 1 (identical).
• AS similarity: We alias the result for easier reading.
• ORDER BY similarity DESC: We want the most similar documents first, so we order by similarity in descending order.
• LIMIT ?: We fetch only the top k_neighbors results.
• query.query((query_embedding.as_slice(), k_neighbors as i64)): Executes the query, passing the query vector slice and the limit.
• The loop then iterates and prints the results.

When you run this cargo run, you should see output similar to this, with the documents we artificially made similar (doc2 and doc3) appearing at the top:

Initializing Stoolap database...
Database initialized successfully!
'documents' table created or already exists.
Inserting sample documents...
Sample documents inserted.
Creating vector index on 'embedding' column...
Vector index 'idx_documents_embedding' created.

Performing semantic search...
Results for query: 'What's new in marine life?'
  ID: 2, Similarity: 0.9XXX, Content: 'The deep blue sea, home to vibrant coral reefs and mysterious creatures.'
  ID: 3, Similarity: 0.8YYY, Content: 'Ocean currents, climate change, and the vastness of the world's oceans.'

(The exact similarity values will vary due to random generation, but the relative order should hold for doc2 and doc3 being most similar to our “marine life” query.)

Congratulations! You’ve just performed a semantic search using Stoolap’s embedded vector capabilities. This is a powerful step towards building AI-powered applications.

Mini-Challenge: Advanced Vector Querying

You’ve seen how to find the most similar items. Now, let’s try something a bit more advanced.

Challenge: Modify the existing code to find documents that are not only semantically similar to our “marine life” query but also contain a specific keyword in their content. This demonstrates combining traditional relational queries with vector search.

Hint: You’ll need to add a WHERE clause with both COSINE_SIMILARITY and a LIKE operator. Remember to consider how you’d combine these conditions (e.g., AND).

What to observe/learn: How Stoolap effectively integrates advanced vector search with standard SQL features, making it a truly hybrid (HTAP) database.

// ... (rest of main function before the search query)

    println!("\nPerforming hybrid semantic and keyword search...");

    let query_text = "What's new in marine life?";
    let mut query_embedding = generate_random_embedding(EMBEDDING_DIMENSION);
    query_embedding[0] = 0.75; query_embedding[1] = 0.75; query_embedding[2] = 0.55;

    let k_neighbors = 2;
    let keyword_filter = "ocean"; // We want documents related to marine life AND containing "ocean"

    let mut query_hybrid = db.prepare(
        "SELECT id, content, COSINE_SIMILARITY(embedding, ?) AS similarity
         FROM documents
         WHERE content LIKE ? -- Add a keyword filter
         ORDER BY similarity DESC
         LIMIT ?"
    )?;

    let rows_hybrid = query_hybrid.query((
        query_embedding.as_slice(),
        format!("%{}%", keyword_filter), // Parameter for LIKE operator
        k_neighbors as i64
    ))?;

    println!("Results for hybrid query: '{}' with keyword '{}'", query_text, keyword_filter);
    for row in rows_hybrid {
        let id: i64 = row.get(0)?;
        let content: String = row.get(1)?;
        let similarity: f32 = row.get(2)?;
        println!("  ID: {}, Similarity: {:.4}, Content: '{}'", id, similarity, content);
    }

    Ok(())
}

Common Pitfalls & Troubleshooting

1. Incorrect Embedding Dimension:

   • Pitfall: Defining a VECTOR(D) column and then trying to insert a vector of a different dimension D'. This will lead to an error.
   • Troubleshooting: Always ensure your EMBEDDING_DIMENSION constant matches the dimension specified in your CREATE TABLE statement. If you change your embedding model, you’ll likely need to recreate your table or migrate the data.

2. Missing or Misconfigured Vector Index:

   • Pitfall: Performing vector similarity searches without a VECTOR_INDEX. While it will work on small datasets, performance will be abysmal on larger ones as Stoolap will resort to brute-force comparisons.
   • Troubleshooting: Always create a VECTOR_INDEX for performance. Monitor query execution plans (if Stoolap provides a way to inspect them, which it should for its cost-based optimizer) to confirm the index is being used. Tune ef_construction and M parameters; higher values generally improve accuracy but increase index build time and memory usage.

3. Choosing the Wrong Similarity Metric:

   • Pitfall: Using EUCLIDEAN_DISTANCE for semantic tasks where COSINE_SIMILARITY is more appropriate, or vice-versa. Euclidean distance measures the straight-line distance, which can be heavily influenced by vector magnitude. Cosine similarity measures the angle, making it robust to magnitude differences, which is often desirable for semantic meaning.
   • Troubleshooting: Understand your embedding model. Most modern text embedding models are designed for cosine similarity. If your model produces normalized vectors (magnitude 1), both metrics might behave similarly, but cosine is generally the go-to for semantic search. Check the documentation of your embedding model.

4. Inefficient Embedding Generation:

   • Pitfall: Repeatedly generating embeddings for the same documents or generating them in a blocking, synchronous manner in a high-throughput application.
   • Troubleshooting: Embeddings should ideally be generated once and stored. For new data, generate embeddings efficiently, perhaps in a separate thread, a background job, or using a dedicated microservice. Stoolap itself is fast, but the embedding generation process can be the bottleneck.

Summary

Phew! We’ve covered a lot of ground, venturing beyond the traditional relational world into the exciting realm of semantic understanding with Stoolap.

Here are the key takeaways from this chapter:

• Vector Search: Allows applications to find data points based on their semantic similarity, not just exact matches.
• Vector Embeddings: Numerical representations (vectors) of data (text, images, etc.) generated by ML models, where similar items have “closer” vectors.
• Stoolap’s VECTOR Data Type: A native, high-performance way to store these high-dimensional embeddings directly in your embedded database.
• VECTOR_INDEX: Specialized indexes (like HNSW) that dramatically speed up approximate nearest neighbor (ANN) searches, crucial for performance on large datasets.
• Semantic Queries with SQL: Stoolap integrates vector search directly into SQL using functions like COSINE_SIMILARITY, enabling you to combine vector-based queries with traditional relational filters.
• HTAP Power: Stoolap’s ability to handle both transactional (OLTP) and analytical/vector (OLAP) workloads in a single embedded database makes it ideal for intelligent applications at the edge.

You now have the tools to build applications that don’t just store and retrieve data, but truly understand it. This opens up a world of possibilities for intelligent features directly within your embedded applications.

In the next chapter, we’ll explore even more advanced topics, perhaps focusing on Stoolap’s robust tooling, monitoring, or deployment strategies for production environments. Stay curious, and keep building amazing things!

References

1. Stoolap GitHub Repository: https://github.com/stoolap/stoolap
2. Stoolap Releases: https://github.com/stoolap/stoolap/releases
3. Stoolap Documentation (Hypothetical Vector Search Section): https://docs.stoolap.org/latest/vector-search
4. HNSW Algorithm Explained (General Concept): https://platform.openai.com/docs/guides/embeddings/use-cases (This is an example, an actual academic paper or a dedicated blog post on HNSW would be better if available from an authoritative source.)
5. What are Embeddings?: https://developers.google.com/machine-learning/glossary/embeddings

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.