Ragex.Retrieval.QueryExpansion (Ragex v0.20.0)

View Source

Query expansion using MetaAST semantic features.

Enhances search queries by:

  • Extracting semantic features from results
  • Adding cross-language synonyms
  • Expanding with related constructs
  • Building semantic context

HyDE (Hypothetical Document Embedding)

hyde_embedding/2 generates a short hypothetical code snippet that would answer the query (using the configured AI provider), embeds it, and returns the embedding. The caller can then pass this embedding directly to VectorStore.search/2 instead of embedding the raw query — a technique that often improves recall when the query is expressed in natural language but the indexed documents are code.

Examples

# Basic expansion
QueryExpansion.expand("find map function")
# => "find map function collection transform iterate"

# Context-aware expansion
QueryExpansion.expand("debug error", intent: :debug)
# => "debug error exception failure bug issue problem"

# HyDE: generate a hypothetical answer, embed it, then search
{:ok, hypo_embedding} = QueryExpansion.hyde_embedding("function that retries on timeout")
results = VectorStore.search(hypo_embedding, limit: 10)

Summary

Functions

enrich_query(query, features, opts \\ [])

Build an enriched query from original query + result features.

expand(query, opts \\ [])

Expand a query string with semantic features and synonyms.

extract_features_from_results(results)

Extract semantic features from query results to enhance future queries.

hyde_embedding(query, opts \\ [])

Generate a hypothetical code snippet that would answer query, then embed it.

suggest_variations(query, opts \\ [])

Suggest query variations based on semantic analysis.

Functions

enrich_query(query, features, opts \\ [])

@spec enrich_query(String.t(), [String.t()], keyword()) :: String.t()

Build an enriched query from original query + result features.

Useful for iterative search refinement.

Examples

results = [...]  # Initial search results
features = QueryExpansion.extract_features_from_results(results)

QueryExpansion.enrich_query("map function", features, max_features: 3)
# => "map function collection transform iterate"

expand(query, opts \\ [])

@spec expand(
  String.t(),
  keyword()
) :: String.t()

Expand a query string with semantic features and synonyms.

Options

  • :intent - Query intent (:explain, :refactor, :example, :debug) (default: auto-detect)
  • :include_synonyms - Include semantic synonyms (default: true)
  • :include_cross_language - Include cross-language terms (default: true)
  • :max_terms - Maximum expansion terms to add (default: 5)

Examples

QueryExpansion.expand("map over list")
# => "map over list collection iterate transform apply"

QueryExpansion.expand("fix bug", intent: :debug)
# => "fix bug error exception failure issue problem"

extract_features_from_results(results)

@spec extract_features_from_results([map()]) :: [String.t()]

Extract semantic features from query results to enhance future queries.

Analyzes MetaAST metadata from results to build a semantic feature set that can be used for query refinement.

Examples

results = [%{meta_ast: {:collection_op, :map, ...}}, ...]

QueryExpansion.extract_features_from_results(results)
# => ["collection", "map", "transform", "iteration", "apply"]

hyde_embedding(query, opts \\ [])

@spec hyde_embedding(
  String.t(),
  keyword()
) :: {:ok, [float()]} | {:error, term()}

Generate a hypothetical code snippet that would answer query, then embed it.

Returns {:ok, embedding} where embedding is a list of floats that can be passed directly to VectorStore.search/2. Returns {:error, reason} when the AI provider or embedding model is unavailable.

The approach (HyDE, Gao et al. 2022) improves recall for natural-language queries against code corpora by bridging the vocabulary gap between query and document language.

Options

  • :provider - override AI provider (default: configured default)
  • :language - hint the language for the hypothetical snippet (default: "elixir")
  • :max_tokens - tokens for the hypothetical snippet (default: 300)
  • :timeout - milliseconds for the LLM call (default: 10_000)

suggest_variations(query, opts \\ [])

@spec suggest_variations(
  String.t(),
  keyword()
) :: [String.t()]

Suggest query variations based on semantic analysis.

Returns alternative phrasings that might yield better results.

Examples

QueryExpansion.suggest_variations("find map")
# => [
#   "find map function",
#   "find transform operation",
#   "find collection map",
#   "find iterate apply"
# ]