IntentForge v2 — API Reference

Base URL (Production): https://api.oxiverse.com

Base URL (Development): http://localhost:4000

Protocol: HTTPS (production) / HTTP (development)

Format: JSON

Authentication: None (public API)

Table of Contents

• Endpoints
  • GET /
  • GET /health
  • GET /search
  • GET /search/fast
  • GET /images
  • GET /videos
  • GET /news
  • GET /spellcheck
  • POST /goals
  • POST /goals/quick
  • GET /goals/:goal_id
  • POST /goals/:goal_id/answers
  • GET /goals/leaderboard
• Query Parameters
• Response Structures
• Advanced Query Operators
• Pagination
• Intent Classification
• Constraint Extraction
• Spell Correction
• Geolocation & Local Queries
• Scoring & Ranking
• Caching
• Error Handling
• Performance & Stress Test Results
• Goals API
• Examples
• Architecture
• Getting Started (verified)

Getting Started (verified)

The steps below were executed on 2026-08-05 against the already-running dev stack (localhost:4000). The stack is brought up with make dev-up (see Makefile → builds services/docker-compose.dev.yml). All example responses are real and traceable to docs/_generated/api-transcript.md.

1. Health check

curl -s http://localhost:4000/health
# → OK   (HTTP 200)

2. Root identifier

curl -s http://localhost:4000/
# → IntentForge-v2 Gateway   (HTTP 200, text/plain)

3. First real search

curl -s "http://localhost:4000/search?q=rust+async+web+framework" | head -c 400
# → {"category":"informational","confidence":0.6,"constraints":["+async","+rust","+web"], ...}
#    (HTTP 200, ~4.9 s cold; ~15 ms on repeat within the 5-min cache)

4. Reading the response

• intent — the detailed subtype (technical, informational, comparison, fresh, navigational, how-to, local, transactional).
• category — the coarse bucket (navigational / informational / transactional).
• confidence — a real float (~0.3–0.9); not a fixed 0.75.
• distribution — full intent-probability breakdown across all 8 subtypes.
• results[] — ranked MergedResult objects (see below).
• results_before_filter / results_after_filter / total — counts for constraint diagnostics.

5. Intent classes (verified live)

6. Error handling (verified live)

curl -s -o /dev/null -w "%{http_code}" "http://localhost:4000/search?q="
# → 400  (body: {"error":"empty_query","message":"Query parameter 'q' is empty",...})
curl -s -o /dev/null -w "%{http_code}" "http://localhost:4000/search?q=r"
# → 400  (body: {"error":"invalid_query","message":"Query has no retrievable content (stopword-only or single character)",...})
# Note: a protected single char like `go` or `rust` is EXEMPT and returns 200.

Endpoints

GET /

Returns a plain-text identifier string.

Response

200 OK
Content-Type: text/plain

IntentForge-v2 Gateway

GET /health

Health check endpoint. Returns "OK" when the gateway is running.

Response

200 OK
Content-Type: text/plain

OK

GET /search

Full search endpoint. Queries multiple backends (SearXNG via VPN, local index) in parallel, classifies intent, extracts constraints, auto-corrects spelling, deduplicates, scores, and ranks results.

Query Parameters

Response 200 OK

Verified shape (this session): a successful /search returns these top-level keys (observed on every live response):

query, intent, category, confidence, constraints, structured_constraints, expanded_queries, distribution, results, results_before_filter, results_after_filter, total, limit, offset, has_more.

Optionally present: applied_constraints (when operators/negations are applied), spell_corrected_query (when a correction fired), query_quality (only on low/junk queries), deep_result, price_verified (transactional).

geo_location, warnings, ignored_constraints were absent from all observed successful responses (declared-but-omitted None fields).

confidence is a real float in ~0.30–0.90, not always 0.75 — the value depends on the query and the intent engine.

Example (real response truncated; full body in docs/_generated/api-transcript.md block 12 — python web framework not django):

{
  "query": "python web framework not django",
  "intent": "technical",
  "category": "informational",
  "confidence": 0.60,
  "constraints": ["+python", "+web", "-django"],
  "applied_constraints": ["not:django"],
  "structured_constraints": {
    "positive": ["python", "web"],
    "negative": ["django"],
    "entities": [],
    "language": null,
    "file_types": [],
    "sites": [],
    "phrases": [],
    "intitle": [],
    "inurl": [],
    "intext": [],
    "related": []
  },
  "distribution": {
    "navigational": 0.39,
    "informational": 0.16,
    "technical": 0.08,
    "how-to": 0.11,
    "comparison": 0.03,
    "fresh": 0.04,
    "transactional": 0.16,
    "local": 0.03
  },
  "results": [
    {
      "url": "https://www.infoworld.com/article/2338670/3-python-web-frameworks-for-beautiful-front-ends.html",
      "title": "3 Python web frameworks for beautiful front ends - InfoWorld",
      "content": "We'll look at three Python web frameworks that follow this paradigm...",
      "score": 1.0,
      "authority": 1.0,
      "quality": 1.0,
      "is_local": false,
      "sources": ["duckduckgo"]
    }
  ],
  "results_before_filter": 25,
  "results_after_filter": 25,
  "total": 25,
  "limit": 24,
  "offset": 0,
  "has_more": true
}

Note: expanded_queries and distribution are present on every response; they were omitted from the abridged example above for brevity. See block 12 in the transcript for the full body.

Status Codes

All error bodies are JSON (see Error Handling). Verified live: blocks 18–25 in docs/_generated/api-transcript.md.

GET /search/fast

Fast search endpoint. Returns results from the local crawl index only — no SearXNG, no intent analysis, no constraint extraction. Designed for instant feedback (~100ms) while /search runs in parallel.

Query Parameters

Response 200 OK

Verified (this session): the real body is \{ "count": <int>, "results": [ ... ], "source": <str> \}. The top-level source field IS present (observed value "local" on this instance); each result also carries sources: ["local"] internally. count reflects the number returned (default 10 in this run; limit is accepted but the local fast path returned 10).

{
  "count": 10,
  "source": "local",
  "results": [
    {
      "url": "https://rustwebframework.org/",
      "title": "Rust Web Framework",
      "content": "Rust Web Framework GitHub Getting started ...",
      "score": 1.0,
      "authority": 0.70,
      "is_local": true,
      "quality": 1.0,
      "sources": ["local"]
    }
  ]
}

Raw observed body: docs/_generated/api-transcript.md block 13.

Notes

• No intent classification, constraint extraction, or spell correction.
• No pagination metadata (total, has_more, etc.) — only returns a snapshot.
• Results come from the local crawl index only.
• Useful for frontend: call /search/fast + /search in parallel for instant + full results.

GET /images

Image search via SearXNG (categories=images). Returns a flat list with image + page metadata.

Query Parameters

Response 200 OK — top-level \{ count, query, results[] \}. Each result (verified live, 2026-08-05):

{
  "count": 32,
  "query": "rust programming",
  "results": [
    {
      "title": "Getting started - Rust Programming Language",
      "url": "https://rust-lang.org/learn/get-started/",
      "image_url": "https://www.rust-lang.org/static/images/rust-social-wide.jpg",
      "thumbnail_url": "https://ts2.mm.bing.net/th?id=OIP.W8KBrJgmsIlYtn24AhHfSQHaDt&pid=15.1",
      "description": "Getting started - Rust Programming Language",
      "source": "bing images",
      "score": 0.9000000357627869
    }
  ]
}

Verified fields: title, url, image_url, thumbnail_url, description, source, score. Note the image endpoint returns image_url (full image) and thumbnail_url — not the thumbnail field used by /videos. Full raw body: docs/_generated/_round_v2_raw.md block ### IMAGES rust programming.

GET /videos

Video search via SearXNG (categories=videos). Returns a flat list with thumbnail (not thumbnail_url) and video_id.

Query Parameters

Response 200 OK — top-level \{ count, query, results[] \}. Each result (verified live, 2026-08-05):

{
  "count": 31,
  "query": "rust tutorial",
  "results": [
    {
      "title": "Learn Rust Programming - Complete Course 🦀",
      "url": "https://www.youtube.com/watch?v=BpPEoZW5IiY",
      "description": "1.2M views - Jun 8, 2023 - YouTube - freeCodeCamp.org",
      "thumbnail": "https://th.bing.com/th/id/OVP.X9INETUn2tEG8KJL2Wrl3QHgFo?w=243&h=136&c=7&rs=1&qlt=70&o=7&pid=2.1&rm=3",
      "video_id": "",
      "source": "bing videos",
      "score": 0.6499999761581421
    }
  ]
}

Verified fields: title, url, description, thumbnail, video_id (observed empty in this run), source, score. Note thumbnail here vs thumbnail_url in /images. video_id was empty on all observed results. Full raw body: docs/_generated/_round_v2_raw.md block ### VIDEOS rust tutorial.

GET /news

News search via SearXNG (categories=news). Returns a flat list with published_at.

Query Parameters

Response 200 OK — top-level \{ count, query, results[] \}. Each result (verified live, 2026-08-05):

{
  "count": 39,
  "query": "artificial intelligence",
  "results": [
    {
      "title": "As computer science enrollments drop, artificial intelligence classes fill up",
      "url": "https://www.msn.com/en-us/money/careersandeducation/as-computer-science-enrollments-drop-artificial-intelligence-classes-fill-up/ar-AA29jdXb",
      "description": "Hiring for entry-level software developers has slowed, and college enrollment in computer science is declining ...",
      "published_at": "",
      "source": "bing news",
      "score": 0.800000011920929
    }
  ]
}

Verified fields: title, url, description, published_at (observed empty string "" on most bing-news items; hackernews items carried ISO timestamps like "2019-11-13T23:17:23"), source ("bing news" / "hackernews"), score. Full raw body: docs/_generated/_round_v2_raw.md block ### NEWS artificial intelligence.

GET /spellcheck

Spelling-correction preview. Exposes the engine's in-process SymSpell + LinSpell index (the same one /search uses to auto-correct) as a standalone "did you mean?" service, so a client can warn the user before issuing a search. No LLM, no network, no extra indexing — it reads the dictionary that is already built at gateway startup.

Query Parameters

Response 200 OK — top-level \{ query, corrected, changed, corrections[] \}:

{
  "query": "pythn programing langauge",
  "corrected": "python programming language",
  "changed": true,
  "corrections": [
    { "original": "pythn", "suggestion": "python", "in_dictionary": false },
    { "original": "programing", "suggestion": "programming", "in_dictionary": true },
    { "original": "langauge", "suggestion": "language", "in_dictionary": false }
  ]
}

Verified (this round, 2026-08-09): a typo string returns changed: true with a correction per changed token. Protected brands/tech terms (openai, rust, kubernetes, …) are never "corrected" — "openai rust tutorial" returns changed: false and an empty corrections array. URL tokens, code tokens (. / @ # $ or containing a digit), and very short words (< 4 chars) are skipped by the corrector and omitted entirely from the corrections array — the endpoint only lists tokens it actually proposed fixing, so the client never flags skipped tokens as typos. They are still preserved verbatim in the whole-query corrected string. The corrected string matches what /search runs for the same query (verified 2026-08-09: /spellcheck?q=pythn+programing+langauge → corrected: "python programming language", and /search?q=pythn+programing+langauge returns "query":"python programming language"). Example: /spellcheck?q=pythn+kubernetes.io returns changed:true with corrections containing only pythn→python (the kubernetes.io URL token is skipped and absent from corrections, but retained in corrected).

Empty query returns 400 with the standard error envelope (same shape as /search):

{ "error": "empty_query", "message": "Query parameter 'q' is empty", "results": [], "query": "", "corrected": "", "changed": false, "corrections": [] }

Notes

• Pure function of the query + the built dictionary; no per-query tuned constants, no domain allow/deny lists.
• The endpoint is additive — it does not change /search ranking, negation gating, or calibration. It is a read-only preview of the existing correction path.

# See what the engine would correct in a query
curl "http://localhost:4000/spellcheck?q=pythn+programing+langauge"
# → {"query":"pythn programing langauge","corrected":"python programming language","changed":true,"corrections":[...]}

Query Parameters

All search endpoints accept the following standard parameters:

Advanced Query Operators

The /search endpoint parses a rich set of operators directly from the query string. These are extracted into applied_constraints and structured_constraints in the response.

Verification status (this session, 2026-08-05): the following operators were exercised live and confirmed to populate structured_constraints + applied_constraints: site: (block 32), filetype: (block 33), intitle: (block 34), inurl: (block 35), after: (block 36), and the natural-language negation not X (block 12 → negative:["django"], applied_constraints:["not:django"]). The fresh/today intent auto-produced after_date+before_date = today (block 9). The price: / price_min: / price_max: operators and lang: / intext: / related: were NOT exercised this session — treat their extraction as unverified.

Multiple operators can be combined: site:arxiv.org site:wikipedia.org quantum computing → both sites are applied (observed).

Natural language date ranges are automatically converted:

• "past 7 days", "last week", "this month" → after:YYYY-MM-DD before:YYYY-MM-DD
• "yesterday", "today", "recent", "latest", "fresh"

Pagination

The /search endpoint supports cursor-free pagination via limit and offset parameters.

Response fields:

Example pagination flow:

# Page 1
curl "http://localhost:4000/search?q=python&limit=10&offset=0"
# → has_more: true, total: 47

# Page 2
curl "http://localhost:4000/search?q=python&limit=10&offset=10"
# → has_more: true, total: 47

# Page 5 (last page)
curl "http://localhost:4000/search?q=python&limit=10&offset=40"
# → has_more: false, total: 47

Response Structures

UnifiedResponse (from /search)

MergedResult (from /search)

The unified result type for the main search endpoint. Results can come from local index, web search engines, or both. When a URL appears in multiple sources, sources are merged and a consensus boost is applied.

StructuredConstraints (from /search)

Intent Classification

The intent engine classifies queries into detailed subtypes. The category field maps these to standard search categories.

The distribution field provides a full breakdown across all intent types as float probabilities (summing to ~1.0), useful for threshold-based decision making.

Constraint Extraction

The intent engine extracts positive and negative constraints from natural language queries.

Negative constraint patterns:

• not X → "python web framework not django" → negative: ["django"]
• without X → "text editor without vim" → negative: ["vim"]
• except X → "javascript framework except react" → negative: ["react"]
• besides X → "css framework besides bootstrap" → negative: ["bootstrap"]
• excluding X → "database excluding mongodb" → negative: ["mongodb"]
• no X → "linux distro no ubuntu" → negative: ["ubuntu"]
• minus X → "frontend framework minus angular" → negative: ["angular"]
• other than X → "programming language other than java" → negative: ["java"]
• alternative to X → "search engine alternative to google" → negative: ["google"]
• instead of X → "static site generator instead of jekyll" → negative: ["jekyll"]

Positive constraints are extracted from the remaining meaningful terms (excluding stop words and negation terms).

Smart substring matching for negation: The engine prevents false-positive hits for negative constraints:

• "not java" does NOT filter out "javascript" results
• "not go" does NOT filter out "google" or "django" results
• Short terms (< 3 chars) use exact-match only
• Terms must dominate the token length (≥ 75%) for compound matches

Synonym expansion for negative constraints:

• aws → also matches "amazon", "amazon web services"
• gcp → also matches "google cloud", "google"
• azure → also matches "microsoft", "microsoft azure"
• vscode → also matches "vs code", "visual studio code"
• google workspace → also matches "gsuite", "google docs", etc.

Spell Correction

The API automatically detects and corrects spelling errors in queries using a two-stage SymSpell + LinSpell approach with an embedded 15,000+ word frequency dictionary.

When a correction is applied:

• The query field reflects the corrected query
• The spell_corrected_query field contains the original (uncorrected) query
• The expanded_queries array includes expansions of the corrected form

Protection against false positives:

• Protected brand/tech terms (openai, kubernetes, podman, etc.) are NEVER corrected
• Tech terms with unusual character bigrams (e.g. "podman") are not English-ified
• Short words (< 4 chars; MIN_CORRECT_LENGTH) and known 3-letter terms are left unchanged
• URLs, code terms, and words with numbers/special characters are never touched
• Single-character substitutions between two natural-looking words are blocked (prevents "ramen" → "raven")

Examples:

Geolocation & Local Queries

The API supports two geolocation mechanisms:

1. IP-Derived Geolocation

When the GeoLite2 database is available, the client's IP address is used to approximate location and return regionally relevant results.

2. Query-Derived Location

If the query explicitly mentions a location from the gazetteer, it overrides the IP-derived location. This means a user in India searching "restaurants in tokyo" gets Japan-localized results.

Response field: geo_location

{
  "geo_location": {
    "country_code": "JP",
    "country_name": "Japan",
    "region": null,
    "city": "tokyo",
    "postal_code": null,
    "latitude": null,
    "longitude": null,
    "time_zone": null
  }
}

Location Gazetteer

The built-in gazetteer maps 70+ countries and 40+ major cities to ISO-3166 country codes. Whole-word matching prevents false positives (e.g. "java" the language never matches "Japan").

Supported countries: US, GB, CA, AU, NZ, DE, FR, ES, IT, PT, NL, IE, SE, NO, DK, FI, PL, AT, CH, BE, RU, UA, TR, GR, JP, CN, KR, IN, SG, HK, BR, MX, AR, AE, SA, EG, IL, TH, VN, ID, MY, PH, ZA, NG, KE, CZ, HU, RO, HR, SI, and more.

Supported cities: Tokyo, London, Paris, Berlin, Madrid, Rome, Amsterdam, Dublin, Stockholm, Oslo, Copenhagen, Helsinki, Moscow, Kyiv, Istanbul, Athens, Beijing, Shanghai, Seoul, Delhi, Mumbai, Bangalore, Singapore, Sydney, Melbourne, Auckland, New York, San Francisco, Los Angeles, Chicago, Seattle, Boston, Austin, Toronto, Vancouver, Sao Paulo, Mexico City, Dubai, Cairo, Bangkok, Jakarta, Cape Town, Lagos, and more.

Scoring & Ranking

Results are scored using a multi-signal ranking algorithm:

Domain authority signals (fully algorithmic, no hardcoded lists):

• .edu / .gov TLDs: +0.3
• .org / .net TLDs: +0.1
• .ac.uk academic TLDs: +0.3
• Documentation subdomains (docs., api., dev.): +0.25
• Documentation paths (/docs/, /api/, /reference/): +0.2
• Clean bare domains (2-part host): +0.1
• Deep descriptive paths with long segments: +0.2 – 0.35
• Code repo patterns (/owner/repo/): +0.1
• Spam/clickbait path patterns: -0.2
• Too many subdomains (≥ 5 parts): -0.1

Freshness half-lives (per intent):

Alternative listing page detection: When a query uses negative constraints like "not react not vue", pages titled "Top 10 React Alternatives" are detected using:

• Title comparison signals (alternative, vs, best N, comparison) — 70% weight
• URL path signals (/alternatives/, /vs/, /compare/) — 20% weight
• Content marker signals — 10% weight

These pages receive a negative-constraint exemption so they still rank well despite mentioning excluded terms.

Caching

All endpoints cache responses by query. The cache key is the trimmed, lowercased query combined with a pagination key (limit/count/n and offset), so two requests for the same query with different limit/offset do not share a cached body (see handle_search cache-key logic).

Verified behavior (2026-08-05, dev instance localhost:4000): The /search cache TTL is 5 minutes (300 s), not 30 minutes. Repeating an identical /search?q=rust%20async%20web%20framework request returned in ~15 ms (vs ~4–5 s cold), confirming an in-memory cache. Source: services/gateway/src/main.rs cache-key builder + 5-min TTL comment at the cache-check block.

The cache is reset on the first request after expiry, not on a fixed interval. This means traffic spikes only see at most one cache-miss request per TTL window.

Latency figures (measured, this session):

• Cold (uncached) /search: ~3.8–4.9 s wall-clock (includes SearXNG-over-VPN + Tor fan-out, intent engine, indexer, ranking).
• Cached /search repeat: ~3–15 ms.
• /search/fast: ~13 ms (local index only).
• /images: ~2–4 s cold.
• /news: ~2.3 s cold.
• Error responses (400): ~3–24 ms.

Error Handling

Verified (this session, 2026-08-05): All error responses return HTTP 400 with a JSON body. There are two distinct error codes, both observed live:

Error bodies always include the full UnifiedResponse envelope (with null for intent/category/confidence/distribution, empty results: [], and an empty structured_constraints object), plus error, message, and (for gibberish) query_quality. See docs/_generated/api-transcript.md blocks 18–25 for the exact raw bodies.

Notes on the documented-but-not-observed cases:

• The 200 + \{"error":"upstream_unavailable",...\} failure mode described below was NOT triggered in this session — all upstreams (SearXNG via VPN, SearXNG2 via Tor, local indexer) were healthy. Treat it as aspirational/unverified until reproduced. The API is designed never to return 5xx; internal errors are handled gracefully (partial results from whichever backends succeeded, or an empty results array with a diagnostic message).
• warnings and geo_location fields are declared in the UnifiedResponse struct but were absent from every successful /search response observed this session (they serialize as None and are omitted). Do not assume they are always present.

Upstream failure diagnostics (documented, unverified this session): When all search engines fail, the response is documented to include:

• error: "upstream_unavailable"
• message: "All upstream search engines timed out or failed to respond. This is a temporary upstream/connectivity issue, not a genuine zero-hit. Please retry."
• warnings: ["No web results were returned by the upstream search engines for this query."]

Performance & Stress Test Results

Results from live testing of the development instance (localhost:4000), measured this session (2026-08-05):

Notes / caveats:

• The earlier doc claimed a ~10.1 s cold start and 2.8–5.7 ms "warm" latency. This session measured ~3.8–4.9 s cold and ~3–15 ms cached — the upstream fan-out dominates; absolute numbers vary with VPN/Tor egress and upstream engine load.
• Concurrent / stress testing was not re-run this session; the 5-simultaneous-200-OK figure from prior docs is unverified here.
• Cache-hit ratio approaches 100% for repeated queries within the 5-min TTL window.

Sources of latency breakdown (uncached):

• SearXNG backend queries: 2–5s (parallel, includes VPN/Tor routing)
• Intent classification: < 10ms
• Deduplication + scoring: < 5ms
• Spell correction: < 1ms

Recommended frontend pattern:

// Fire both requests simultaneously
const [fast, full] = await Promise.all([
  fetch('/search/fast?q=' + encodeURIComponent(query)).then(r => r.json()),
  fetch('/search?q=' + encodeURIComponent(query)).then(r => r.json())
]);

// Show fast results immediately (~100ms)
renderResults(fast.results);

// Replace with full results when ready (~3-5s uncached, ~5ms cached)
renderResults(full.results);

Examples

Basic search

curl "https://api.oxiverse.com/search?q=python+programming"

Search with negative constraint

curl "https://api.oxiverse.com/search?q=python+web+framework+not+django"

Search with multiple negative constraints

curl "https://api.oxiverse.com/search?q=javascript+not+java+not+typescript"

Site-restricted search

curl "https://api.oxiverse.com/search?q=site:arxiv.org+transformer+attention"

Multi-site restricted search

curl "https://api.oxiverse.com/search?q=site:arxiv.org+site:wikipedia.org+quantum+computing"

File type filter

curl "https://api.oxiverse.com/search?q=react+filetype:pdf"

In-title search

curl "https://api.oxiverse.com/search?q=intitle:rust+async+framework"

Comparison query

curl "https://api.oxiverse.com/search?q=react+vs+vue+vs+angular+2026"

Location-aware search

curl "https://api.oxiverse.com/search?q=best+restaurants+in+tokyo+not+sushi"

Recency / date-constrained search

curl "https://api.oxiverse.com/search?q=latest+AI+news+past+7+days"

Price-constrained search

curl "https://api.oxiverse.com/search?q=price%3A%3C50+wireless+headphones"

Paginated search

# Page 1: first 5 results
curl "https://api.oxiverse.com/search?q=python+tutorial&limit=5&offset=0"

# Page 2: next 5 results
curl "https://api.oxiverse.com/search?q=python+tutorial&limit=5&offset=5"

How-to query with expanded queries

curl "https://api.oxiverse.com/search?q=how+to+deploy+docker+compose+production"

Spell correction demonstration

# Query with typos — the engine autocorrects
curl "https://api.oxiverse.com/search?q=rust+progamming+langauge"

Image search

curl "https://api.oxiverse.com/images?q=rust+logo"

Video search

curl "https://api.oxiverse.com/videos?q=kubernetes+tutorial"

News search

curl "https://api.oxiverse.com/news?q=AI+news"

Fast local-only search

curl "https://api.oxiverse.com/search/fast?q=python"

Parallel fast + full search (frontend pattern)

// Fire both requests simultaneously
const [fast, full] = await Promise.all([
  fetch('/search/fast?q=' + encodeURIComponent(query)).then(r => r.json()),
  fetch('/search?q=' + encodeURIComponent(query)).then(r => r.json())
]);

// Show fast results immediately (~100ms)
renderResults(fast.results);

// Replace with full results when ready (~3-5s)
renderResults(full.results);

Extract results with jq

# Get top-5 titles
curl -s "https://api.oxiverse.com/search?q=python" | jq '.results[:5][] | {title, url, score}'

# Get negative constraints
curl -s "https://api.oxiverse.com/search?q=python+not+django" | jq '.structured_constraints.negative'

# Get intent and confidence
curl -s "https://api.oxiverse.com/search?q=how+to+deploy+docker" | jq '{intent, confidence, category}'

# Get spell correction info
curl -s "https://api.oxiverse.com/search?q=rust+progamming" | jq '{query, spell_corrected_query}'

# Get pagination metadata
curl -s "https://api.oxiverse.com/search?q=python&limit=5" | jq '{total, limit, offset, has_more}'

Architecture

Client → Traefik (SSL) → Gateway (port 4000)
                              │
                              ├→ Intent Engine (classify + extract constraints)
                              │   ├── Intent classification (8 subtypes)
                              │   ├── Constraint extraction (positive/negative/operators)
                              │   ├── Spell correction (SymSpell + LinSpell)
                              │   ├── Location detection (gazetteer)
                              │   └── Query expansion
                              │
                              ├→ SearXNG (web search via VPN — bing, brave, duckduckgo, startpage, mojeek)
                              ├→ SearXNG2 (parallel VPN fan-out — additional engine instances)
                              ├→ Local Index (crawled pages via local indexer)
                              │
                              ├→ Deduplicate + Score + Rank → Response
                              │   ├── URL deduplication + source merging
                              │   ├── Domain authority (algorithmic)
                              │   ├── Freshness decay (per-intent half-life)
                              │   ├── Intent boost (structural signals)
                              │   ├── Consensus boost (multi-source bonus)
                              │   ├── Content quality (Shannon entropy)
                              │   ├── Alternative listing page detection
                              │   └── Constraint filtering
                              │
                              └── Media Endpoints
                                  ├── /images (via SearXNG categories=images)
                                  ├── /videos (via SearXNG categories=videos)
                                  └── /news (via SearXNG categories=news)

All search backends are queried in parallel with configurable timeouts. Results are deduplicated by URL, scored using multi-signal ranking, filtered by constraints, paginated, and returned as a unified list.

Privacy: All outbound search requests go through VPN (Gluetun) or Tor. No user data, tracking, or analytics is included in responses or used for ranking.

Goals API

The Goals feature transforms a user's long-term goal (e.g. "build an AI assistant", "write a fantasy novel", "start a SaaS business") into a personalized, phased roadmap with curated resources, deadlines, and deliverables.

The system works in two flows:

1. Discovery Flow (POST /goals → questions → POST /goals/:id/answers → roadmap)
2. Quick Flow (POST /goals/quick → immediate full roadmap)

Flow Overview

Discovery Flow:
  Client                                    Gateway
    │                                         │
    ├─ POST /goals  { goal: "..." } ──────────┤
    │                                         ├── classify_goal() → intent engine
    │                                         ├── search_resources() → 20 results from /search
    │                                         ├── generate_questions() → domain-specific questions
    │                                         ├── GoalStore.insert() → goal_0001
    │◄──────── { goal_id, questions } ────────┤
    │                                         │
    ├─ POST /goals/{id}/answers { answers } ──┤
    │                                         ├── generate_roadmap() → phased plan
    │                                         ├── GoalStore.update_roadmap()
    │◄─────────── { roadmap } ────────────────┤

Quick Flow (one-shot):
  Client                                    Gateway
    │                                         │
    ├─ POST /goals/quick  { goal: "..." } ────┤
    │                                         ├── classify_goal()
    │                                         ├── search_resources()
    │                                         ├── default_answers (Q1=timeline, Q2=hours)
    │                                         ├── generate_roadmap() → full plan
    │◄─────── { goal_id, roadmap } ───────────┤

Endpoints

Goals Error Codes (verified live, 2026-08-05)

All Goals errors return a JSON body with error + message.

See docs/_generated/_round_v2_raw.md for the exact raw bodies (GOALS update progress ... and the corrected phase_id:1 blocks).

POST /goals

Creates a new goal and returns domain-specific questions tailored to the goal type.

Request Body

{
  "goal": "build a full-stack web app for project management with team collaboration"
}

Response 200 OK

{
  "goal_id": "goal_0001",
  "goal": "build a full-stack web app for project management with team collaboration",
  "intent": "technical",
  "questions": [
    {
      "id": 1,
      "question": "What is your target timeline for this goal?",
      "description": "How much calendar time do you want to allocate? This sets the pacing of each phase.",
      "options": [
        "1 month — Quick sprint",
        "3 months — Quarter project",
        "6 months — Half-year journey",
        "12 months — Year-long mastery",
        "Flexible — No strict deadline"
      ],
      "type": "single_choice"
    },
    {
      "id": 2,
      "question": "How many hours per week can you dedicate?",
      "description": "Consistency matters more than intensity...",
      "options": ["1-5 hours — Casual", "5-10 hours — Evenings", "10-20 hours — Half-time", "20+ hours — Full-time"],
      "type": "single_choice"
    },
    {
      "id": 3,
      "question": "What architecture pattern do you want to follow?",
      "description": "The architecture shapes how your components communicate and scale.",
      "options": [
        "Monolithic — simple, single deployable",
        "Microservices — independent, deployable services",
        "Serverless — functions as a service",
        "Event-driven — message queues and async processing",
        "Jamstack — static frontend + APIs"
      ],
      "type": "single_choice"
    }
  ],
  "total_questions": 7,
  "created_at": "2026-07-29T12:00:00Z",
  "next_step": {
    "method": "POST",
    "path": "/goals/goal_0001/answers",
    "body": {
      "answers": [
        {"question_id": 1, "answer": "..."}
      ]
    }
  }
}

Status Codes

Input validation: Goal must be at least 3 characters.

POST /goals/:goal_id/answers

Submits answers to the questions from POST /goals and generates a personalized phased roadmap.

Request Body

{
  "answers": [
    {"question_id": 1, "answer": "3 months — Quarter project"},
    {"question_id": 2, "answer": "5-10 hours — Evenings & weekends"},
    {"question_id": 3, "answer": "Microservices — independent, deployable services"},
    {"question_id": 4, "answer": "Hybrid — SQL + cache layer (Redis)"},
    {"question_id": 5, "answer": "Container / Kubernetes (Docker, EKS, GKE)"},
    {"question_id": 6, "answer": "WebSockets for live bidirectional communication"},
    {"question_id": 7, "answer": "A completed product ready for users"}
  ]
}

Response 200 OK

{
  "goal_id": "goal_0001",
  "goal": "build a full-stack web app for project management with team collaboration",
  "intent": "technical",
  "roadmap": {
    "title": "Your Personalized Roadmap: build a full-stack web app...",
    "overview": "A 12-week journey (5-10 hours/week) across 4 phases.",
    "phases": [
      {
        "id": 1,
        "title": "Architecture & Planning",
        "description": "Design the system architecture, choose your tech stack...",
        "duration_weeks": 3,
        "deadline": "2026-08-19 (buffer: 2026-08-26)",
        "buffer_days": 7,
        "objectives": [
          "Design system architecture and component diagram",
          "Choose tech stack and dependencies",
          "Set up development environment and CI/CD",
          "Define API contracts and data models"
        ],
        "resources": [
          {
            "title": "How to Build a Project Management App: Step-by-Step",
            "url": "https://example.com/tutorial",
            "resource_type": "article",
            "description": "A comprehensive guide to building..."
          }
        ],
        "deliverables": [
          "Architecture document with diagrams",
          "Tech stack decision record",
          "Development environment with CI/CD"
        ],
        "completion_type": "foundation",
        "is_completed": false
      }
    ],
    "total_duration_weeks": 12,
    "total_buffer_days": 28
  },
  "created_at": "2026-07-29T12:00:00Z",
  "status": "active",
  "completed_phases": 0,
  "total_phases": 4,
  "score": 0
}

Status Codes

Error Codes

GET /goals/:goal_id

Retrieves the goal status and full roadmap (if answers have been submitted).

Response 200 OK

{
  "goal_id": "goal_0001",
  "goal": "build a full-stack web app for project management with team collaboration",
  "intent": "technical",
  "roadmap": { ... },
  "created_at": "2026-07-29T12:00:00Z",
  "status": "active",
  "completed_phases": 0,
  "total_phases": 4,
  "score": 0
}

Status field values: "active" (in progress), "completed" (all phases completed).

Status Codes

Error Codes

GET /goals/leaderboard

Returns all goals sorted by score (descending). Max 50 entries.

Response 200 OK

Returns a JSON array (list) of goal leaderboard entries, sorted by score (descending).

[
  {
    "goal_id": "goal_0001",
    "goal": "build a full-stack web app...",
    "user_name": "Anonymous",
    "score": 0,
    "completed_phases": 0,
    "total_phases": 4,
    "created_at": "2026-07-29T12:00:00Z"
  }
]

POST /goals/quick

One-shot endpoint that creates a goal and generates a full roadmap immediately without asking questions. Uses sensible defaults (3-month timeline, 5-10 hours/week).

Request Body

{
  "goal": "build a full-stack web app for managing personal finances"
}

Response 200 OK

{
  "goal_id": "goal_0002",
  "goal": "build a full-stack web app for managing personal finances",
  "intent": "technical",
  "resource_count": 12,
  "roadmap": {
    "title": "Your Personalized Roadmap: build a full-stack web app...",
    "overview": "A 12-week journey (5-10 hours/week) across 4 phases.",
    "phases": [ ... ],
    "total_duration_weeks": 12,
    "total_buffer_days": 28
  },
  "created_at": "2026-07-29T12:00:05Z",
  "status": "active",
  "completed_phases": 0,
  "total_phases": 4,
  "score": 0
}

resource_count represents the total distributed resources curated across all roadmap phases.

Input validation: Goal must be at least 3 characters.

POST /goals/:goal_id/phases/:phase_id/complete

Marks a specific 1-indexed phase as completed, recalculating progress score (+100 pts per phase, +500 bonus pts upon 100% completion).

Request Body None (empty body).

Response 200 OK

{
  "goal_id": "goal_0001",
  "goal": "build a web app",
  "completed_phase_id": 1,
  "completed_phases": 1,
  "total_phases": 4,
  "score": 100,
  "status": "active",
  "roadmap": { ... }
}

POST /goals/:goal_id/progress

Sets specific phase completion status via JSON payload.

Request Body

{
  "phase_id": 1,
  "is_completed": true
}

Response 200 OK

{
  "goal_id": "goal_0001",
  "goal": "build a web app",
  "phase_id": 1,
  "is_completed": true,
  "completed_phases": 1,
  "total_phases": 4,
  "score": 100,
  "status": "active",
  "roadmap": { ... }
}

Domain-Specific Question Banks

The /goals endpoint detects the goal's domain using keyword analysis and returns tailored questions.

All questions are "type": "single_choice" with curated options.

Roadmap Structure

The generated roadmap contains:

Phase Fields

Resource Fields

Phase Sequencing by Domain

Deadline Calculation

Deadlines are computed from the current system time at request time, not hardcoded. The timeline answer (Q1) determines total duration:

Each phase gets equal weeks (total_weeks / phases). Each phase has a hard deadline + 7-day buffer.

Resource Curation

Resources are sourced from the search API (GET /search?q=\{goal\}&limit=20) — the same engine used for web search. Results are categorized by URL pattern:

• youtube.com, youtu.be, vimeo.com → "video"
• /docs/, /api/, /reference/, /wiki/ → "documentation"
• arxiv.org, researchgate.net, acm.org, ieee.org → "paper"
• Everything else → "article"

Resources are distributed round-robin across phases. If the search returns 20 results for a 4-phase roadmap, each phase gets 5 resources.

If search fails (timeout or zero results), fallback resources with Google search links are generated per phase.

Intent Classification

Goals are classified using the intent engine (same endpoint used by /search) or keyword detection. Observed intent values (verified live, 2026-08-05) include learning (e.g. "learn to build a privacy-first search engine using Rust"), creative-writing (e.g. "write a novel in 6 months"), and technical. Other documented goal domains include ai-ml, web-app, api-backend, mobile, systems, devops, research, creative-design, business, lifestyle, general-tech, and general.

Note on question count (verified): a creative-writing goal returned total_questions: 4 (timeline, hours, 2× free_text). The domain-specific question-bank table below lists creative-writing as 6 questions — this may not match the live generator, which can emit a smaller tailored set. Treat the per-domain counts as descriptive, not a hard contract.

The classification uses keyword detection first (fast path), then falls back to the intent engine HTTP call.

Examples

Create a goal and get questions (AI/ML domain)

curl -s -X POST "http://localhost:4000/goals" \
  -H "Content-Type: application/json" \
  -d '{"goal":"build a recommendation engine using deep learning"}' | jq

Create a goal and get questions (Research domain)

curl -s -X POST "http://localhost:4000/goals" \
  -H "Content-Type: application/json" \
  -d '{"goal":"research and publish a paper on transformer optimization"}' | jq '.questions'

Submit answers and get a roadmap

GOAL_ID=$(curl -s -X POST "http://localhost:4000/goals" \
  -H "Content-Type: application/json" \
  -d '{"goal":"build a mobile fitness tracking app"}' | jq -r '.goal_id')

curl -s -X POST "http://localhost:4000/goals/$GOAL_ID/answers" \
  -H "Content-Type: application/json" \
  -d '{
    "answers": [
      {"question_id": 1, "answer": "3 months — Quarter project"},
      {"question_id": 2, "answer": "10-20 hours — Half-time commitment"},
      {"question_id": 3, "answer": "Cross-platform (React Native, Flutter)"},
      {"question_id": 4, "answer": "Custom REST/GraphQL API"},
      {"question_id": 5, "answer": "Full offline-first with background sync"},
      {"question_id": 6, "answer": "A working prototype I can demo"}
    ]
  }' | jq '.roadmap.phases[] | {id, title, deadline, completion_type}'

Get goal status

curl -s "http://localhost:4000/goals/goal_0001" | jq '{status, completed_phases, total_phases}'

Quick one-shot roadmap

curl -s -X POST "http://localhost:4000/goals/quick" \
  -H "Content-Type: application/json" \
  -d '{"goal":"build a rust web framework"}' | jq '.roadmap.phases[].title'

Leaderboard

curl -s "http://localhost:4000/goals/leaderboard" | jq '.entries[] | {goal, total_phases}'

Extract phase resources (with jq)

# Get all resources across all phases
curl -s -X POST "http://localhost:4000/goals/quick" \
  -H "Content-Type: application/json" \
  -d '{"goal":"learn kubernetes"}' | jq '.roadmap.phases[].resources[] | {title, resource_type, url}'

Goals Architecture

Client → Gateway (port 4000)
              │
              ├→ POST /goals
              │     ├── classify_goal() → Intent Engine (port 3005)
              │     ├── search_resources() → Search API (localhost:4000/search)
              │     ├── generate_questions() → domain detection → question bank
              │     └── GoalStore (in-memory HashMap)
              │
              ├→ POST /goals/:id/answers
              │     ├── generate_roadmap() → phase_content (domain-aware)
              │     ├── curate_resources() → round-robin distribution
              │     └── GoalStore.update_roadmap()
              │
              ├→ POST /goals/quick
              │     ├── classify_goal() + search_resources()
              │     ├── default_answers (timeline=3mo, hours=5-10)
              │     └── generate_roadmap() → immediate result
              │
              ├→ GET /goals/:id
              │     └── GoalStore.get()
              │
              └→ GET /goals/leaderboard
                    └── GoalStore.leaderboard() → sorted by score

Data Flow: Goals are stored in-memory (non-persistent across restarts). Resources are fetched in real-time from the search API. The intent engine classifies each goal for phase content customization. Deadlines are computed from the current system time + user's timeline answer.