← Back to API docs

Search API

The Search API provides managed full-text search with App Engine Search semantics: structured documents; text, atom, number, date, and geo fields; a boolean query language; facets; sorting; and pagination. All paths below are relative to a search instance you provision in the console.

Authentication and base URL

Send requests to https://api.altengine.net with an organization API key as a bearer token. Reads require a read grant, writes a write grant, and deletes a full grant — see Authentication.

Authorization: Bearer ae_yourkeyid.your-api-key-secret

Every path is scoped to an instance and a namespace: /v1/search/{instance}/ns/{namespace}/idx/{index}/…. Within a namespace, documents live in named indexes; the namespace partitions data for multi-tenancy. Use _default for the default namespace (it can't be an empty path segment).

Search requests are rate-limited per instance to a sustained 100 requests per second (6,000 per minute) by default, with a matching burst. Each search instance has its own budget, so one busy app never throttles another — and Search is separate from the Channel API, so a busy channel workload won't touch your search traffic either. The limit spans every Search request across an instance's indexes and keys. You can set a lower limit per instance in the dashboard (Search → your instance → Settings); a configured value can only reduce the rate below the plan maximum, never raise it. Over the limit, a request returns 429 with a RATE_LIMITED code; back off and retry.

Endpoints

Documents

A document has a string id; an optional numeric rank (used as the default sort — descending — when a query specifies no sort; when omitted it defaults to the number of seconds since 2011-01-01, the App Engine rank epoch); an optional lang language tag; a list of fields; and an optional list of facets. Fields are multi-valued and dynamic: two documents in the same index may carry different fields.

{
  "id": "f1",
  "rank": 12345,
  "lang": "en",
  "fields": [
    { "name": "title",    "type": "text",   "value": "Up in the Air" },
    { "name": "genre",    "type": "atom",   "value": "drama" },
    { "name": "rating",   "type": "number", "value": 4 },
    { "name": "released", "type": "date",   "value": "2009-12-04" },
    { "name": "loc",      "type": "geo",    "value": { "lat": 37.77, "lng": -122.41 } }
  ],
  "facets": [
    { "name": "genre", "type": "atom",   "value": "drama" },
    { "name": "year",  "type": "number", "value": 2009 }
  ]
}

Facets are not fields. fields make a value searchable; facets make it countable. Faceting on a value written only as a field returns nothing — not an error, just an empty result, which is easy to misread as "there is no such value". To both search and facet on something, write it in both places, as genre does above.

Field types

Put documents

Send up to 200 documents per request. Putting a document with an existing id replaces it. The response returns the ids written.

curl -X POST https://api.altengine.net/v1/search/catalog/ns/_default/idx/films/documents \
  -H "Authorization: Bearer $ALTENGINE_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "documents": [ { "id": "f1", "fields": [ { "name": "title", "type": "text", "value": "Up in the Air" } ] } ] }'

# → { "ids": ["f1"] }

Bulk ingest

To load a large dataset quickly, fill each request to the 200-document maximum and keep about 6–8 requests in flight per index — that sustains on the order of 2,000 documents per second. Push harder and writes are throttled with a 429; honor the Retry-After — pause briefly and retry the same batch (a put is idempotent by id, so retries are safe). Prefer full 200-document batches over many small puts. Writes to different indexes are independent, so loading multiple indexes at once raises your total throughput.

Get, list, and delete

Paths are relative to /v1/search/catalog/ns/_default/idx/films.

# Batch get by id (missing ids are omitted from the response)
POST …/documents/get
{ "ids": ["f1", "f2"] }
# → { "documents": [ … ] }

# List documents (id order). Params: start_id, include_start, limit, ids_only
GET …/documents?limit=50
# → { "documents": [ … ] }   (or { "ids": [ … ] } when ids_only=true)

# Batch delete by id (requires a full grant)
POST …/documents/delete
{ "ids": ["f1", "f2"] }
# → { "deleted": 2 }

Deleting the last document in an index removes the index itself — put a document again to recreate it (it picks up the instance's current stemming setting automatically). An index is just its documents; there are no per-index settings, so an empty one doesn't linger.

Search

Post a search request to an index. Only query is required; an empty query matches all documents.

{
  "query": "genre:comedy rating > 3",
  "limit": 20,
  "offset": 0,
  "ids_only": false,
  "returned_fields": ["title", "rating"],
  "sort": [{ "expr": "rating", "desc": true, "default": 0 }],
  "facet_discover": 5,
  "facet_refinements": [{ "name": "genre", "value": "scifi" }],
  "total_hits_accuracy": 1000
}

Request fields

Response

{
  "total_hits": 42,
  "total_hits_exact": true,
  "returned": 1,
  "results": [
    {
      "id": "f1",
      "rank": 12345,
      "score": 1.7,
      "document": {
        "id": "f1",
        "fields": [
          { "name": "title", "type": "text", "value": "Up in the Air" }
        ]
      }
    }
  ],
  "cursor": "eyJvIjoyMH0",
  "facets": [
    {
      "name": "genre",
      "type": "atom",
      "values": [
        { "value": "drama",  "count": 12 },
        { "value": "comedy", "count": 8 }
      ]
    }
  ]
}

total_hits is the number of matches, counted exactly only up to total_hits_accuracy. When total_hits_exact is false, total_hits is a lower bound — render it as "N+". Pagination is independent of the count: a cursor is present whenever more pages remain, so keep following it until it is absent.

Query language

The query language mirrors App Engine's Search syntax. Terms are combined with implicit AND.

The ~ stemming operator only matches word variants (~running → run) when stemming is enabled for the instance (Search → your instance → Settings → Stemming; off by default, since it costs extra storage). With it off, ~word behaves like a plain term. Enabling it applies to documents written from then on — existing documents aren't retroactively stemmed, so re-put them to include them.

Facets

Attach facets to documents to enable faceted navigation. Set facet_discover in a search to have altengine return the most common facet values over the matches, then pass the ones a user picks back as facet_refinements to narrow the result set. Atom facets return value counts; number facets return half-open [min, max) ranges with counts.

Snippets

Add a snippet block to a search request to get match-highlighted excerpts back with each result. Each result then carries a snippet map of field name to excerpt.

{
  "query": "rose",
  "snippet": {
    "fields": ["body"],
    "max_tokens": 32,
    "pre_tag": "<b>",
    "post_tag": "</b>",
    "ellipsis": "…"
  }
}

The response adds a snippet object to each result, mapping field name to excerpt:

{ "id": "d1", "rank": 42, "snippet": { "body": "a <b>rose</b> by any other name…" } }

Everything in an excerpt except your pre_tag/post_tag is HTML-escaped, so stored markup in a document comes back inert — an excerpt is safe to render, and the only live tags in it are the ones you asked for (XSS-safe).

Only tokenized fields (text, html, tokenprefix) can be excerpted; atom, number, date and geo are not in the full-text index and are never snippetable (html excerpts from its tag-stripped text). A field with no match is omitted from the map rather than returned empty. A term matched only via the prefix index or via stemming (~) may not produce an excerpt, since excerpts come from the exact-text index. Snippets are computed only when requested.

Synonyms

Synonyms are instance configuration, not part of a search request. Define them in the dashboard (Search → your instance → Synonyms) and every index in the instance picks them up on the next query. They cost no storage and need no reindex — the dictionary rewrites the parsed query, so an edit takes effect immediately. There are two kinds of rule:

• Equivalent sets — every member expands to every other. With laptop, notebook, macbook in one set, laptop finds notebooks and notebook finds laptops.
• One-way rules — the source term expands to each of its targets, but not the reverse. Use them for hypernyms: with shoe → sneaker, trainer, searching shoe finds sneakers, while searching sneaker does not drag in every shoe.

Rules that share a term merge, and expansion is additive — the original term always remains one branch of the OR, so a synonym can only widen a query, never drop a document it already matched. Multi-word members work on both sides: a multi-word member matches consecutive words (or a quoted phrase), longest rule first, and a multi-word target is matched as a phrase. Expansion inherits field scope (title:laptop becomes title:(laptop OR notebook)), applies inside NOT, and never rewrites numeric or date comparisons.

Limits: 200 rules, 20 members per rule, 5 words per member, and 16 KB serialized. The byte cap is the one that binds — the dictionary is read on every search, so its size is a per-query cost.

Computed numeral synonyms

Instead of listing every number, two optional toggles on the same Synonyms page make a number and its written forms equivalent automatically, for the range 1–3999:

• Digits ↔ Roman numerals (2 ↔ ii).
• Digits ↔ spelled-out words (2 ↔ two).

Each toggle is off (the default), one-way (digit → form only), or both directions (also form → digit). So with digit → Roman enabled, searching godfather 2 also finds The Godfather II, but godfather ii is left alone.

The reverse direction is opt-in because it is ambiguous. Single letters are never converted (so iPhone X and Vitamin C stay text), but a few whole words are genuinely valid Roman numerals (e.g. mix = 1009). Spelled-out words use one canonical American spelling (1999 = "one thousand nine hundred ninety nine"), so the year-style "nineteen ninety nine" is not recognized. The two edges are independent and don't chain, and like all synonyms this is additive and needs no reindex.

Query rules (merchandising)

Also instance configuration, managed in the dashboard (Search → your instance → Rules). When a query matches a rule, the rule can pin documents to positions, hide documents, and attach arbitrary data to the response.

The search response then carries rule_data — one entry per matching rule that had data attached:

{ "results": [ … ], "total_hits": 42, "rule_data": [{ "banner": "summer-sale" }] }

• Match condition — a rule fires when the query is exactly its configured text (the whole query, normalized) or contains it (word-boundary aware, so cat matches "cat food" but not "concatenate"). Matching is case- and whitespace-insensitive, and runs against the raw query, before synonym expansion — a rule matches what the user typed.
• Pin — places a document at a 0-based position in the full result list, so paging works naturally and a pinned document doesn't reappear on page 2. A pinned document is returned whether or not it matches the query, but it must exist — a pin naming a deleted document is ignored and doesn't leave a hole. Pinned documents count toward total_hits.
• Hide — excluded from results, total_hits, and facet counts.
• Data — echoed back untouched in rule_data; it never affects matching, counting or pagination.

If a document is both pinned and hidden, hide wins. Every matching rule applies — pins and hides merge, and the first rule to pin a document wins its position.

Limits: 200 rules, 20 pins and 20 hides per search (resolved across all matching rules), and 16 KB serialized.

Field collapsing (distinct)

Add a collapse block to a search request to keep only the top-ranked document(s) per distinct value of a field — "one result per brand", "one film per year".

{
  "query": "shoe",
  "collapse": { "field": "brand", "limit": 1 }
}

• field must be an atom or number field (collapsing on tokenized text is meaningless). An unknown or wrong-typed field is a 400.
• limit is documents kept per group (default 1 = pure dedup, clamped to a max of 10). With limit: 2 you get the two best per value.

Documents with no value for the field are each their own group — never merged together or dropped. A multi-valued document collapses by its minimum value. total_hits and facet counts stay uncollapsed — they reflect matching documents, not groups. Collapse only shapes the returned page, and composes with snippets, facets, and rules.

v1 limitation: collapse can't yet be combined with an explicit sort or scorer (both return a 400) — collapse is rank-ordered for now. It is exact within the top-by-rank window and approximate beyond it, like a field sort.

Cost

Search bills on queries plus the three axes Datastore uses — reads, writes, and stored data; see the pricing page for exact rates.

The practical upshot: beyond the flat per-query fee, cost tracks the work a query actually does. Selective queries over indexed fields are cheap; repeated queries are cheaper still — served from cache, they bill only the query. On the write side, lean documents — only the fields you search on — keep both writes and storage down.

Limits

Index and document ids must be printable ASCII, may not start with !, and may not use the reserved __*__ form.