From JSON to one small response.

A record becomes independently readable values

Consider a product record with a title, price, description and specifications. A list screen might need only its title and price. A detail screen might need the description later. The file can hold the complete collection while each screen asks for the fields it uses.

{
  "item-101": {
    "title": "Oak desk",
    "price": 129,
    "description": "A longer description of the desk...",
    "specifications": { "width_cm": 120, "material": "oak" }
  }
}

RIDA writes each top-level field value as compact UTF-8 JSON. With the default codec, each value gets its own raw deflate stream. An encrypted build seals that compressed stream with AES-256-GCM. Values sit consecutively in the data file; their boundaries are recorded in the separate index.

title          → JSON → deflate → optional encryption → value block
price          → JSON → deflate → optional encryption → value block
description    → JSON → deflate → optional encryption → value block
specifications → JSON → deflate → optional encryption → value block

The data file has no header to fetch first. The reader knows the codec and boundaries from its local index, so it can decode an individual block without downloading the preceding blocks.

The index supplies an address before the request

Your app supplies a record ID and field name. The reader locates the record’s catalog row, finds the field’s position and length, and turns those numbers into an HTTP byte range.

These addresses illustrate the calculation rather than a live dataset. For a start of 14,231 and length of 143, the inclusive end is 14,373. The app sends one range for the selected stored block.

GET /ItemsData.rida
Range: bytes=14231-14373

206 Partial Content
Content-Range: bytes 14231-14373/TOTAL_FILE_BYTES
Content-Length: 143

The body contains those 143 stored bytes. The reader decrypts them when needed, decompresses them when needed, parses their JSON and returns the original value. HTTP headers and network overhead are additional to the body; a 143-byte value does not mean the whole exchange uses exactly 143 bytes.

The four files have different jobs

The data holds values and is the file you serve. The index holds their addresses and decoding settings. The optional map associates folder names with numbered IDs. The optional key enables decryption. Your host does not need to understand the RIDA format to serve its byte ranges.

1. Publisher: prepares JSON and builds a matching set of files.
2. Storage or CDN: serves the data file through HTTPS byte ranges.
3. Application: carries the index and, where needed, the map.
4. Reader: finds the byte range locally, fetches the stored value and decodes it.

A numbered ID such as 1/1/5 identifies a position in a folder tree. The map lets your reader resolve a meaningful folder name to that position. The index, rather than the map, contains the byte addresses.

Opening a remote reader with its local index requires no network request. This assumes the app has already received the index; distributing that file to users still involves a download or bundling it with the application.

A reader stays open while the user browses

Open one reader for the dataset and reuse it for subsequent actions. A list view can request a summary, and selecting an item can request its description. The catalog is already open and the reader can reuse its HTTP connection.

import rida

with rida.RemoteRidaFile(
    "https://data.example.com/ItemsData.rida",
    index="ItemsIndex.rida",
) as data:
    summary = data.get_many("item-101", ["title", "price"])
    description = data.get("item-101", "description")
    whole_record = data.get_record("item-101")

get_many selects several fields from one record. It merges nearby stored blocks when the intervening bytes cost less than another response header. A large gap can produce separate requests. get_record retrieves all the record’s fields; it is useful when a screen genuinely needs the complete record.

Reusing the reader avoids repeatedly loading its catalog and starting a new connection. The timing depends on the user’s network, the host, the requested value’s size and local decoding work; a benchmark is not a guaranteed latency for every device.

Small reads depend on how you divide the JSON

RIDA’s smallest addressable piece is a complete stored field value. It works for large values too, but requesting a large value retrieves that entire block.

If specifications contains a 20 MB object, asking for that field returns all its contents. The reader does not jump straight to width_cm inside the object. If the app regularly needs width on its own, store width as a separate field. If users read a document by section, separate section records can make each section independently accessible.

Choose boundaries according to what the application requests. A tiny value per field can create more index entries, encryption overhead and HTTP requests. Selected fields that are usually needed together can be fetched with get_many. Large combined values are useful when the app always needs them together.

Hosting, caching and changing the dataset

RIDA chooses which bytes to request. The storage provider or CDN decides how it serves and caches those bytes. RIDA’s local index cache stores catalog information; it is separate from a CDN’s cache of the data file.

A CDN might cache the whole object, use its own slicing system or fetch from its origin. Its file-size limits, cache configuration and billing still apply. A small range read does not by itself guarantee a cache hit or free delivery.

When the source changes, rebuild and distribute the new index with the new data file. Byte addresses from one build should not be used against another. Versioned URLs let a reader use an explicitly matching version.

RIDA includes Python 2.0.2 and Node.js 2.0.1 readers for its static dataset format. It does not provide accounts, an upload service, subscriptions, full-text search or a browser JavaScript reader. The Build guide explains preparing files, and the Read guide covers local and remote lookups in detail.