Build a dataset
without a database.

Choose a JSON shape your app can read

RIDA works with any subject: products, customer records, research, translations or application content. Each record is a JSON object. Its fields become the values your app can request by record ID and field name.

An object keyed by record ID

{
  "item-101": { "title": "Desk", "price": 129, "description": "Oak desk" },
  "item-102": { "title": "Chair", "price": 59, "description": "Folding chair" }
}

rida build items.json ItemsData.rida

An array with an ID field

[
  { "id": "item-101", "title": "Desk", "price": 129 },
  { "id": "item-102", "title": "Chair", "price": 59 }
]

rida build items-array.json ItemsData.rida --id-key id

A folder with one record per file

records/
  item-101.json
  item-102.json

rida build records/ ItemsData.rida

Keep IDs predictable and unique. Use consistent field names for the same kind of value. Different records can omit fields; they do not need to hold identical data.

Understand the scanner’s choices

The scanner measures records, field names, field density and stored sizes for the selected value codec. The group decision and the dense/sparse index decision answer different questions.

Several groups: which fields should be stored?

JSON groups found: 3
Distinct fields across all groups: 34
Fields shared by every group: 28

group         records   fields   not shared
products          300       34            6
services          300       28            0
bundles           300       34            6

field 'weight': not in services

Example estimates for 900 records:
Separate groups: 382.7 KB data, 93.9 KB catalog
Common fields:   333.8 KB data, 82.8 KB catalog, 100.0% density
All fields:      382.7 KB data, 98.7 KB catalog, 94.1% density

These illustrative sizes come from the manual’s 900-record sample. The builder offers three ways forward and estimates data and catalog sizes for your own input:

1. One file per group: each group keeps its own fields.
2. Common fields: one combined file containing only fields every group shares. In this example, six group-specific fields, including weight, are not stored, even for products that have them.
3. All fields (union): one combined file retaining every field any group uses. A record with no weight stores nothing for that field.

Choose common only when the omitted fields are not needed. Choose union when you want to preserve them. Group comparison samples a few hundred records per group; its sizes are estimates, not the final build totals.

Dense or sparse: how should the index be arranged?

Dense gives each record a slot for every field and tends to suit records that contain most fields. Sparse lists only fields actually present and tends to suit uneven records. RIDA measures both and recommends a layout. This changes the catalog representation, not which values are kept.

rida scan records/
rida build records/ ItemsData.rida --mode dense
rida build records/ ItemsData.rida --mode sparse

Select the records folder

If the input contains a configuration file alongside a folder of records, select the records group. --records-dir items searches that subfolder and its descendants; unrelated groups outside it are excluded.

rida scan ./my-data --records-dir items
rida build ./my-data ItemsData.rida --records-dir items --yes

--yes accepts the build confirmation for an automated command. The builder refuses to overwrite an existing output file.

IDs, compression and memory

Numbered folder IDs identify each record by its position in the folder tree. Repeated filenames in separate folders remain distinct. The map preserves the original folder names so the reader can use human names.

--id-scheme named uses filenames as record IDs instead, adds suffixes such as 1-2 for repeats, and writes no folder map. Records use numeric rather than plain alphabetical filename ordering.

RIDA reads one JSON record at a time. The current record and the address catalog still use RAM, so an individual record must fit in memory. An explicit --mode dense or --mode sparse stages only that catalog layout, reducing peak catalog memory and CPU work.

Very small or already compressed values can become larger under deflate. Scan with the codec you intend to build so its stored-size estimates reflect that choice.

rida scan items.json --value-codec plain
rida build items.json ItemsData.rida --value-codec plain

Build a complete dataset from start to finish

Start in the folder containing your JSON input. For a first build, run the interactive command so you can see the detected groups, measured sizes and remaining choices before confirming the output.

rida build ./records

The builder identifies the input, asks which records to use when needed, and lets you name the output. Choose how values are stored and whether they are encrypted before the full value measurements. It then recommends an index layout and asks for the remaining settings and final confirmation.

You can leave the data URL blank and host the file later. In that case, give a remote reader the matching local index explicitly. The chosen URL does not upload the data or create a CDN account.

See what the build is doing

Scanning field names: 100% | 479,552/479,552
Measuring values:    100% | 479,552/479,552
Compressing catalog: 1/1 (100%)
Writing RIDA data:   100% | bytes written/total bytes

Field scanning identifies the available fields. Measuring values encodes, compresses and, for an encrypted build, seals the values so sizes reflect the selected storage settings. Catalog compression packs the address index. The final write assembles the stored data. When a total is not known yet, progress reports the count as records are read.

Open the result before publishing it

An example folder build produces the following files. The map is optional and the key is created only when encryption is enabled.

ItemsData.rida    stored values
ItemsIndex.rida   catalog and decoding information
ItemsMap.json     folder names and numbered IDs, when needed
ItemsKey.rida     private key, for an encrypted build

Use the actual filenames printed by the build; an output named Items.rida keeps that data filename. Inspect the index, list its fields and read a known record. Folder builds usually use numbered IDs; object and array inputs use the IDs supplied in the input.

rida inspect ItemsData.rida
rida fields ItemsData.rida
rida folders ItemsData.rida
rida get ItemsData.rida 1 title

The ID 1 above is an example, not a universal record ID. For a nested folder it might be 1/1/1; for the object example earlier it is item-101. Choose an ID your particular build contains.

Build encrypted values

pip install rida-format
rida build ./records ItemsData.rida --encrypt
rida get ItemsData.rida 1 title --key "$(cat ItemsKey.rida)"

Keep the generated key safely. Upload the data file to your chosen host, put the matching index and optional map with the reader, and arrange key delivery separately if encrypted. The Security guide explains reading with a key and what encryption protects.

Publish a matching set and update it together

RIDA produces static files. Upload the data to storage or a CDN with byte-range support; your application distributes or installs the local index and map. A new user’s device does not inherit the builder computer’s cache.

with rida.RemoteRidaFile(
    "https://data.example.com/v1/ItemsData.rida",
    index="ItemsIndex.rida",
) as data:
    print(data.get("item-101", "title"))

This last example uses the object input with item-101. For a folder input, use that build’s numbered or mapped record ID instead.

Changing the JSON collection means building a new set. Field lengths and positions can change even if record names stay the same. Publish the new data, index and map together, and keep the corresponding key when encrypted. A versioned data URL makes it easier to keep old app versions reading their matching dataset while a new version rolls out.