# Data dictionary

Every table in the pinned database — its layer, what it holds, its columns, its row count and which API context reads it — generated from the database itself.

- Page: <https://docs.gurbanisoul.com/data/data-dictionary/>
- Updated: 2026-10-28
- From the Sri Guru Granth Sahib Ji — Knowledge Base wiki. Sri Guru Granth Sahib Ji is quoted verbatim, cited by Ang, and never altered.

---

<!-- generated by tools/gen_data_dictionary.py from the pinned database — do not edit; run the generator -->
Every table in the pinned database: 40 tables, of which 4 are full-text indexes (their 19 internal shadow tables are not listed). The page is generated from the database by `tools/gen_data_dictionary.py`, and CI fails if it differs from the database the platform pins, so the counts and columns below are those of the dataset in the lock.

| The pinned dataset | |
|---|---|
| Built by | [`Algorythmos-AI/sggs-data`](https://github.com/Algorythmos-AI/sggs-data) at `efbf74e4` |
| `DATASET_VERSION` (the lock) | `1.3.0` |
| Database build version (`meta.version`) | `1.3.0`, built 2026-10-26 |
| SHA-256 | `4e6777c54140d7c4…` |
| Size | 103.6 MiB |
| Lines · Angs | 60,658 · 1,430 |

Which version is which, and how a new dataset arrives, is on [the dataset pin](https://docs.gurbanisoul.com/data/dataset-pin/). The
database is opened read-only by the API; **Read by** names the bounded contexts that declare the table
(`TABLES` in each module, enforced by `webapp/tests/test_declared_tables.py`).

## Scripture and structure

The verbatim text and the structure derived from it. Everything else refers to these rows.

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`authors`](#authors) | 36 | insights, reader | Each contributor with the Ang span and number of lines attributed. |
| [`lines`](#lines) | 60,658 | insights, reader, search, verify | One row per printed line: the verbatim Gurmukhi with its Ang, raag, section, author and composition; the source of truth ([anatomy of a line record](https://docs.gurbanisoul.com/data/line-record/)). |
| [`meta`](#meta) | 9 | reader | Facts about the build as key/value pairs: version, build date, edition, totals. |
| [`raags`](#raags) | 31 | insights, knowledge, reader | Each raag with its Ang span, line and shabad counts, Roman name and order in the Granth. |
| [`sections`](#sections) | 15 | reader | Each section of the Granth with its Ang span and line count. |
| [`word_freq`](#word_freq) | 29,312 | search | Every distinct Gurmukhi word and how often it occurs. |

### authors

`name` TEXT · `first_ang` INT · `last_ang` INT · `n_lines` INT

### lines

`id` INTEGER · `ang` INT · `pdf_page` INT · `raag` TEXT · `section` TEXT · `author` TEXT · `comp_type` TEXT · `ghar` TEXT · `comp_id` INT · `line_no` INT · `is_rahao` INT · `is_header` INT · `markers` TEXT · `gurmukhi` TEXT · `text` TEXT · `translit` TEXT · `translit_norm` TEXT · `fl_g` TEXT · `fl_r` TEXT · `skeleton` TEXT · `stanza_index` INT · `pada_total` INT · `source_category` TEXT

### meta

`key` TEXT · `value` TEXT

### raags

`name` TEXT · `first_ang` INT · `last_ang` INT · `n_lines` INT · `n_shabads` INT · `roman` TEXT · `seq` INT

### sections

`name` TEXT · `first_ang` INT · `last_ang` INT · `n_lines` INT

### word_freq

`word` TEXT · `n` INT

## Search indexes

Built from the scripture tables at rebuild time; the search and verify contexts read them.

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`canon_tokens`](#canon_tokens) | 24,676 | search | Every distinct token of the transliteration, used to tell a known Roman word from a misspelling. |
| [`fts`](#fts) | 60,658 | reader, search, verify | Full-text index over `lines`: the text, transliteration, Roman fold, first letters and consonant skeleton. |
| [`fts_shabad`](#fts_shabad) | 4,528 | search | Full-text index over each composition's Roman consonant skeleton, for queries that span lines. |
| [`fts_tri`](#fts_tri) | 55,508 | — | Trigram index over the transliteration, for partial and misspelt Roman queries. |
| [`variants`](#variants) | 79,840 | search | Spelling and phonetic variants, each mapped to the Gurmukhi word it stands for, with frequency, score and type. |

### canon_tokens

`token` TEXT

### fts

FTS5 index; tokenizer `unicode61`, with 18 Gurmukhi signs counted as part of a token.

`text` · `translit` · `translit_norm` · `fl_g` · `fl_r` · `skeleton`

### fts_shabad

FTS5 index; tokenizer `unicode61` (the default).

`tnorm` · `comp_id`

### fts_tri

FTS5 index; tokenizer `trigram`.

`translit` · `line_id`

### variants

`variant` TEXT · `gurmukhi` TEXT · `translit` TEXT · `freq` INT · `score` REAL · `rtype` TEXT

## The English translation

A separate, labelled layer. It never replaces or mixes with the Gurmukhi.

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`fts_en`](#fts_en) | 58,039 | search | Full-text index over the English translation. |
| [`sources`](#sources) | 2 | — | Where each translation comes from: attribution, licence terms and the date ingested. |
| [`translations`](#translations) | 58,039 | insights, reader, search | The English translation, one row per translated line, with its source and match quality. |

### fts_en

FTS5 index; tokenizer `unicode61` (the default).

`text` · `line_id`

### sources

`source_id` TEXT · `kind` TEXT · `attribution` TEXT · `license` TEXT · `ingest_date` TEXT

### translations

`line_id` INT · `lang` TEXT · `source` TEXT · `text` TEXT · `match_quality` TEXT

## Banis (Nitnem)

The daily prayers as ordered references to lines, plus the few lines that come from outside the Granth.

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`bani_lines`](#bani_lines) | 7,910 | reader | The ordered lines of each bani, each a reference to a Granth line or to an extra line. |
| [`banis`](#banis) | 23 | reader | Each bani and variant: titles, category, order, length and where it comes from. |
| [`extra_lines`](#extra_lines) | 1,191 | reader | Lines of banis that are not in the Granth, with their source and review status. |

### bani_lines

`bani_id` INTEGER · `seq` INTEGER · `line_group` INTEGER · `line_id` INTEGER · `extra_id` INTEGER

### banis

`bani_id` INTEGER · `key` TEXT · `variant` TEXT · `is_default` INTEGER · `title_gm` TEXT · `title_en` TEXT · `category` TEXT · `order_no` INTEGER · `n_lines` INTEGER · `n_groups` INTEGER · `has_extra` INTEGER · `estimated_minutes` INTEGER · `description_en` TEXT · `source_label` TEXT

### extra_lines

`extra_id` INTEGER · `source` TEXT · `panna` INTEGER · `gurmukhi` TEXT · `translit` TEXT · `is_header` INTEGER · `shabados_line_id` TEXT · `review_status` TEXT

## Insights (precomputed analytics)

Descriptive statistics over structure, the theme tags and the English. See [the insights context](https://docs.gurbanisoul.com/architecture/insights-context/).

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`author_analytics`](#author_analytics) | 36 | insights | Per-author counts and stylometry of the English translation, with a reliability flag. |
| [`author_dialogues`](#author_dialogues) | 17 | insights | A Guru's salok set directly after a Bhagat's salok in the Kabir and Farid collections, by line range. |
| [`author_distinctive_terms`](#author_distinctive_terms) | 195 | insights | The Gurmukhi words most characteristic of each voice (log-odds; pen-names and function words excluded). |
| [`author_facts`](#author_facts) | 36 | insights | Per-voice facts from the Gurmukhi: one verse count, Angs present (bit mask), raags, raags no other voice uses, stylometry. |
| [`author_mentions`](#author_mentions) | 249 | insights | Verses in which one voice names another: Bhagats named together, the Gurus praised by the Bhatts. |
| [`author_resonance`](#author_resonance) | 679 | insights | How often one voice's lines land nearest another's, as lift. |
| [`concept_lines`](#concept_lines) | 62,221 | insights, reader, search | Which lines carry which theme. |
| [`concepts`](#concepts) | 54 | insights, reader, search | The curated themes. |
| [`line_neighbors`](#line_neighbors) | 532,212 | insights | For each line, the lines whose English is closest by TF-IDF cosine (0.30 and above). |
| [`raag_analytics`](#raag_analytics) | 31 | insights | Per-raag counts and theme emphasis. |
| [`shabad_neighbors`](#shabad_neighbors) | 43,840 | insights | For each composition, the ten with the most similar theme profile. |
| [`theme_fingerprint`](#theme_fingerprint) | 2,316 | insights | How strongly each author or raag carries each theme, as lift over the whole Granth. |
| [`theme_network`](#theme_network) | 2,528 | insights | Theme pairs that occur in the same shabads, with PPMI and Jaccard. |
| [`vaar_units`](#vaar_units) | 1,443 | insights | Each Vaar's saloks and pauris in reading order. |
| [`vaars`](#vaars) | 22 | insights | The Vaars: span, pauri and salok counts, and who wrote each part. |

### author_analytics

`author` TEXT · `n_lines` INTEGER · `n_shabads` INTEGER · `n_raags` INTEGER · `n_tokens` INTEGER · `mattr_100` REAL · `hapax_pct` REAL · `avg_words_line` REAL · `avg_lines_shabad` REAL · `top_themes` TEXT · `is_reliable` INTEGER

### author_dialogues

`seq` INTEGER · `bhagat` TEXT · `guru` TEXT · `ang` INTEGER · `bhagat_first_id` INTEGER · `bhagat_last_id` INTEGER · `guru_first_id` INTEGER · `guru_last_id` INTEGER

### author_distinctive_terms

`author` TEXT · `term` TEXT · `z_score` REAL · `count_author` INTEGER · `count_rest` INTEGER · `rank` INTEGER

### author_facts

`author` TEXT · `n_verses` INTEGER · `n_compositions` INTEGER · `n_rahao` INTEGER · `n_angs` INTEGER · `first_ang` INTEGER · `last_ang` INTEGER · `first_line_id` INTEGER · `ang_mask` TEXT · `raags_json` TEXT · `signature_raags_json` TEXT · `outside_raag_verses` INTEGER · `chhap_count` INTEGER · `n_tokens` INTEGER · `mattr_100` REAL · `words_per_verse` REAL

### author_mentions

`line_id` INTEGER · `ang` INTEGER · `src_author` TEXT · `mentioned` TEXT · `form` TEXT

### author_resonance

`src_author` TEXT · `dst_author` TEXT · `edges` INTEGER · `mean_score` REAL · `lift` REAL

### concept_lines

`concept` TEXT · `line_id` INT · `term` TEXT

### concepts

`concept` TEXT · `gurmukhi_terms` TEXT · `description` TEXT

### line_neighbors

`line_id` INTEGER · `neighbor_id` INTEGER · `score` REAL

### raag_analytics

`raag` TEXT · `n_lines` INTEGER · `n_shabads` INTEGER · `n_authors` INTEGER · `dominant_author` TEXT · `dominant_author_pct` REAL · `n_themes` INTEGER · `top_themes` TEXT

### shabad_neighbors

`comp_id` INTEGER · `neighbor_comp_id` INTEGER · `rank` INTEGER · `score` REAL

### theme_fingerprint

`entity_type` TEXT · `entity_id` TEXT · `concept` TEXT · `n_tagged` INTEGER · `entity_total` INTEGER · `entity_rate` REAL · `corpus_rate` REAL · `lift` REAL · `is_reliable` INTEGER

### theme_network

`source` TEXT · `target` TEXT · `shabad_count` INTEGER · `source_n` INTEGER · `target_n` INTEGER · `ppmi` REAL · `jaccard` REAL

### vaar_units

`vaar_id` INT · `seq` INT · `kind` TEXT · `comp_id` INT · `author` TEXT · `n_lines` INT · `pauri_no` INT · `first_line_id` INT · `ang` INT · `theme` TEXT

### vaars

`vaar_id` INTEGER · `raag` TEXT · `roman` TEXT · `first_ang` INT · `last_ang` INT · `n_pauris` INT · `n_saloks` INT · `pauri_author` TEXT · `salok_authors` TEXT · `cross_author` INT · `title` TEXT

## Knowledge (timing and forms)

Attributed scholarship about the text. See [the knowledge context](https://docs.gurbanisoul.com/architecture/knowledge-context/).

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`raag_timing_claims`](#raag_timing_claims) | 46 | knowledge | When each raag is sung according to a cited source: pahar, season or occasion, with confidence. |
| [`shabd_musical_markers`](#shabd_musical_markers) | 4,531 | knowledge | Ghar, partaal, rahao and other musical markers stated in a composition's heading. |
| [`shabd_poetic_genre`](#shabd_poetic_genre) | 57 | knowledge | A composition's named genre when its heading states it. |
| [`shabd_raag_map`](#shabd_raag_map) | 4,531 | knowledge | Each composition's raag, first Ang and first line. |
| [`shabd_structural_form`](#shabd_structural_form) | 4,531 | knowledge | A composition's form when its heading states it; otherwise NULL, never guessed. |
| [`timing_sources`](#timing_sources) | 5 | knowledge, reader | The sources behind the timing claims and their tradition. |

### raag_timing_claims

`id` INTEGER · `raag_name` TEXT · `claim_type` TEXT · `pahar` INTEGER · `time_start` TEXT · `time_end` TEXT · `season` TEXT · `occasion` TEXT · `source_id` INTEGER · `confidence` TEXT · `notes` TEXT

### shabd_musical_markers

`comp_id` INTEGER · `ghar` INTEGER · `partaal` INTEGER · `has_rahao` INTEGER · `has_rahao_dooja` INTEGER · `dhunni` TEXT · `jati` TEXT · `source_label` TEXT

### shabd_poetic_genre

`comp_id` INTEGER · `genre` TEXT · `source_label` TEXT

### shabd_raag_map

`comp_id` INTEGER · `raag_name` TEXT · `first_ang` INTEGER · `first_line_id` INTEGER

### shabd_structural_form

`comp_id` INTEGER · `form` TEXT · `pada_count` INTEGER · `source_label` TEXT

### timing_sources

`id` INTEGER · `name` TEXT · `tradition` TEXT · `url` TEXT · `notes` TEXT

## Build records

Bookkeeping written by the rebuild; no route serves it except as metadata.

| Table | Rows | Read by | What it holds |
|---|--:|---|---|
| [`analytics_meta`](#analytics_meta) | 12 | insights | How the analytics were built: methods, framing, build dates. |
| [`timing_migrations`](#timing_migrations) | 1 | — | The timing layer's applied migrations and whether the scripture baseline was verified. |

### analytics_meta

`key` TEXT · `value` TEXT

### timing_migrations

`id` INTEGER · `name` TEXT · `applied_at` TEXT · `baseline_verified` INTEGER