Type catalog reference

Every registered type, grouped by data category, with its real summary, example, and parameters. Generated from the live registry.

On this page

Every entry below is generated straight from the running type registry, and a test in this project's own release gate fails the moment this page and the registry disagree — what you're reading was true against a live build, not remembered by hand. See the build inventory for the flat progress table this page expands on.

C.1 — Core scalar and temporal types#

Numeric#

numeric.currency#

Money amounts, rounded to whatever the currency's smallest unit is.

Priority P0 · dtype float64 · equivalence continuous

Example params: distribution=LogNormal(family='lognormal', log_mean=3.9, log_sd=0.6) truncate=None currency='USD'

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
currency Literal['AED', 'ARS', 'AUD', 'BHD', 'BIF', 'BRL', 'CAD', 'CHF', 'CLP', 'CNY', 'COP', 'CZK', 'DJF', 'DKK', 'EGP', 'EUR', 'GBP', 'GNF', 'HKD', 'HUF', 'IDR', 'ILS', 'INR', 'IQD', 'ISK', 'JOD', 'JPY', 'KES', 'KMF', 'KRW', 'KWD', 'LYD', 'MXN', 'MYR', 'NGN', 'NOK', 'NZD', 'OMR', 'PEN', 'PHP', 'PLN', 'PYG', 'RON', 'RWF', 'SAR', 'SEK', 'SGD', 'THB', 'TND', 'TRY', 'TWD', 'UAH', 'UGX', 'USD', 'VND', 'VUV', 'XAF', 'XOF', 'XPF', 'ZAR'] 'USD' ISO 4217 code. It sets the precision.

numeric.decimal#

Numbers with a decimal point, to as many places as you say.

Priority P0 · dtype float64 · equivalence continuous

Example params: distribution=Normal(family='normal', mean=3.1, sd=0.4) truncate=Truncation(minimum=0.0, maximum=4.0) decimals=2

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
decimals int 2 Digits kept after the decimal point.

numeric.integer#

Whole numbers drawn from a distribution you choose.

Priority P0 · dtype int64 · equivalence continuous

Example params: distribution=Normal(family='normal', mean=75.0, sd=12.0) truncate=Truncation(minimum=0.0, maximum=100.0)

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.

numeric.measurement#

A number with its unit attached, at a fixed precision.

Priority P0 · dtype string · equivalence continuous

Example params: distribution=Normal(family='normal', mean=72.5, sd=9.0) truncate=Truncation(minimum=35.0, maximum=None) unit='kg' decimals=1

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
unit str — Unit symbol, appended to every value.
decimals int 2 Digits kept after the decimal point.

numeric.percentage#

A share of a whole, written either as 0 to 100 or as 0 to 1.

Priority P0 · dtype float64 · equivalence continuous

Example params: distribution=Beta(family='beta', alpha=5.0, beta=3.0) truncate=None basis='percent' decimals=1

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
basis Literal['percent', 'fraction'] 'percent' Whether the column reads 0-100 or 0-1. The distribution is a proportion either way.
decimals int 1 Digits kept after the decimal point.

numeric.scientific#

Numbers written as a mantissa and an exponent, like 1.23e-09.

Priority P0 · dtype string · equivalence continuous

Example params: distribution=LogNormal(family='lognormal', log_mean=-9.0, log_sd=2.0) truncate=None significant_figures=3

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
significant_figures int 3 Digits shown in the mantissa.

Boolean and constant#

boolean.biased#

A true/false column where you choose how often it is true.

Priority P0 · dtype bool · equivalence categorical

Example params: probability=0.3

Field Type Default Description
probability float 0.5 Probability of drawing true.

constant.literal#

The same fixed text in every row.

Priority P0 · dtype string · equivalence exact

Example params: value='SenSym'

Field Type Default Description
value str — The literal written into every row.

Identifiers#

identifier.check_digit#

An identifier whose last digit validates the ones before it.

Priority P0 · dtype string · equivalence unique

Example params: scheme='luhn' length=16 prefix='4'

Field Type Default Description
scheme Literal['luhn', 'imei', 'ean8', 'ean13', 'isbn13', 'isbn10'] 'luhn' Which check-digit scheme to satisfy.
length int | None None Total digits including the check digit. Only Luhn allows a choice; the other schemes have one length and reject this.
prefix str | None None Fixed leading digits — an issuer range, a bookland prefix, a TAC. The scheme's own default when omitted.

identifier.pattern#

A string built to a pattern you write, like a postcode or a licence plate.

Priority P0 · dtype string · equivalence structural

Example params: pattern='[A-Z]{2}[0-9]{2} [0-9][A-Z]{2}'

Field Type Default Description
pattern str — A bounded regular expression. Quantifiers must have an upper bound: write {0,8} rather than * and {1,8} rather than +.

identifier.sequential#

A running number that counts up from a value you choose -- use it for a row index, a numeric counter, or an auto-increment id.

Priority P0 · dtype int64 · equivalence exact

Example params: start=1000 step=1

Field Type Default Description
start int 1 Value of the first row.
step int 1 Increment between rows. May be negative.

identifier.uuid4#

A random version-4 UUID, formatted the usual way.

Priority P0 · dtype string · equivalence unique

Example params: ``

Categorical#

categorical.likert#

A survey scale coded as whole numbers from 1 upwards.

Priority P0 · dtype int64 · equivalence categorical

Example params: points=5 weights=(0.05, 0.15, 0.3, 0.35, 0.15)

Field Type Default Description
points int 5 Number of scale points.
weights tuple[float, ...] | None None Relative frequency per point, low to high.

categorical.lookup#

A value borrowed from a column in another table, so the two tables link up.

Priority P0 · dtype inherited · equivalence structural

Example params: source=Reference(table='product', column='sku')

Field Type Default Description
source Reference — The column whose values this one draws from — usually a key column of a dimension table.

categorical.weighted#

Picks from a list of labels, as often as you say for each one.

Priority P0 · dtype string · equivalence categorical

Example params: categories=('freshman', 'sophomore', 'junior', 'senior') weights=(0.31, 0.26, 0.23, 0.2)

Field Type Default Description
categories tuple[str, ...] — The category labels.
weights tuple[float, ...] | None None Relative weights, one per category. Uniform when omitted.

color.hair#

A real hair colour -- natural by default, with dyed/fantasy shades available on request.

Priority P0 · dtype string · equivalence categorical

Example params: include_dyed=True weights=None

Field Type Default Description
include_dyed bool False Also include dyed/fantasy shades (blue, pink, purple, green, platinum) alongside the natural ones, each a small real minority rather than an equal share.
weights tuple[float, ...] | None None Relative frequency per colour, in the order this type's own labels are listed. The defaults above when omitted.

colour.value#

A colour from the sixteen basic web colours, written as a name, a hex code, or r,g,b.

Priority P0 · dtype string · equivalence categorical

Example params: representation='hex'

Field Type Default Description
representation Literal['name', 'hex', 'rgb'] 'name' How the colour is written into the column.

Text primitives#

text.paragraphs#

Paragraphs of sentences, separated by a blank line.

Priority P0 · dtype string · equivalence structural

Example params: count=CountRange(minimum=1, maximum=2) sentences_per_paragraph=CountRange(minimum=2, maximum=4) words_per_sentence=CountRange(minimum=6, maximum=14)

Field Type Default Description
count CountRange CountRange(minimum=1, maximum=2) How many paragraphs per row.
sentences_per_paragraph CountRange CountRange(minimum=2, maximum=4) Length of each paragraph.
words_per_sentence CountRange CountRange(minimum=6, maximum=14) Length of each sentence.

text.sentences#

Whole sentences, capitalised and ending in a full stop.

Priority P0 · dtype string · equivalence structural

Example params: count=CountRange(minimum=1, maximum=3) words_per_sentence=CountRange(minimum=6, maximum=12)

Field Type Default Description
count CountRange CountRange(minimum=1, maximum=2) How many sentences per row.
words_per_sentence CountRange CountRange(minimum=6, maximum=14) Length of each sentence.

text.slug#

A URL-safe handle - a few words, a separator, and an optional number.

Priority P0 · dtype string · equivalence unique

Example params: words=2 separator='-' digits=4

Field Type Default Description
words int 2 How many words in the slug.
separator Literal['-', '_', '.'] '-' What goes between the words.
digits int 4 Length of the numeric suffix. 0 omits it.

text.words#

A few lowercase words from the built-in word list.

Priority P0 · dtype string · equivalence structural

Example params: count=CountRange(minimum=2, maximum=4)

Field Type Default Description
count CountRange CountRange(minimum=1, maximum=3) How many words per row.

Temporal#

temporal.arrival#

Event times that come out in order, busier at the hours you say are busy.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: start=datetime.datetime(2024, 3, 1, 0, 0) end=datetime.datetime(2024, 3, 31, 23, 59, 59) hour_weights=(0.2, 0.1, 0.08, 0.06, 0.06, 0.12, 0.35, 0.75, 1.2, 1.6, 1.8, 1.7, 1.5, 1.6, 1.75, 1.65, 1.4, 1.2, 1.1, 1.0, 0.9, 0.7, 0.5, 0.3) weekday_weights=(1.0, 1.05, 1.05, 1.0, 0.95, 0.45, 0.35) burstiness=4.0

Field Type Default Description
start <class 'datetime.datetime'> datetime.datetime(2024, 1, 1, 0, 0) When the window opens.
end <class 'datetime.datetime'> datetime.datetime(2024, 12, 31, 23, 59, 59) When the window closes.
hour_weights tuple[float, ...] | None None Relative arrival rate for each hour of the day, midnight first. Flat when omitted.
weekday_weights tuple[float, ...] | None None Relative arrival rate for each day of the week, Monday first. Flat when omitted.
burstiness float | None None How much one hour's rate varies from the next beyond what the weights say. Smaller is burstier. Omit for an exactly Poisson process.

temporal.birthdate#

A date of birth that matches the age distribution you describe.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: distribution=Normal(family='normal', mean=21.0, sd=3.5) truncate=Truncation(minimum=17.0, maximum=45.0) as_of=datetime.date(2026, 1, 1)

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
as_of <class 'datetime.date'> datetime.date(2026, 1, 1) The date the ages are measured on.

temporal.business_date#

A date that only lands on working days, with holidays excluded.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: start=datetime.date(2024, 1, 1) end=datetime.date(2024, 12, 31) weekdays=(0, 1, 2, 3, 4) holidays=(datetime.date(2024, 1, 1), datetime.date(2024, 7, 4), datetime.date(2024, 12, 25))

Field Type Default Description
start <class 'datetime.date'> datetime.date(2024, 1, 1) Earliest date, inclusive.
end <class 'datetime.date'> datetime.date(2024, 12, 31) Latest date, inclusive.
weekdays tuple[int, ...] (0, 1, 2, 3, 4) Which days of the week are eligible. Monday is 0.
holidays tuple[datetime.date, ...] () Dates to exclude, whatever day of the week they fall on.

temporal.calendar_name#

A day-of-week or month name, in full or abbreviated.

Priority P0 · dtype string · equivalence categorical

Example params: kind='weekday' style='long' weights=(0.19, 0.2, 0.2, 0.19, 0.17, 0.03, 0.02)

Field Type Default Description
kind Literal['weekday', 'month'] 'weekday' Which cycle to name.
style Literal['long', 'short'] 'long' Full name or three-letter abbreviation.
weights tuple[float, ...] | None None Relative frequency per name, Monday first or January first.

temporal.date#

A calendar date picked at random from a range you set.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: start=datetime.date(2023, 9, 1) end=datetime.date(2024, 6, 30)

Field Type Default Description
start <class 'datetime.date'> datetime.date(2020, 1, 1) Earliest date, inclusive.
end <class 'datetime.date'> datetime.date(2024, 12, 31) Latest date, inclusive.

temporal.duration#

How long something took, drawn from a distribution.

Priority P0 · dtype timedelta64 · equivalence continuous

Example params: distribution=LogNormal(family='lognormal', log_mean=1.6, log_sd=0.8) truncate=Truncation(minimum=0.5, maximum=180.0) unit='minutes'

Field Type Default Description
distribution Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf — The family and its natural parameters. Required: a numeric column with no stated distribution is the thing this product exists to replace.
truncate Truncation | None None Optional bounds, in the distribution's own units.
unit Literal['seconds', 'minutes', 'hours', 'days'] 'minutes' The unit the distribution is stated in.

temporal.recurring_schedule#

The next occurrence of a repeating appointment, one per row.

Priority P0 · dtype datetime64 · equivalence exact

Example params: anchor=datetime.datetime(2024, 1, 8, 9, 0) every=2 unit='weeks'

Field Type Default Description
anchor <class 'datetime.datetime'> datetime.datetime(2024, 1, 1, 9, 0) The first occurrence.
every int 1 How many units apart.
unit Literal['hours', 'days', 'weeks', 'months', 'years'] 'weeks' The repeat interval's unit.

temporal.time_of_day#

A clock time with no date, held as the time elapsed since midnight.

Priority P0 · dtype timedelta64 · equivalence continuous

Example params: earliest=datetime.time(8, 30) latest=datetime.time(17, 0) resolution_seconds=900

Field Type Default Description
earliest <class 'datetime.time'> datetime.time(0, 0) Earliest time, inclusive.
latest <class 'datetime.time'> datetime.time(23, 59, 59) Latest time, inclusive.
resolution_seconds int 1 Step between possible values. 60 gives times on the minute.

temporal.timestamp#

A date and time together, picked from a range.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: start=datetime.datetime(2024, 3, 1, 9, 0) end=datetime.datetime(2024, 3, 31, 17, 0) resolution_seconds=60

Field Type Default Description
start <class 'datetime.datetime'> datetime.datetime(2024, 1, 1, 0, 0) Earliest instant, inclusive.
end <class 'datetime.datetime'> datetime.datetime(2024, 12, 31, 23, 59, 59) Latest instant, inclusive.
resolution_seconds int 1 Step between possible values.

temporal.timezone#

An IANA timezone name, from the list the app ships with.

Priority P0 · dtype string · equivalence categorical

Example params: zones=('America/New_York', 'America/Chicago', 'America/Denver', 'America/Los_Angeles') weights=(0.47, 0.21, 0.09, 0.23)

Field Type Default Description
zones tuple[str, ...] | None None Restrict to these IANA names. All of the shipped list when omitted.
weights tuple[float, ...] | None None Relative frequency per zone, in the order given.

C.2 — Identity and geography#

Personal name#

name.first#

A real given name, drawn as often as real people were given it.

Priority P0 · dtype string · equivalence structural

Example params: sex=None gender=None birth_decade=None most_common=None

Field Type Default Description
sex Optional[Literal['female', 'male']] None Draw only from names registered for this sex. Left unset, the whole table is in play and the column mixes the two in their published proportion.
gender Reference | None None A demographics.gender column of the same table, read row for row, so that each row's name agrees with its own recorded gender rather than the column's.
birth_decade int | None None Popularity as of this decade, named by its first year — 1880 to 2020. Left unset, all fifteen decades are added together, which is a distribution nobody was ever born into and is the right default only for a column with no age beside it.
most_common int | None None Use only the this-many most-used name and sex pairs. Left unset, all 117,820 are in play, which is what makes the tail realistic.

name.full#

The name parts you already have, joined into one string.

Priority P0 · dtype string · equivalence structural

Example params: first=Reference(table='person', column='first_name') last=Reference(table='person', column='last_name') middle=None prefix=None suffix=None

Field Type Default Description
first Reference — A name.first column of the same table.
last Reference — A name.last column of the same table.
middle Reference | None None A name.middle column. Rows where it is blank get no middle segment, so a middle column under §5.3.5 missingness composes the way it reads.
prefix Reference | None None A name.prefix column, written in front.
suffix Reference | None None A name.suffix column, written after.

name.last#

A real surname, drawn as often as real people have it.

Priority P0 · dtype string · equivalence structural

Example params: ethnicity=None most_common=None

Field Type Default Description
ethnicity Optional[Literal['White', 'Hispanic or Latino', 'Black or African American', 'Asian', 'Two or more races', 'Some other race', 'American Indian or Alaska Native', 'Native Hawaiian or Other Pacific Islander']] None Draw from the surnames of this group rather than of everybody.
most_common int | None None Use only the this-many most frequent surnames. Left unset, all 162,253 are in play, which is what makes the tail realistic and the cardinality high.

name.middle#

A middle name, or a middle initial.

Priority P0 · dtype string · equivalence structural

Example params: sex=None gender=None birth_decade=None most_common=None initial_only=False

Field Type Default Description
sex Optional[Literal['female', 'male']] None Draw only from names registered for this sex. Left unset, the whole table is in play and the column mixes the two in their published proportion.
gender Reference | None None A demographics.gender column of the same table, read row for row, so that each row's name agrees with its own recorded gender rather than the column's.
birth_decade int | None None Popularity as of this decade, named by its first year — 1880 to 2020. Left unset, all fifteen decades are added together, which is a distribution nobody was ever born into and is the right default only for a column with no age beside it.
most_common int | None None Use only the this-many most-used name and sex pairs. Left unset, all 117,820 are in play, which is what makes the tail realistic.
initial_only bool False Write J. rather than James. The point is included, so a composed full name can join the parts with spaces and not have to know which it got.

name.prefix#

A title in front of the name.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

name.suffix#

A generational or credential suffix after the name.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

Digital identity#

digital.avatar#

An identicon per row - a small symmetric pattern, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: cells=5 pixels_per_cell=12 density=0.5

Field Type Default Description
cells int 5 Cells along one side of the grid.
pixels_per_cell int 12 How many pixels wide one cell is drawn.
density float 0.5 Chance that a cell is inked. The realised fraction is this per free cell, and the mirror copies it rather than drawing again.

digital.email#

A mail address in a domain that is guaranteed never to deliver.

Priority P0 · dtype string · equivalence structural

Example params: words=2 separator='.' digits=0 local=None domain='example'

Field Type Default Description
words int 2 How many lexicon words the handle is made of when no column is referenced. Ignored when one is.
separator Literal['', '.', '_', '-'] '.' What goes between those words. Empty runs them together.
digits int 2 Length of the numeric tail that keeps two people of the same name apart. 0 omits it.
local Reference | None None A text column of the same table — a name, or a digital.username column — to build the part before the @ from. Given none, it is made of lexicon words.
domain Literal['example', 'invalid', 'test'] 'example' Which reserved space the domain comes from: the three example.* names, or a lexicon word under .invalid or .test.

digital.password_hash#

A stored-credential string - no password produces it, and none is written down.

Priority P0 · dtype string · equivalence unique

Example params: scheme='bcrypt'

Field Type Default Description
scheme Literal['bcrypt', 'argon2id', 'sha256', 'md5'] 'bcrypt' Which storage format the string imitates.

digital.username#

A login handle, optionally the one this row's person would have chosen.

Priority P0 · dtype string · equivalence structural

Example params: words=2 separator='.' digits=2 name=None

Field Type Default Description
words int 2 How many lexicon words the handle is made of when no column is referenced. Ignored when one is.
separator Literal['', '.', '_', '-'] '.' What goes between those words. Empty runs them together.
digits int 2 Length of the numeric tail that keeps two people of the same name apart. 0 omits it.
name Reference | None None A text column of the same table — a surname, or a composed full name — to build the handle from. Given none, the handle is made of lexicon words.

Demographics#

demographics.age#

An age in whole years, drawn uniformly across a range.

Priority P0 · dtype int64 · equivalence continuous

Example params: minimum=20 maximum=70 distribution=None

Field Type Default Description
minimum int 18 The youngest age in the column, inclusive.
maximum int 90 The oldest age in the column, inclusive.
distribution Optional[Annotated[Normal | LogNormal | Uniform | Exponential | Gamma | Beta | Weibull | Triangular | Pareto | Cauchy | ChiSquare | StudentT | Logistic | Gumbel | Shape | Bernoulli | Binomial | Poisson | NegBinomial | Geometric | HyperGeometric | Zipf, FieldInfo(annotation=NoneType, required=True, discriminator='family')]] None Optional shape for the ages within the range above -- a real population is rarely flat between minimum and maximum. Omit for a uniform draw (the default); state one (e.g. a normal centred on a median age) to shape it. Always clipped back to minimum/maximum, whatever the family says on its own.

demographics.education#

Highest level of education completed, listed from least to most.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

demographics.employer#

A plausible company name, assembled from word lists.

Priority P0 · dtype string · equivalence structural

Example params: legal_suffix=True

Field Type Default Description
legal_suffix bool True Append LLC, Inc., Corp. and the like.

demographics.ethnicity#

Race and Hispanic origin as one column, in the standard categories.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

demographics.gender#

Sex or gender, as two categories or as four.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None categories='expanded'

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.
categories Literal['binary', 'expanded'] 'binary' Two categories, or four including non-binary.

demographics.income#

An income column stated as a median and a Gini coefficient.

Priority P0 · dtype float64 · equivalence continuous

Example params: median=59000.0 gini=0.48

Field Type Default Description
median float 59000.0 The middle income. Half the column falls below it, exactly.
gini float 0.48 The Gini coefficient: 0 is everybody equal, 1 is one earner taking everything. Around 0.48 for US household income.

demographics.language#

The language spoken at home.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

demographics.marital_status#

Married, never married, divorced, widowed or separated.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

demographics.nationality#

The country a person belongs to.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

demographics.occupation#

One of the twenty-two standard major occupation groups.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

Address#

address.city#

The city the postal code sits in.

Priority P0 · dtype string · equivalence structural

Example params: postal_code=None

Field Type Default Description
postal_code Reference | None None An address.postal_code column of the same table. Given one, this column describes that row's ZIP; given none, it describes a place of its own.

address.coordinates#

A latitude or a longitude for the postal code.

Priority P0 · dtype float64 · equivalence continuous

Example params: postal_code=None axis='latitude' jitter=True

Field Type Default Description
postal_code Reference | None None An address.postal_code column of the same table. Given one, this column describes that row's ZIP; given none, it describes a place of its own.
axis Literal['latitude', 'longitude'] — Which half of the pair this column holds.
jitter bool True Move the point off the area's internal point, so that two rows of the same ZIP are not the same place. Unset, every row of a ZIP gets its centre.

address.country#

The country, in whichever form your schema wants.

Priority P0 · dtype string · equivalence exact

Example params: form='name'

Field Type Default Description
form Literal['name', 'iso2', 'iso3'] 'name' United States, US or USA (ISO 3166-1).

address.county#

The county the postal code sits in.

Priority P0 · dtype string · equivalence structural

Example params: postal_code=None

Field Type Default Description
postal_code Reference | None None An address.postal_code column of the same table. Given one, this column describes that row's ZIP; given none, it describes a place of its own.

address.full#

The parts you already have, composed into one mailing line.

Priority P0 · dtype string · equivalence structural

Example params: street=Reference(table='customer', column='street') city=Reference(table='customer', column='city') state=Reference(table='customer', column='state') postal_code=Reference(table='customer', column='zip') unit=None

Field Type Default Description
street Reference — An address.street column of the same table.
city Reference — An address.city column of the same table.
state Reference — An address.state column of the same table.
postal_code Reference — An address.postal_code column.
unit Reference | None None An address.unit column. Rows where it is blank get no second segment, so a unit column under §5.3.5 missingness composes the way it reads.

address.postal_code#

A real US postal code, optionally with its four-digit add-on.

Priority P0 · dtype string · equivalence structural

Example params: state=None plus_four=False

Field Type Default Description
state str | None None Two-letter USPS abbreviation. Left unset, the whole country is in play.
plus_four bool False Render the four-digit add-on code as well: 62704-1234.

address.state#

The state, as a two-letter code or spelled out.

Priority P0 · dtype string · equivalence categorical

Example params: postal_code=None form='code'

Field Type Default Description
postal_code Reference | None None An address.postal_code column of the same table. Given one, this column describes that row's ZIP; given none, it describes a place of its own.
form Literal['code', 'name'] 'code' CA or California.

address.street#

A house number and a street name that do not exist.

Priority P0 · dtype string · equivalence structural

Example params: highest_number=9999 abbreviate=False

Field Type Default Description
highest_number int 9999 House numbers run from 1 to this. Larger ranges are real — grid-numbered cities reach six figures — and cost nothing but rendered width.
abbreviate bool False Maple St rather than Maple Street.

address.timezone#

The IANA timezone of the postal code, where federal law settles it.

Priority P0 · dtype string · equivalence categorical

Example params: postal_code=None

Field Type Default Description
postal_code Reference | None None An address.postal_code column of the same table. Given one, this column describes that row's ZIP; given none, it describes a place of its own.

address.unit#

The second line - apartment, suite, floor.

Priority P0 · dtype string · equivalence structural

Example params: ``

Contact#

contact.address_international#

A well-formed Australian, Canadian or British address.

Priority P1 · dtype string · equivalence structural

Example params: countries=None highest_number=999

Field Type Default Description
countries tuple[str, ...] | None None ISO 3166-1 alpha-2 codes to draw from, equally weighted. Every country with a transcribed layout when omitted.
highest_number int 999 House numbers run from 1 to this.

contact.line_type#

Whether a number is a mobile, a landline or VoIP.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

contact.phone#

A US phone number, punctuated as you choose, that cannot reach anybody.

Priority P0 · dtype string · equivalence structural

Example params: style='dashes' area_codes=None

Field Type Default Description
style Literal['dashes', 'parentheses', 'dotted', 'plain', 'e164'] 'dashes' How the number is punctuated.
area_codes tuple[str, ...] | None None Draw the area code from these instead of from every geographic code. Three digits each, first digit 2-9.

contact.phone_international#

A phone number in one of the four countries that reserve a range for fiction.

Priority P1 · dtype string · equivalence structural

Example params: style='e164' countries=None

Field Type Default Description
style Literal['e164', 'national'] 'e164' +441632960123, or the national format the country itself writes (01632 960123).
countries tuple[str, ...] | None None ISO 3166-1 alpha-2 codes to draw from, equally weighted. Every country with a reserved range when omitted.

Government-style identifiers#

government.drivers_licence#

A driver's licence number in the shape one of the fifty-one jurisdictions issues.

Priority P0 · dtype string · equivalence structural

Example params: states=None

Field Type Default Description
states tuple[str, ...] | None None USPS abbreviations to draw from, equally weighted. All fifty-one jurisdictions with a published format when omitted.

government.national_id#

A US Social Security number drawn from the range that was never issued.

Priority P0 · dtype string · equivalence unique

Example params: separators=True

Field Type Default Description
separators bool True Write it as 666-12-3456 rather than 666123456.

government.passport#

A nine-digit passport number, optionally with the check digit a reader recomputes.

Priority P0 · dtype string · equivalence unique

Example params: check_digit=True

Field Type Default Description
check_digit bool False Append the ICAO 9303 check digit, as the machine-readable zone carries it. Ten characters instead of nine.

C.3 — Relational and graph#

Household#

household.parent#

The row number of a person's parent, or -1.

Priority P0 · dtype int64 · equivalence structural

Example params: shape=HouseholdShape(households=250000, size_shares=(0.3, 0.34, 0.15, 0.13, 0.06, 0.02), layout='households', couple_rate=0.62)

Field Type Default Description
shape HouseholdShape PydanticUndefined The household layout. Must match the unit column's.

household.partner#

The row number of a person's spouse or partner, or -1.

Priority P0 · dtype int64 · equivalence structural

Example params: shape=HouseholdShape(households=250000, size_shares=(0.3, 0.34, 0.15, 0.13, 0.06, 0.02), layout='households', couple_rate=0.62)

Field Type Default Description
shape HouseholdShape PydanticUndefined The household layout. Must match the unit column's.

household.surname#

The surname a person actually carries, given their household's.

Priority P0 · dtype string · equivalence structural

Example params: own=Reference(table='person', column='surname') shape=HouseholdShape(households=250000, size_shares=(0.3, 0.34, 0.15, 0.13, 0.06, 0.02), layout='households', couple_rate=0.62) partner_takes=0.68 partner_hyphenates=0.06 separator='-'

Field Type Default Description
own Reference — A surname column — what this person would be called on their own. The head's is the household's, and the rule decides who else carries it.
shape HouseholdShape PydanticUndefined The household layout. Must match the unit column's.
partner_takes float 0.68 Share of partners who take the head's surname.
partner_hyphenates float 0.06 Share of partners who join both with a hyphen. The rest keep their own.
separator str '-' What a hyphenated name joins with.

household.unit#

Which household a person belongs to.

Priority P0 · dtype int64 · equivalence structural

Example params: shape=HouseholdShape(households=250000, size_shares=(0.3, 0.34, 0.15, 0.13, 0.06, 0.02), layout='households', couple_rate=0.62)

Field Type Default Description
shape HouseholdShape PydanticUndefined The household layout. Every household column of a table repeats it.

Organisation#

org.level#

How many managers sit between a person and the top.

Priority P0 · dtype int64 · equivalence categorical

Example params: shape=OrgShape(span=7, max_depth=6, layout='org', managers=25000)

Field Type Default Description
shape OrgShape PydanticUndefined The reporting tree. Every organisation column of a table repeats it.

org.manager#

The row a person reports to, or -1 for the root.

Priority P0 · dtype int64 · equivalence structural

Example params: shape=OrgShape(span=7, max_depth=6, layout='org', managers=25000)

Field Type Default Description
shape OrgShape PydanticUndefined The reporting tree. Must match the level column's.

org.team#

Which department a person is in - the same as their manager's.

Priority P0 · dtype string · equivalence categorical

Example params: shape=OrgShape(span=7, max_depth=6, layout='org', managers=25000) department_level=1 names=('Engineering', 'Sales', 'Marketing', 'Finance', 'People', 'Operations', 'Legal', 'Customer Support', 'Product', 'Research', 'Facilities', 'Security') executive='Executive'

Field Type Default Description
shape OrgShape PydanticUndefined The reporting tree. Must match the level column's.
department_level int 1 The level a department is rooted at. Everybody below inherits it; everybody above is in all of them.
names tuple[str, ...] ('Engineering', 'Sales', 'Marketing', 'Finance', 'People', 'Operations', 'Legal', 'Customer Support', 'Product', 'Research', 'Facilities', 'Security') Department names, in the order the level's rows take them.
executive str 'Executive' What the rows above the department level are called.

Schema structure#

schema.category_tree#

A self-referencing hierarchy laid out depth-first.

Priority P0 · dtype int64 · equivalence exact

Example params: widths=(12, 8, 10)

Field Type Default Description
widths tuple[int, ...] (12, 8, 10) How many nodes each level holds per parent, top level first. (12, 8, 10) is twelve departments, each with eight categories, each with ten subcategories.

schema.foreign_key#

A key into a parent table, with the join's shape under control.

Priority P0 · dtype int64 · equivalence structural

Example params: parents=10000 cardinality='skewed' unlinked_share=0.0

Field Type Default Description
parents int 10000 Rows in the parent table. Every value points at one of them.
cardinality Literal['uniform', 'skewed', 'even'] 'uniform' How children are spread over parents: evenly by chance (uniform), concentrated on a few popular parents (skewed), or exactly equally (even).
unlinked_share float 0.0 Share of rows that point at nothing, for an optional foreign key.

schema.junction#

One side of a many-to-many table, with no pair appearing twice.

Priority P0 · dtype int64 · equivalence structural

Example params: side='left' left_rows=5000 right_rows=200

Field Type Default Description
side Literal['left', 'right'] 'left' Which of the two keys this column holds.
left_rows int 5000 Rows in the table on the left.
right_rows int 200 Rows in the table on the right.

schema.primary_key#

A unique key that is deliberately not the row number.

Priority P0 · dtype int64 · equivalence unique

Example params: space=100000000 start=0 ordered=False

Field Type Default Description
space int 100000000 How large the key range is. Must exceed the table's row count, because a space smaller than the table would give two rows the same key.
start int 0 Added to every key, for a schema whose ids begin somewhere.
ordered bool False Hand keys out in order instead of scattering them. Off by default: a key that ascends with the row is a key everything downstream will quietly rely on.

Social#

social.follower_edge#

One end of a follower edge, in a graph with a power-law in-degree.

Priority P1 · dtype int64 · equivalence structural

Example params: end='target' accounts=200000 mean_degree=8.0 tail_index=1.6

Field Type Default Description
end Literal['target', 'source'] 'target' Which end this column holds: the account being followed, or the one following it.
accounts int 200000 How many accounts the graph has. Both ends index into them.
mean_degree float 8.0 Followers per account, averaged over all of them. The edge count is this times the account count, and the run has to fit inside it.
tail_index float 1.6 Pareto index of the in-degree. Lower is heavier: at 1.6 the busiest account has thousands of followers, at 5 almost everybody has about the mean.

C.4 — Business and finance#

Payments and accounts#

account.balance#

An account balance, including the accounts with less than nothing in them.

Priority P0 · dtype float64 · equivalence continuous

Example params: median=2400.0 p99_multiple=40.0 overdrawn_share=0.08 overdraft_median=95.0 overdraft_p99_multiple=8.0 minor_units=2

Field Type Default Description
median float 2400.0 The middle balance among accounts in credit.
p99_multiple float 40.0 How many times the median the 99th percentile is. Balances are more unequal than transactions are.
overdrawn_share float 0.08 Share of accounts in overdraft. Their balances are negative and small.
overdraft_median float 95.0 The middle overdraft, as a positive number. Reported negative.
overdraft_p99_multiple float 8.0 How many times the median the deepest one percent of overdrafts are. Far tighter than the credit side, because an overdraft is bounded by a limit and a balance is not.
minor_units int 2 Decimal places.

account.number#

A deposit account number, unique by construction and not in row order.

Priority P0 · dtype string · equivalence unique

Example params: digits=10

Field Type Default Description
digits int 10 Length of the number. US institutions issue anything from 6 to 17.

account.routing#

A bank identifier - a US ABA routing number or an IBAN.

Priority P0 · dtype string · equivalence categorical

Example params: scheme='aba' institutions=64

Field Type Default Description
scheme Literal['aba', 'iban'] 'aba' A nine-digit US ABA routing number, or an IBAN. Not a presentation choice: they identify different things in different systems.
institutions int 64 How many distinct banks the column draws from. A transaction file has far fewer banks than rows.

payment.amount#

One card transaction, from a distribution with a long right tail.

Priority P0 · dtype float64 · equivalence continuous

Example params: median=28.5 p99_multiple=22.0 minor_units=2

Field Type Default Description
median float 28.5 The middle transaction. Half the column falls below it, exactly.
p99_multiple float 22.0 How many times the median the 99th percentile is. Card spend is heavy: the top percent of transactions runs an order of magnitude above the typical one.
minor_units int 2 Decimal places the currency is quoted in. Two for dollars and euros, zero for yen.

payment.card_number#

A Luhn-valid card number in a range no payment network routes.

Priority P0 · dtype string · equivalence unique

Example params: digits=16 grouped=False

Field Type Default Description
digits int 16 Length including the check digit. Sixteen is usual; fifteen is the American Express shape and nineteen the longest ISO allows.
grouped bool False Render in four-digit groups separated by spaces, the way a card is printed and a form displays it.

payment.card_security#

The expiry or the security code printed on a card, one column each.

Priority P0 · dtype string · equivalence structural

Example params: field='cvv' first_year=2026 years=5 digits=3

Field Type Default Description
field Literal['expiry', 'cvv'] 'cvv' Which of the two this column holds. Declare one column of each.
first_year int 2026 Earliest expiry year in the column. Ignored by a CVV column.
years int 5 How many years of expiries the column spans. Cards are issued on three- to five-year terms, so a portfolio spans a few.
digits int 3 Security-code length. Three for most networks, four for the American Express shape. Ignored by an expiry column.

payment.merchant#

The descriptor a card statement shows, not the merchant's legal name.

Priority P0 · dtype string · equivalence structural

Example params: aggregated_share=0.35 store_number=True

Field Type Default Description
aggregated_share float 0.35 Share of transactions billed through a payment facilitator, which puts its own tag on the front of the descriptor.
store_number bool True Append a store number, the way a chain's descriptor carries.

payment.merchant_category#

A real four-digit merchant category code, drawn from the published table.

Priority P0 · dtype string · equivalence structural

Example params: trades='all' representation='code'

Field Type Default Description
trades Literal['all', 'services', 'goods'] 'all' Narrow to categories that predominantly furnish services, or that predominantly provide goods, as Rev. Proc. 2004-43 classifies them.
representation Literal['code', 'description', 'both'] 'code' Write the category as its four-digit code, its description, or the code then the description.

Markets#

market.holding#

One line of a portfolio, as a share of it, and the portfolio adds up.

Priority P0 · dtype float64 · equivalence continuous

Example params: positions=25 concentration=1.0 representation='weight' decimals=6

Field Type Default Description
positions int 25 Lines in one portfolio, in consecutive rows. The weights across them sum to one.
concentration float 1.0 How evenly the portfolio is spread. Below one it is dominated by a few names; well above one every line is nearly the same size.
representation Literal['weight', 'percent'] 'weight' A fraction summing to one, or a percentage summing to a hundred.
decimals int 6 Places the weight is reported to. The residual from rounding is put on the largest line, so the column still adds up.

market.price_gbm#

A closing price on a geometric Brownian path - the textbook equity model.

Priority P0 · dtype float64 · equivalence continuous

Example params: steps=252 initial_price=100.0 step_years=0.003968253968253968 drift=0.07 volatility=0.2

Field Type Default Description
steps int 252 Observations per instrument, in consecutive rows. 252 is a trading year.
initial_price float 100.0 Where every series starts. Series are rebased to a common level, which is what a normalised price panel does; multiply by another column to vary it.
step_years float 0.003968253968253968 Length of one step in years, which is what makes the drift and volatility annual figures. A trading day is 1/252.
drift float 0.07 Expected continuously compounded return a year, before the volatility correction.
volatility float 0.2 Annualised standard deviation of log returns.

market.price_jump#

A Brownian price that sometimes gaps, at an intensity you set.

Priority P0 · dtype float64 · equivalence continuous

Example params: steps=252 initial_price=100.0 step_years=0.003968253968253968 drift=0.07 volatility=0.2 jumps_a_year=3.0 jump_mean=-0.02 jump_sd=0.06

Field Type Default Description
steps int 252 Observations per instrument, in consecutive rows. 252 is a trading year.
initial_price float 100.0 Where every series starts. Series are rebased to a common level, which is what a normalised price panel does; multiply by another column to vary it.
step_years float 0.003968253968253968 Length of one step in years, which is what makes the drift and volatility annual figures. A trading day is 1/252.
drift float 0.07 Expected continuously compounded return a year, before the volatility correction.
volatility float 0.2 Annualised standard deviation of log returns.
jumps_a_year float 3.0 Expected number of jumps a year. Three is a name with a quarterly earnings surprise.
jump_mean float -0.02 Average jump, as a log return. Negative because the jumps that matter mostly go down.
jump_sd float 0.06 Standard deviation of the jump size, in log return.

market.price_ou#

A price pulled back towards a level - a commodity, a spread, a real exchange rate.

Priority P0 · dtype float64 · equivalence continuous

Example params: steps=252 initial_price=100.0 step_years=0.003968253968253968 mean_price=100.0 reversion=4.0 volatility=0.25

Field Type Default Description
steps int 252 Observations per instrument, in consecutive rows. 252 is a trading year.
initial_price float 100.0 Where every series starts. Series are rebased to a common level, which is what a normalised price panel does; multiply by another column to vary it.
step_years float 0.003968253968253968 Length of one step in years, which is what makes the drift and volatility annual figures. A trading day is 1/252.
mean_price float 100.0 The level the series is pulled towards, in price units.
reversion float 4.0 Speed of reversion a year. A half-life of about ln(2) over this, so 4 is roughly two months.
volatility float 0.25 Annualised volatility of the log price.

market.ticker#

A ticker symbol, from a pool the size of your instrument universe.

Priority P0 · dtype string · equivalence categorical

Example params: symbols=64 letters=4 prefix=''

Field Type Default Description
symbols int 64 Distinct instruments in the column. A trade file has far more rows than instruments.
letters int 4 Length of the symbol. One to five, as the US exchanges allocate them.
prefix str '' Fixed leading letters. A test environment usually reserves one, which is the only way to keep a generated symbol away from a listed one.

market.volume#

Shares traded, heavy-tailed, and clustered so busy days come in runs.

Priority P0 · dtype int64 · equivalence continuous

Example params: steps=252 median=850000.0 p99_multiple=9.0 persistence=0.6

Field Type Default Description
steps int 252 Observations per instrument, in consecutive rows.
median float 850000.0 The middle day's volume.
p99_multiple float 9.0 How many times the median the busiest one percent of days are.
persistence float 0.6 How much of a day's unusualness carries into the next. Zero makes days independent; above about 0.9 a quiet month is followed by a quiet month.

Lending#

lending.amortisation#

One column of a repayment schedule that reconciles exactly.

Priority P0 · dtype float64 · equivalence continuous

Example params: component='balance' loan_principal=Reference(table='loans', column='principal') periods=60 annual_rate=0.065 minor_units=2

Field Type Default Description
component Literal['payment', 'interest', 'principal', 'balance'] 'balance' Which part of the period's row: the level payment, the interest in it, the principal in it, or the balance left after it.
loan_principal Reference — A principal column of the loan table — one row per loan, not per period.
periods int 60 Payments in the schedule, in consecutive rows. Every loan runs the same number, which is what makes the layout positional.
annual_rate float 0.065 One rate for the whole book. A rate per loan is a per-category override, not a parameter here.
minor_units int 2 Decimal places for money.

lending.credit_score#

A credit score on the 300 to 850 scale, leaning towards the top as real ones do.

Priority P0 · dtype int64 · equivalence continuous

Example params: mean=715.0 concentration=6.0 minimum=300 maximum=850

Field Type Default Description
mean float 715.0 Mean score of the population. Around 715 for the United States, which is an order-of-magnitude figure rather than a measurement.
concentration float 6.0 How tightly scores cluster on that mean. Smaller spreads them towards both ends; below about 2 the column is closer to bimodal than to a population.
minimum int 300 Bottom of the scale.
maximum int 850 Top of the scale.

lending.loan#

One field of a loan record - principal, rate or term - that agrees with the other two.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='term' principal=Reference(table='loans', column='principal') term=None median=18000.0 p99_multiple=22.0 rounding=100.0 base_rate=0.055 term_premium=0.004 rate_spread=0.015

Field Type Default Description
field Literal['principal', 'rate', 'term'] 'principal' Which of the three this column holds. Declare one column of each.
principal Reference | None None A lending.loan principal column. Required by a term column, which is how big loans get long terms.
term Reference | None None A lending.loan term column. Required by a rate column, which is how long terms get higher rates.
median float 18000.0 The middle loan. Only a principal column uses it.
p99_multiple float 22.0 How many times the median the largest one percent of loans are.
rounding float 100.0 Principals are advanced in round amounts. 100 rounds to the nearest hundred.
base_rate float 0.055 Annual rate on the shortest term, before the term premium. Only a rate column uses it.
term_premium float 0.004 Extra annual rate at a one-year term, growing with the square root of the term. A flattening curve, because a real term structure is concave.
rate_spread float 0.015 Idiosyncratic spread around the priced rate, standing for everything the term does not explain.

Accounting#

accounting.account#

One line of a chart of accounts - a number, a name, or both.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None representation='code' classes=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.
representation Literal['code', 'name', 'both'] 'code' Write the account as its number, its name, or the number then the name.
classes tuple[Literal['Asset', 'Liability', 'Equity', 'Revenue', 'Expense'], ...] | None None Narrow to some account classes. All five when omitted.

accounting.ageing#

Which ageing bucket a receivable or payable falls in.

Priority P0 · dtype string · equivalence categorical

Example params: days_overdue=None settled_share=0.62 mean_overdue=34.0

Field Type Default Description
days_overdue Reference | None None A column of days past due. Given one, the bucket is read off it; given none, the days are drawn here.
settled_share float 0.62 Share not yet due, which is the Current bucket. Ignored when a days column is given.
mean_overdue float 34.0 Average days past due among the items that are. Ignored when a days column is given.

accounting.depreciation#

One period of an asset's depreciation, on a schedule that retires exactly its cost.

Priority P0 · dtype float64 · equivalence continuous

Example params: component='book_value' method='straight_line' asset_cost=Reference(table='assets', column='cost') periods=60 salvage_share=0.0 minor_units=2

Field Type Default Description
component Literal['charge', 'accumulated', 'book_value'] 'book_value' This period's charge, the accumulated depreciation after it, or the book value left.
method Literal['straight_line', 'declining_balance', 'sum_of_years'] 'straight_line' Straight line, reducing balance at double the straight-line rate, or sum-of-years digits.
asset_cost Reference — A cost column of the asset register — one row per asset, not per period.
periods int 60 Periods in the schedule, in consecutive rows. Every asset runs the same number.
salvage_share float 0.0 Residual value at the end, as a share of cost. The schedule stops there rather than at zero.
minor_units int 2 Decimal places for money.

accounting.invoice#

What an invoice comes to, one row per document.

Priority P0 · dtype float64 · equivalence continuous

Example params: median=1850.0 p99_multiple=28.0 minor_units=2

Field Type Default Description
median float 1850.0 The middle invoice.
p99_multiple float 28.0 How many times the median the largest invoices are.
minor_units int 2 Decimal places for money.

accounting.invoice_line#

One line of an invoice, in a set that sums to the header exactly.

Priority P0 · dtype float64 · equivalence continuous

Example params: invoice_total=Reference(table='invoices', column='total') lines=5 concentration=1.2 minor_units=2

Field Type Default Description
invoice_total Reference — An accounting.invoice column — one row per document, not per line.
lines int 5 Lines on one invoice, in consecutive rows. They sum to the header.
concentration float 1.2 How evenly the total is spread across the lines. Below one most invoices have one dominant line.
minor_units int 2 Decimal places for money.

accounting.journal_line#

One posting of a journal entry whose lines balance exactly.

Priority P0 · dtype float64 · equivalence continuous

Example params: component='amount' lines=4 debits=2 median=4200.0 p99_multiple=30.0 concentration=1.4 minor_units=2

Field Type Default Description
component Literal['amount', 'debit', 'credit'] 'amount' The signed amount, or the debit and credit columns a ledger prints.
lines int 4 Lines in one entry, in consecutive rows. At least two, or nothing can balance.
debits int 2 How many of those lines are debits. The rest are credits.
median float 4200.0 The middle entry, in whole currency.
p99_multiple float 30.0 How many times the median the largest entries are.
concentration float 1.4 How evenly an entry's total is split across its lines on one side. Below one it lands mostly on a single line.
minor_units int 2 Decimal places for money.

accounting.tax_line#

The tax on the amount beside it, at one of the rates in play.

Priority P0 · dtype float64 · equivalence continuous

Example params: component='tax' net_amount=Reference(table='invoices', column='total') rates=(0.0, 0.05, 0.07, 0.0825, 0.1) weights=None minor_units=2

Field Type Default Description
component Literal['tax', 'gross', 'rate'] 'tax' The tax charged, the gross including it, or the rate applied.
net_amount Reference — The net amount this line is taxed on.
rates tuple[float, ...] (0.0, 0.05, 0.07, 0.0825, 0.1) The rates in play, as fractions. A jurisdiction mix, not a schedule: several rates appear in one ledger because several places do.
weights tuple[float, ...] | None None Relative frequency per rate. Equal when omitted.
minor_units int 2 Decimal places for money.

accounting.trial_balance#

An account's balance, in a listing that nets to zero.

Priority P0 · dtype float64 · equivalence continuous

Example params: account=Reference(table='ledger', column='account') accounts=40 median=85000.0 p99_multiple=25.0 minor_units=2

Field Type Default Description
account Reference — An accounting.account column of codes, one row per account.
accounts int 40 Accounts in one trial balance, in consecutive rows. Their balances sum to zero.
median float 85000.0 The middle account balance, ignoring its sign.
p99_multiple float 25.0 How many times the median the largest balances are. A ledger is far more unequal than a journal.
minor_units int 2 Decimal places for money.

Filings#

filings.balance_sheet#

One line of a balance sheet where assets equal liabilities plus equity.

Priority P0 · dtype float64 · equivalence continuous

Example params: line='cash' cash=None receivables=None inventory=None fixed_assets=None total_assets=Reference(table='filings', column='total_assets') payables=None debt=None total_liabilities=None total_equity=None share_capital=None median_assets=64000000.0 p99_multiple=90.0 share_spread=0.05 gearing=0.55 payables_share=0.4 capital_share=0.3 capital_spread=0.8 minor_units=0

Field Type Default Description
line Literal['cash', 'receivables', 'inventory', 'fixed_assets', 'total_assets', 'payables', 'debt', 'total_liabilities', 'share_capital', 'retained_earnings', 'total_equity'] 'cash' Which line of the balance sheet this column holds.
cash Reference | None None The cash line.
receivables Reference | None None The receivables line.
inventory Reference | None None The inventory line.
fixed_assets Reference | None None The fixed assets line.
total_assets Reference | None None The total assets line.
payables Reference | None None The payables line.
debt Reference | None None The debt line.
total_liabilities Reference | None None The total liabilities line.
total_equity Reference | None None The total equity line.
share_capital Reference | None None The share capital line.
median_assets float 64000000.0 The middle filer's total assets.
p99_multiple float 90.0 How many times the median the largest filers are.
share_spread float 0.05 How much the asset mix varies between filers, on each share.
gearing float 0.55 Liabilities as a share of assets. The rest is equity, which is what makes the sheet balance.
payables_share float 0.4 Payables as a share of liabilities.
capital_share float 0.3 Share capital as a share of equity, before the spread. Retained earnings are the remainder.
capital_spread float 0.8 How much that share varies between filers, as a lognormal sigma so the mean stays exactly capital_share. Wide enough that some filers have raised more than they have kept, which is an accumulated deficit.
minor_units int 0 Decimal places.

filings.cash_flow#

One line of a cash-flow statement that reconciles.

Priority P0 · dtype float64 · equivalence continuous

Example params: line='investing' operating=Reference(table='filings', column='operating') investing=None financing=None opening_cash=None net_change=None median_operating=6400000.0 p99_multiple=60.0 investing_ratio=-0.55 financing_ratio=-0.2 ratio_spread=0.25 opening_multiple=1.8 minor_units=0

Field Type Default Description
line Literal['operating', 'investing', 'financing', 'net_change', 'opening_cash', 'closing_cash'] 'operating' Which line of the statement this column holds.
operating Reference | None None Cash from operations.
investing Reference | None None Cash used in investing.
financing Reference | None None Cash from financing.
opening_cash Reference | None None Cash brought forward.
net_change Reference | None None The net change line.
median_operating float 6400000.0 The middle filer's operating cash flow. Positive: a going concern usually generates cash.
p99_multiple float 60.0 How many times the median the largest filers are.
investing_ratio float -0.55 Investing cash flow as a share of operating. Negative because a growing company spends on assets.
financing_ratio float -0.2 Financing cash flow as a share of operating. Negative when debt is being repaid and dividends paid.
ratio_spread float 0.25 How much those two ratios vary between filers.
opening_multiple float 1.8 Opening cash as a multiple of operating cash flow.
minor_units int 0 Decimal places.

filings.identifier#

A central index key, or the accession number of one filing.

Priority P0 · dtype string · equivalence structural

Example params: format='cik' filers=500 year=2024

Field Type Default Description
format Literal['cik', 'accession'] 'cik' A ten-digit central index key, or an accession number naming one submission.
filers int 500 Distinct filers in the column. A filings table has far more rows than registrants.
year int 2024 Filing year an accession number carries. 1993 is the first year EDGAR holds.

filings.income_statement#

One line of an income statement that ties to the lines around it.

Priority P0 · dtype float64 · equivalence continuous

Example params: line='cost_of_sales' revenue=Reference(table='filings', column='revenue') cost_of_sales=None gross_profit=None operating_expenses=None operating_income=None tax=None median_revenue=48000000.0 p99_multiple=90.0 gross_margin=0.42 opex_ratio=0.31 ratio_spread=0.08 tax_rate=0.21 minor_units=0

Field Type Default Description
line Literal['revenue', 'cost_of_sales', 'gross_profit', 'operating_expenses', 'operating_income', 'tax', 'net_income'] 'revenue' Which line of the statement this column holds.
revenue Reference | None None The revenue line.
cost_of_sales Reference | None None The cost of sales line.
gross_profit Reference | None None The gross profit line.
operating_expenses Reference | None None The operating expenses line.
operating_income Reference | None None The operating income line.
tax Reference | None None The tax line.
median_revenue float 48000000.0 The middle filer's revenue.
p99_multiple float 90.0 How many times the median the largest filers are. Company size is more unequal than almost anything else this catalog generates.
gross_margin float 0.42 Gross profit as a share of revenue, before the per-filer spread.
opex_ratio float 0.31 Operating expenses as a share of revenue, before the spread.
ratio_spread float 0.08 How much the margins vary between filers, as a standard deviation on the share. Zero makes every filer identical.
tax_rate float 0.21 Rate applied to operating income. No tax is charged on a loss, which is the simplification this type makes and states.
minor_units int 0 Decimal places. Filings round.

filings.narrative#

A paragraph of management discussion whose words agree with the figures.

Priority P1 · dtype string · equivalence structural

Example params: current=Reference(table='filings', column='revenue') prior=Reference(table='filings', column='prior_revenue') subject='Revenue'

Field Type Default Description
current Reference — This period's figure — usually revenue.
prior Reference — The comparative figure for the period before.
subject str 'Revenue' What the paragraph is about, as it should read at the start of a sentence.

Catalog and orders#

retail.category#

A product's place in a two-level taxonomy.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None depth='aisle'

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.
depth Literal['department', 'aisle'] 'aisle' The department alone, or the department and the aisle within it.

retail.discount_code#

A coupon code that says what it does.

Priority P0 · dtype string · equivalence categorical

Example params: codes=40 percentages=(5, 10, 15, 20, 25, 30)

Field Type Default Description
codes int 40 Distinct codes in circulation. A campaign runs a handful, not one per order.
percentages tuple[int, ...] (5, 10, 15, 20, 25, 30) The discounts on offer, which become the number in the code.

retail.order#

An order number, unique by construction and carrying its date.

Priority P0 · dtype string · equivalence unique

Example params: first_year=2024 days=365

Field Type Default Description
first_year int 2024 Earliest year in the column.
days int 365 How many consecutive days of trading the column spans.

retail.order_status#

Where an order has got to, further along the older it is.

Priority P0 · dtype string · equivalence categorical

Example params: age_days=None settle_days=6.0 cancelled_share=0.04 returned_share=0.06

Field Type Default Description
age_days Reference | None None A column of days since the order was placed. Given one, older orders are further along; given none, the stage is drawn flat over the lifecycle.
settle_days float 6.0 Days by which a typical order has reached Delivered. Ignored without an age column.
cancelled_share float 0.04 Share of orders cancelled before despatch.
returned_share float 0.06 Share of delivered orders sent back.

retail.price#

A shelf price, ending where a retailer would have put it.

Priority P0 · dtype float64 · equivalence continuous

Example params: median=24.0 p99_multiple=18.0 minimum=1.0 charm=True weights=None

Field Type Default Description
median float 24.0 The middle product's price.
p99_multiple float 18.0 How many times the median the dearest one percent are.
minimum float 1.0 The cheapest thing the shop sells. The distribution's low tail is floored here, because a snapped price with nothing below it comes out at zero.
charm bool True Snap to a retail ending. Off, the price is the raw figure rounded to the penny, which is what a wholesale or contract price looks like.
weights tuple[float, ...] | None None Relative frequency per ending, in the order 99, 95, 00, 50. The defaults when omitted.

retail.product_name#

A product name assembled from three word lists.

Priority P0 · dtype string · equivalence structural

Example params: material=True

Field Type Default Description
material bool True Include the material word. A shorter name is two words rather than three.

retail.shipping#

How an order is being sent, or what that costs.

Priority P0 · dtype string · equivalence categorical

Example params: field='method' method=None weights=None surcharge=0.0

Field Type Default Description
field Literal['method', 'cost'] 'method' The carrier service, or its charge. Declare one column of each.
method Reference | None None A retail.shipping method column. Required by a cost column, which is how the charge matches the service.
weights tuple[float, ...] | None None Relative frequency per method. The defaults when omitted.
surcharge float 0.0 Added to every non-free method, for a fuel or handling charge.

retail.sku#

A stock code, distinct per product and repeated across order lines.

Priority P0 · dtype string · equivalence structural

Example params: products=2000 prefix_letters=3

Field Type Default Description
products int 2000 Distinct products in the column. An order-line table has far more rows than products.
prefix_letters int 3 Letters in front of the serial, standing for a department or a supplier.

Baskets#

basket.item#

Which product is on this line, given what else is in the basket.

Priority P0 · dtype int64 · equivalence continuous

Example params: lines=6 products=5000 themes=40 affinity=0.7

Field Type Default Description
lines int 6 Lines in one basket, in consecutive rows.
products int 5000 Products in the catalogue. The value is an index into it.
themes int 40 Groups the catalogue divides into — aisles, occasions, whatever goes together. A basket draws one and shops mostly inside it.
affinity float 0.7 Share of a basket's lines drawn from its own theme rather than from the whole catalogue. Zero makes the lines independent.

basket.line#

How many units are on the line, or what the line came to.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='amount' unit_price=Reference(table='basket', column='price') quantity=Reference(table='basket', column='quantity') single_share=0.62 maximum=12 minor_units=2

Field Type Default Description
field Literal['quantity', 'amount'] 'quantity' The number of units, or the line's money value. Declare one of each.
unit_price Reference | None None A price column. Required by an amount column, which is the quantity times it.
quantity Reference | None None A basket.line quantity column. Required by an amount column.
single_share float 0.62 Share of lines that are a single unit. Most of a basket is ones.
maximum int 12 The largest quantity on any one line.
minor_units int 2 Decimal places for money.

basket.total#

What the basket came to, carried on every line of it.

Priority P0 · dtype float64 · equivalence continuous

Example params: line_amount=Reference(table='basket', column='amount') lines=6 minor_units=2

Field Type Default Description
line_amount Reference — A basket.line amount column of the same table, one row per line.
lines int 6 Lines in one basket. Must match the item column's.
minor_units int 2 Decimal places for money.

Reviews#

review.body#

Two sentences of review text, keyed to the rating.

Priority P0 · dtype string · equivalence structural

Example params: stars=Reference(table='review', column='stars')

Field Type Default Description
stars Reference — A review.rating column of the same table. The body is written to agree with it.

review.helpfulness#

How many people voted on the review, and how many said yes.

Priority P0 · dtype int64 · equivalence continuous

Example params: field='helpful' votes=Reference(table='review', column='votes') silent_share=0.68 tail_index=1.6 maximum=5000 helpful_share=0.74 agreement=6.0

Field Type Default Description
field Literal['votes', 'helpful'] 'votes' Total votes cast, or how many of them called the review helpful.
votes Reference | None None A review.helpfulness votes column. Required by a helpful column, which cannot exceed it.
silent_share float 0.68 Share of reviews nobody votes on at all. Most of them.
tail_index float 1.6 Power-law index of the vote tail among the reviews that get any. Lower is heavier: the top review pulls further away from the rest.
maximum int 5000 Ceiling on the votes one review can attract.
helpful_share float 0.74 Share of votes that call a review helpful, averaged over reviews.
agreement float 6.0 How much reviews resemble each other in how helpful people found them. Higher pulls every review towards the average share; lower spreads them out.

review.rating#

How many stars, from a J-shaped distribution rather than a bell.

Priority P0 · dtype int64 · equivalence categorical

Example params: shares=(0.13, 0.06, 0.08, 0.16, 0.57) product=Reference(table='review', column='product') products=1000 product_correlation=0.35

Field Type Default Description
shares tuple[float, float, float, float, float] (0.13, 0.06, 0.08, 0.16, 0.57) Share of reviews at one, two, three, four and five stars. Normalised.
product Reference | None None A product key column. Reviews of the same product then agree with each other, at the rate product_correlation sets.
products int 1000 How many distinct products the key column can hold. Only read when one is declared.
product_correlation float 0.35 Share of a rating's variance that belongs to the product rather than the reviewer. Zero makes reviews of one product no more alike than any other pair.

review.title#

A review headline that agrees with the stars beside it.

Priority P0 · dtype string · equivalence categorical

Example params: stars=Reference(table='review', column='stars')

Field Type Default Description
stars Reference — A review.rating column of the same table. The title is written to agree with it.

review.verified#

Whether the platform could tie the review to a purchase.

Priority P0 · dtype bool · equivalence categorical

Example params: centre_share=0.72 stars=Reference(table='review', column='stars') polarisation_penalty=0.55

Field Type Default Description
centre_share float 0.72 Share of three-star reviews the platform could match to a purchase. The rate for the whole column comes out lower, because the penalty only ever subtracts.
stars Reference | None None A review.rating column. Declaring it makes unverified reviews the more polarised ones, which is the pattern a fake-review filter is looking for.
polarisation_penalty float 0.55 Log-odds taken off the verified rate for each star away from the middle of the scale. Zero makes the flag independent of the rating.

Customer behaviour#

customer.churn#

Whether a customer has been silent longer than they usually are.

Priority P0 · dtype bool · equivalence categorical

Example params: recency=Reference(table='customer', column='recency') usual_gap=Reference(table='customer', column='usual_gap') multiple=3.0 fixed_gap_days=45.0 softness=0.6

Field Type Default Description
recency Reference — Days since the customer last bought.
usual_gap Reference | None None The customer's own typical gap between purchases. Declaring it makes the rule relative to their habit rather than to one number for everybody.
multiple float 3.0 How many of their usual gaps a customer can be silent for before they count as churned.
fixed_gap_days float 45.0 The gap to use when no usual_gap column is declared.
softness float 0.6 How blurred the boundary is, in usual gaps. Small makes the rule a step; large makes churn a slow function of how quiet somebody has gone.

customer.interpurchase#

Days from one purchase to the next, for a population that is not all alike.

Priority P0 · dtype float64 · equivalence continuous

Example params: mean_days=45.0 heterogeneity=2.4 customer=Reference(table='order', column='customer') customers=10000

Field Type Default Description
mean_days float 45.0 Average gap across the whole population, in days.
heterogeneity float 2.4 Gamma shape on the purchase rate. Low means customers differ wildly in how often they buy; high means they are nearly all the same. It has to exceed one: at or below one the average gap is infinite and mean_days would be meaningless.
customer Reference | None None A customer key column. Declaring it gives each customer one rate that all of their gaps are drawn from, which is what makes a panel a panel.
customers int 10000 How many distinct customers the key column can hold.

customer.lifetime_value#

What a customer is worth from here on, discounted.

Priority P0 · dtype float64 · equivalence continuous

Example params: order_value=Reference(table='customer', column='order_value') annual_orders=Reference(table='customer', column='annual_orders') margin=0.35 annual_churn=0.25 discount_rate=0.1 minor_units=2

Field Type Default Description
order_value Reference — The customer's average order value.
annual_orders Reference — How many orders they place in a year.
margin float 0.35 Contribution margin on an order.
annual_churn float 0.25 Share of customers lost per year. Its reciprocal is the expected lifetime.
discount_rate float 0.1 Annual discount rate on future margin. Zero values a customer at the undiscounted sum of what they will spend.
minor_units int 2 Decimal places for money.

customer.rfm#

The named RFM segment, computed from the three columns it is named after.

Priority P0 · dtype string · equivalence categorical

Example params: recency=Reference(table='customer', column='recency') frequency=Reference(table='customer', column='frequency') monetary=Reference(table='customer', column='monetary') recency_cuts=(30.0, 90.0, 180.0, 365.0) frequency_cuts=(1.0, 3.0, 6.0, 12.0) monetary_cuts=(50.0, 200.0, 600.0, 2000.0)

Field Type Default Description
recency Reference — Days since the customer last bought. Lower scores higher.
frequency Reference — How many times they have bought. Higher scores higher.
monetary Reference — What they have spent in total. Higher scores higher.
recency_cuts tuple[float, float, float, float] (30.0, 90.0, 180.0, 365.0) Ascending day boundaries between the five recency scores. A customer inside the first is a five.
frequency_cuts tuple[float, float, float, float] (1.0, 3.0, 6.0, 12.0) Ascending order-count boundaries between the five frequency scores.
monetary_cuts tuple[float, float, float, float] (50.0, 200.0, 600.0, 2000.0) Ascending spend boundaries between the five monetary scores.

customer.seasonality#

How much busier than average a date is - a multiplier, not a quantity.

Priority P0 · dtype float64 · equivalence continuous

Example params: date=Reference(table='order', column='placed') annual_amplitude=0.3 peak_day=330 weekly_amplitude=0.15 peak_weekday=5

Field Type Default Description
date Reference — A date or timestamp column. The index is a function of it.
annual_amplitude float 0.3 How far the yearly cycle swings either side of one.
peak_day int 330 Day of the year the annual cycle peaks on. 330 is late November.
weekly_amplitude float 0.15 How far the weekly cycle swings.
peak_weekday int 5 Weekday the weekly cycle peaks on, Monday being zero. 5 is Saturday.

customer.subscription#

Which plan a customer is on, or what they pay for it every month.

Priority P0 · dtype string · equivalence categorical

Example params: field='plan' plan=None shares=(0.55, 0.24, 0.13, 0.06, 0.02) prices=(0.0, 29.0, 99.0, 299.0, 999.0) names=('Free', 'Starter', 'Growth', 'Business', 'Enterprise') discount_share=0.18 maximum_discount=0.35 minor_units=2

Field Type Default Description
field Literal['plan', 'mrr'] 'plan' The plan name, or the monthly revenue it brings in.
plan Reference | None None A customer.subscription plan column. Required by an mrr column, whose value is that plan's price.
shares tuple[float, ...] (0.55, 0.24, 0.13, 0.06, 0.02) Share of customers on each plan, bottom rung first. Normalised.
prices tuple[float, ...] (0.0, 29.0, 99.0, 299.0, 999.0) Monthly list price of each plan, in the same order as the names.
names tuple[str, ...] ('Free', 'Starter', 'Growth', 'Business', 'Enterprise') Plan names, cheapest first.
discount_share float 0.18 Share of paying customers on a negotiated discount. Free-plan customers are never discounted, having nothing to discount.
maximum_discount float 0.35 The deepest discount anybody has negotiated.
minor_units int 2 Decimal places for money.

C.5 — Technical and operational#

Web and app logs#

log.combined#

One line of the Apache and Nginx combined log format.

Priority P0 · dtype string · equivalence structural

Example params: when=Reference(table='access', column='placed') status=Reference(table='access', column='status') path=Reference(table='access', column='path') client=Reference(table='access', column='client') method=Reference(table='access', column='verb') sent=Reference(table='access', column='sent') referrer=Reference(table='access', column='referrer') agent=Reference(table='access', column='agent') user=None utc_offset='+0000' protocol='HTTP/1.1'

Field Type Default Description
when Reference — A timestamp column.
status Reference — A request.status column.
path Reference — A request.path column.
client Reference | None None A network.ipv4 column. Rendered as - when absent.
method Reference | None None A request.method column. Defaults to GET.
sent Reference | None None A request.bytes column. Rendered as - when absent.
referrer Reference | None None A request.referrer column.
agent Reference | None None A request.user_agent column.
user Reference | None None An authenticated username column, if the tier has one.
utc_offset str '+0000' The offset written after the timestamp. The format demands one and it is not derived from the timestamp, which carries no zone.
protocol str 'HTTP/1.1' The request protocol.

log.firewall#

One line of a W3C-extended firewall log.

Priority P1 · dtype string · equivalence structural

Example params: when=Reference(table='access', column='placed') source=Reference(table='access', column='client') destination=Reference(table='access', column='peer') source_port=Reference(table='access', column='port') destination_port=None size=None action_shares=(0.86, 0.139, 0.001)

Field Type Default Description
when Reference — A timestamp column.
source Reference — A network.ipv4 column for the source.
destination Reference — A network.ipv4 column for the destination.
source_port Reference | None None A network.port column for the source.
destination_port Reference | None None A network.port column for the destination.
size Reference | None None A packet size column.
action_shares tuple[float, ...] (0.86, 0.139, 0.001) Share of ALLOW, DROP and INFO-EVENTS-LOST.

log.json#

One line of a structured application log.

Priority P0 · dtype string · equivalence structural

Example params: shares=(0.18, 0.71, 0.078, 0.031, 0.001) incident_window=1000 windows=100000 incident_rate=0.01 when=Reference(table='access', column='placed') service='checkout-api' trace=True

Field Type Default Description
shares tuple[float, ...] (0.18, 0.71, 0.078, 0.031, 0.001) Share of DEBUG, INFO, WARN, ERROR and FATAL outside an incident.
incident_window int 1000 Lines in one window. An incident fills a whole one.
windows int 100000 Windows laid out. Must cover the run.
incident_rate float 0.01 Share of windows that are an incident.
when Reference — A timestamp column.
service str 'checkout-api' The service name on every line.
trace bool True Include a trace identifier, as a traced service would.

log.syslog#

One syslog line, in either of the two formats that exist.

Priority P0 · dtype string · equivalence structural

Example params: shares=(0.18, 0.71, 0.078, 0.031, 0.001) incident_window=1000 windows=100000 incident_rate=0.01 when=Reference(table='access', column='placed') host=Reference(table='access', column='hostname') format='rfc5424' application='checkoutd' hostname='gateway.example.test'

Field Type Default Description
shares tuple[float, ...] (0.18, 0.71, 0.078, 0.031, 0.001) Share of DEBUG, INFO, WARN, ERROR and FATAL outside an incident.
incident_window int 1000 Lines in one window. An incident fills a whole one.
windows int 100000 Windows laid out. Must cover the run.
incident_rate float 0.01 Share of windows that are an incident.
when Reference — A timestamp column.
host Reference | None None A network.hostname column. Defaults to the fixed name below.
format Literal['rfc5424', 'rfc3164'] 'rfc5424' RFC 5424, the current one, or RFC 3164, the one still in the wild.
application str 'checkoutd' The application name.
hostname str 'gateway.example.test' The host name, when no host column is given.

log.windows_event#

One Windows event log record, as its text export.

Priority P1 · dtype string · equivalence structural

Example params: when=Reference(table='access', column='placed') computer=None hostname='WS-0417.example.test' shares=None

Field Type Default Description
when Reference — A timestamp column.
computer Reference | None None A network.hostname column. Defaults to the fixed name below.
hostname str 'WS-0417.example.test' The computer name, when no column is given.
shares tuple[float, ...] | None None Share of each event. Even if omitted.

Request attributes#

request.bytes#

How many bytes of body went back, including the zeroes.

Priority P0 · dtype int64 · equivalence continuous

Example params: median=3400.0 p99_multiple=120.0 status=Reference(table='access', column='status') method=Reference(table='access', column='verb')

Field Type Default Description
median float 3400.0 Median response size in bytes.
p99_multiple float 120.0 How many times the median the 99th percentile is. Assets are enormous next to a JSON response and both are in the same column.
status Reference | None None A request.status column. A 204 and a 304 have no body at all.
method Reference | None None A request.method column. A HEAD has no body whatever the status says.

request.latency#

How long the request took, in milliseconds.

Priority P0 · dtype float64 · equivalence continuous

Example params: median_ms=42.0 p99_multiple=14.0 status=Reference(table='access', column='status') not_modified_multiple=0.18 server_error_multiple=6.5

Field Type Default Description
median_ms float 42.0 The median response time.
p99_multiple float 14.0 How many times the median the 99th percentile sits at. Web latency has a long tail and this is the honest way to say how long.
status Reference | None None A request.status column. Declaring it makes 304s fast and 5xx slow, which is what a latency-by-status panel is for.
not_modified_multiple float 0.18 How much faster a 304 is. It serves no body, so it is mostly a header.
server_error_multiple float 6.5 How much slower a 5xx is. A 504 is slow by definition, and a 500 is usually something that tried for a while before giving up.

request.method#

The HTTP method, at the weights a public web tier actually sees.

Priority P0 · dtype string · equivalence categorical

Example params: methods=('GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'HEAD', 'OPTIONS') shares=(0.831, 0.104, 0.017, 0.008, 0.006, 0.028, 0.006)

Field Type Default Description
methods tuple[str, ...] ('GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'HEAD', 'OPTIONS') The methods this tier accepts.
shares tuple[float, ...] (0.831, 0.104, 0.017, 0.008, 0.006, 0.028, 0.006) Share of each. Normalised.

request.path#

The path part of the request line, with traffic concentrated where it really is.

Priority P0 · dtype string · equivalence structural

Example params: sections=('', 'about', 'blog', 'docs', 'pricing', 'search', 'login', 'account', 'api', 'static', 'products', 'support', 'careers', 'legal') leaves=('index', 'overview', 'getting-started', 'reference', 'changelog', 'faq', 'contact', 'terms', 'privacy', 'settings', 'profile', 'billing', 'download', 'status') identified_share=0.22 identifiers=5000 concentration=4.0

Field Type Default Description
sections tuple[str, ...] ('', 'about', 'blog', 'docs', 'pricing', 'search', 'login', 'account', 'api', 'static', 'products', 'support', 'careers', 'legal') First path segment. Empty means the root.
leaves tuple[str, ...] ('index', 'overview', 'getting-started', 'reference', 'changelog', 'faq', 'contact', 'terms', 'privacy', 'settings', 'profile', 'billing', 'download', 'status') Last path segment.
identified_share float 0.22 Share of paths ending in a numeric id instead of a named leaf.
identifiers int 5000 How many distinct ids the identified paths use.
concentration float 4.0 How unevenly traffic falls across paths. One is flat, which no site is; the default puts about a fifth of all requests on the busiest single path.

request.query#

The query string, when there is one.

Priority P0 · dtype string · equivalence structural

Example params: keys=('q', 'page', 'sort', 'utm_source', 'utm_campaign', 'ref', 'limit', 'lang', 'id', 'tab') values=('1', '2', '10', 'asc', 'desc', 'newsletter', 'email', 'en', 'fr', 'de', 'recent', 'popular', 'widget', 'true', 'false') empty_share=0.63 max_pairs=3

Field Type Default Description
keys tuple[str, ...] ('q', 'page', 'sort', 'utm_source', 'utm_campaign', 'ref', 'limit', 'lang', 'id', 'tab') Parameter names.
values tuple[str, ...] ('1', '2', '10', 'asc', 'desc', 'newsletter', 'email', 'en', 'fr', 'de', 'recent', 'popular', 'widget', 'true', 'false') Parameter values.
empty_share float 0.63 Share of requests with no query string.
max_pairs int 3 Most parameters any one request carries.

request.referrer#

Where the request said it came from, or nothing at all.

Priority P0 · dtype string · equivalence structural

Example params: hosts=('https://search.example.com/', 'https://www.example.org/links', 'https://news.example.net/story', 'https://forum.example.invalid/thread', 'https://social.example.test/post') direct_share=0.41 internal_share=0.44 site='https://www.example.com' origin=Reference(table='access', column='path')

Field Type Default Description
hosts tuple[str, ...] ('https://search.example.com/', 'https://www.example.org/links', 'https://news.example.net/story', 'https://forum.example.invalid/thread', 'https://social.example.test/post') External referrers. Reserved domains only, by default.
direct_share float 0.41 Share with no referrer at all — typed, bookmarked, or stripped.
internal_share float 0.44 Share referred by the site itself. The rest are external.
site str 'https://www.example.com' This site's own origin, for the internal referrers.
origin Reference | None None A request.path column. Declaring it makes an internal referrer a page that exists rather than one made up.

request.status#

The HTTP status code, with failures arriving in bursts rather than at a rate.

Priority P0 · dtype int64 · equivalence categorical

Example params: codes=(200, 201, 204, 301, 302, 304, 400, 401, 403, 404, 429, 500, 502, 503, 504) shares=(0.72, 0.021, 0.014, 0.019, 0.043, 0.108, 0.011, 0.008, 0.004, 0.038, 0.003, 0.005, 0.003, 0.002, 0.001) burst_window=2000 windows=100000 burst_rate=0.01 burst_error_share=0.45

Field Type Default Description
codes tuple[int, ...] (200, 201, 204, 301, 302, 304, 400, 401, 403, 404, 429, 500, 502, 503, 504) The codes this tier can emit.
shares tuple[float, ...] (0.72, 0.021, 0.014, 0.019, 0.043, 0.108, 0.011, 0.008, 0.004, 0.038, 0.003, 0.005, 0.003, 0.002, 0.001) Share of each code outside a burst. Normalised.
burst_window int 2000 Requests in one window. A burst lasts a whole window, so this is the outage length measured in requests rather than in minutes.
windows int 100000 How many windows the column lays out. Must cover the run.
burst_rate float 0.01 Share of windows that are an outage.
burst_error_share float 0.45 Share of requests inside a burst that fail with a 5xx.

request.user_agent#

What the client said it was, crawlers included.

Priority P0 · dtype string · equivalence categorical

Example params: agents=('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15', 'Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0', 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Mobile/15E148 Safari/604.1', 'Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Mobile Safari/537.36', 'curl/8.7.1', 'python-requests/2.32.3') bots=('Mozilla/5.0 (compatible; ExampleBot/2.1; +https://www.example.com/bot)', 'Mozilla/5.0 (compatible; ExampleCrawler/1.0; +https://crawler.example.org/about)', 'ExampleMonitor/3.4 (+https://status.example.net)', 'Mozilla/5.0 (compatible; ExampleFetch/0.9; +https://example.invalid/fetch)') bot_share=0.28

Field Type Default Description
agents tuple[str, ...] ('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15', 'Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0', 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Mobile/15E148 Safari/604.1', 'Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Mobile Safari/537.36', 'curl/8.7.1', 'python-requests/2.32.3') Browser and client agents.
bots tuple[str, ...] ('Mozilla/5.0 (compatible; ExampleBot/2.1; +https://www.example.com/bot)', 'Mozilla/5.0 (compatible; ExampleCrawler/1.0; +https://crawler.example.org/about)', 'ExampleMonitor/3.4 (+https://status.example.net)', 'Mozilla/5.0 (compatible; ExampleFetch/0.9; +https://example.invalid/fetch)') Crawler agents. Invented names by default.
bot_share float 0.28 Share of requests from crawlers and monitors. Higher than people expect.

Session identity#

session.cookie#

A cookie value, constant for the whole session.

Priority P0 · dtype string · equivalence structural

Example params: shape=SessionShape(sessions=200000, layout='sessions', length_shares=(0.41, 0.19, 0.11, 0.08, 0.06, 0.05, 0.035, 0.025, 0.02, 0.01)) form='signed' width=32 version='v2' signature_width=27

Field Type Default Description
shape SessionShape PydanticUndefined How requests group into sessions. Must match the identifier column's.
form Literal['opaque', 'signed'] 'signed' An opaque blob, or the three dot-separated parts a signed cookie has.
width int 32 Base64url characters in the payload.
version str 'v2' The version tag a signed cookie carries in front.
signature_width int 27 Base64url characters in the signature. 27 is a 160-bit MAC, which is what most frameworks emit.

session.identifier#

An opaque session identifier, constant for the whole session.

Priority P0 · dtype string · equivalence structural

Example params: shape=SessionShape(sessions=200000, layout='sessions', length_shares=(0.41, 0.19, 0.11, 0.08, 0.06, 0.05, 0.035, 0.025, 0.02, 0.01)) width=32 prefix=''

Field Type Default Description
shape SessionShape PydanticUndefined How requests group into sessions. Both session columns repeat it.
width int 32 Hex characters in the identifier. 32 is 128 bits, which is what a framework's default session key usually is. Narrow ones are allowed so the collision behaviour can be demonstrated, not because they are a good idea.
prefix str '' Written in front of every identifier, as some frameworks do.

Network#

network.attack_event#

Which attack pattern a row is part of, if any.

Priority P1 · dtype string · equivalence categorical

Example params: patterns=('port-scan', 'credential-stuffing', 'sql-injection-probe', 'path-traversal-probe', 'xss-probe', 'brute-force-login', 'dns-amplification', 'slowloris', 'scraping') shares=None window=64 windows=200000 episode_rate=0.15 intensity=0.7

Field Type Default Description
patterns tuple[str, ...] ('port-scan', 'credential-stuffing', 'sql-injection-probe', 'path-traversal-probe', 'xss-probe', 'brute-force-login', 'dns-amplification', 'slowloris', 'scraping') Pattern names. Labels, not payloads.
shares tuple[float, ...] | None None Share of each pattern within an episode. Even if omitted.
window int 500 Rows in one window. An episode fills a whole window, because a scan is a run of requests and not a scatter of them.
windows int 200000 Windows laid out. Must cover the run.
episode_rate float 0.02 Share of windows holding an episode.
intensity float 0.7 Share of rows inside an episode that belong to the attack. The rest is the ordinary traffic that was happening anyway.

network.domain#

A registrable domain, always on a name that cannot be delegated.

Priority P0 · dtype string · equivalence categorical

Example params: names=('acme', 'globex', 'initech', 'umbrella', 'soylent', 'hooli', 'vandelay', 'wonka', 'cyberdyne', 'tyrell', 'aperture', 'stark') tlds=('test', 'example', 'invalid', 'localhost') iana_share=0.3

Field Type Default Description
names tuple[str, ...] ('acme', 'globex', 'initech', 'umbrella', 'soylent', 'hooli', 'vandelay', 'wonka', 'cyberdyne', 'tyrell', 'aperture', 'stark') Second-level labels. Invented by default.
tlds tuple[str, ...] ('test', 'example', 'invalid', 'localhost') Reserved top-level names. RFC 2606 and RFC 6761 only — this list is checked, and a real TLD is refused.
iana_share float 0.3 Share drawn from example.com, .net and .org instead, which are RFC 2606 §3 names reserved by IANA for exactly this.

network.hostname#

A fully-qualified hostname on a reserved domain.

Priority P0 · dtype string · equivalence structural

Example params: hosts=('www', 'api', 'mail', 'db', 'cache', 'static', 'vpn', 'gateway', 'build', 'monitor', 'backup', 'ldap') numbered_share=0.35 fleet=64 parent=Reference(table='asset', column='domain')

Field Type Default Description
hosts tuple[str, ...] ('www', 'api', 'mail', 'db', 'cache', 'static', 'vpn', 'gateway', 'build', 'monitor', 'backup', 'ldap') The leftmost label.
numbered_share float 0.35 Share of hosts carrying an ordinal, as fleet machines do: web-07.
fleet int 64 Machines per host role.
parent Reference | None None A network.domain column. Declaring it puts the hostnames on domains that appear elsewhere in the table rather than on ones made up here.

network.ipv4#

An IPv4 address inside a CIDR block you name.

Priority P0 · dtype string · equivalence structural

Example params: cidr='203.0.113.0/24' clients=200 concentration=2.2

Field Type Default Description
cidr str '203.0.113.0/24' The block to draw inside. Defaults to RFC 5737 TEST-NET-3, which is reserved for documentation and never routed. It is a /24, so it holds 256 addresses; RFC 2544's 198.18.0.0/15 is the large reserved block if you need more.
clients int 0 How many distinct addresses to use. Zero means the whole block; a small number gives a log where the same hosts keep coming back, which is what a real one looks like.
concentration float 2.2 How unevenly traffic falls across the addresses. One is flat.

network.ipv6#

An IPv6 address inside a prefix, written out in full.

Priority P0 · dtype string · equivalence structural

Example params: prefix='2001:db8::/32'

Field Type Default Description
prefix str '2001:db8::/32' The prefix to draw inside. Defaults to RFC 3849, reserved for documentation and guaranteed not to be routed.

network.mac#

A MAC address no manufacturer ever assigned.

Priority P0 · dtype string · equivalence structural

Example params: separator=':' uppercase=False

Field Type Default Description
separator Literal[':', '-', ''] ':' What joins the octets. Colons, hyphens, or nothing.
uppercase bool False Render the hex in capitals.

network.port#

A TCP or UDP port, service ports and ephemeral ones both.

Priority P0 · dtype int64 · equivalence structural

Example params: services=(443, 80, 53, 22, 8080, 123, 3306, 993, 587, 25, 8443, 445, 3389, 5432, 6379, 143, 389, 110, 995, 21, 20, 23) service_share=0.5 concentration=2.0

Field Type Default Description
services tuple[int, ...] (443, 80, 53, 22, 8080, 123, 3306, 993, 587, 25, 8443, 445, 3389, 5432, 6379, 143, 389, 110, 995, 21, 20, 23) Service ports this capture sees.
service_share float 0.5 Share of rows that are a service port rather than an ephemeral one. Half is right for a capture holding both ends of each connection.
concentration float 2.0 How unevenly traffic falls across the service ports. Real captures are mostly 443 and 80.

network.tls_certificate#

One field of a certificate's metadata.

Priority P0 · dtype string · equivalence structural

Example params: field='subject' subject=Reference(table='asset', column='hostname') authority='Example Root CA'

Field Type Default Description
field Literal['serial', 'fingerprint', 'issuer', 'subject', 'signature_algorithm', 'key_algorithm'] 'serial' Which piece of the certificate this column holds.
subject Reference | None None A network.hostname or network.domain column, for the subject and issuer common names. Without it they are built here.
authority str 'Example Root CA' The issuing authority's name. Invented by default.

Manufacturing#

manufacturing.batch#

The lot a unit was made in.

Priority P0 · dtype string · equivalence exact

Example params: shape=LotShape(units=480) prefix='L' start='2026-01-05' lots_per_day=3

Field Type Default Description
shape LotShape PydanticUndefined Lot layout.
prefix str 'L' Written before the code.
start str '2026-01-05' The date the first lot was made. Lots are date-coded, one per day.
lots_per_day int 3 Lots the line completes in a day.

manufacturing.bom#

One line of a bill of materials.

Priority P1 · dtype int64 · equivalence categorical

Example params: field='quantity' widths=(40, 8, 15) max_quantity=6

Field Type Default Description
field Literal['parent', 'quantity', 'level'] 'parent' The row this line hangs under, how many of it are needed, or how deep it sits. Declare all three: a BOM without quantities cannot be exploded.
widths tuple[int, ...] (8, 5, 4) Children per node at each level. (8, 5, 4) is eight assemblies, each of five sub-assemblies, each of four parts.
max_quantity int 6 Most of one child a parent ever needs.

manufacturing.cycle_time#

How long an operation took, in seconds.

Priority P0 · dtype float64 · equivalence continuous

Example params: station=Reference(table='unit', column='station') medians=(42.0, 186.0, 254.0, 97.0, 61.0, 900.0, 148.0, 74.0, 33.0) names=('OP10-SAW', 'OP20-TURN', 'OP30-MILL', 'OP40-DRILL', 'OP50-DEBURR', 'OP60-HEAT', 'OP70-GRIND', 'OP80-INSPECT', 'OP90-PACK') spread=0.18 default_median=120.0

Field Type Default Description
station Reference | None None A manufacturing.workstation column. Cycle times differ by an order of magnitude across a routing, and one distribution over all of them hides that.
medians tuple[float, ...] (42.0, 186.0, 254.0, 97.0, 61.0, 900.0, 148.0, 74.0, 33.0) Median seconds per station, in the same order as the station names.
names tuple[str, ...] ('OP10-SAW', 'OP20-TURN', 'OP30-MILL', 'OP40-DRILL', 'OP50-DEBURR', 'OP60-HEAT', 'OP70-GRIND', 'OP80-INSPECT', 'OP90-PACK') Station names the medians belong to.
spread float 0.18 Log-scale spread. An operation has a floor and a tail: it cannot beat the machine and it can always jam.
default_median float 120.0 Median for a station not on the list.

manufacturing.defect#

Whether a unit was scrapped.

Priority P0 · dtype bool · equivalence categorical

Example params: dimension=Reference(table='unit', column='bore') nominal=25.0 tolerance=0.05 attribute_rate=0.004

Field Type Default Description
dimension Reference | None None A manufacturing.dimension column. A unit outside the band is scrap, which is the join every yield analysis starts with.
nominal float 25.0 The drawing dimension.
tolerance float 0.05 Plus and minus.
attribute_rate float 0.004 Share scrapped for something no gauge sees — a scratch, a short shot, a missing feature. Independent of the dimension, because those faults are.

manufacturing.dimension#

A measured dimension, against a nominal and a tolerance.

Priority P0 · dtype float64 · equivalence continuous

Example params: nominal=25.0 tolerance=0.05 capability=1.33 centring=0.25 decimals=3

Field Type Default Description
nominal float 25.0 The drawing dimension, in millimetres.
tolerance float 0.05 Plus and minus, in millimetres. A symmetric band.
capability float 1.33 Cpk. Nobody knows their sigma and everybody has this on a chart, so it is the parameter and the spread is derived from it. 1.33 is the usual floor.
centring float 0.25 Where the process sits inside the band, as a fraction of the tolerance. Zero is dead centre, which almost no real process is.
decimals int 3 Places the gauge reports.

manufacturing.material#

The material a part is specified in.

Priority P0 · dtype string · equivalence categorical

Example params: grades=('AISI 304', 'AISI 316L', 'AISI 1045', 'AISI 4140', 'Al 6061-T6', 'Al 7075-T651', 'Ti-6Al-4V', 'C36000', 'PA66-GF30', 'PEEK', 'EN-GJL-250', 'Inconel 718') shares=None part=Reference(table='unit', column='part')

Field Type Default Description
grades tuple[str, ...] ('AISI 304', 'AISI 316L', 'AISI 1045', 'AISI 4140', 'Al 6061-T6', 'Al 7075-T651', 'Ti-6Al-4V', 'C36000', 'PA66-GF30', 'PEEK', 'EN-GJL-250', 'Inconel 718') Standard designations.
shares tuple[float, ...] | None None Share of each. Even if omitted.
part Reference | None None A manufacturing.part_number column. Declaring it makes a part's material a property of the design rather than of the unit, which is what a specification is.

manufacturing.part_number#

The part number of the design a unit is an instance of.

Priority P0 · dtype string · equivalence structural

Example params: families=('BRK', 'HSG', 'SHF', 'GSK', 'VLV', 'PLT', 'CLP', 'BSH') designs=350 revisions=4 concentration=2.4

Field Type Default Description
families tuple[str, ...] ('BRK', 'HSG', 'SHF', 'GSK', 'VLV', 'PLT', 'CLP', 'BSH') Family prefixes.
designs int 350 How many distinct parts the plant makes.
revisions int 4 Revisions in circulation for a given part.
concentration float 2.4 How unevenly volume falls across the parts. A plant runs a few parts in quantity and most of them rarely; at one it would be flat, which no plant is.

manufacturing.qc_measurement#

The process capability the measurements in a lot actually show.

Priority P0 · dtype float64 · equivalence continuous

Example params: dimension=Reference(table='unit', column='bore') shape=LotShape(units=64) nominal=25.0 tolerance=0.05

Field Type Default Description
dimension Reference — A manufacturing.dimension column. The capability is measured on it.
shape LotShape PydanticUndefined Lot layout. The capability is computed over a whole lot, which is the sample a quality system reports it for.
nominal float 25.0 The drawing dimension.
tolerance float 0.05 Plus and minus.

manufacturing.serial#

The serial number of one unit.

Priority P0 · dtype string · equivalence exact

Example params: shape=LotShape(units=480) width=6 first=1

Field Type Default Description
shape LotShape PydanticUndefined Lot layout.
width int 6 Digits in the sequence.
first int 1 The first serial the line issued.

manufacturing.workstation#

Which station a unit is at.

Priority P0 · dtype string · equivalence exact

Example params: stations=('OP10-SAW', 'OP20-TURN', 'OP30-MILL', 'OP40-DRILL', 'OP50-DEBURR', 'OP60-HEAT', 'OP70-GRIND', 'OP80-INSPECT', 'OP90-PACK') routed=True

Field Type Default Description
stations tuple[str, ...] ('OP10-SAW', 'OP20-TURN', 'OP30-MILL', 'OP40-DRILL', 'OP50-DEBURR', 'OP60-HEAT', 'OP70-GRIND', 'OP80-INSPECT', 'OP90-PACK') Station names, in routing order.
routed bool True Visit the stations in order, one row per operation — which is what a traveller record is. Off, each row is an independent station.

Sensor and IoT#

sensor.anomaly#

An injected fault - a one-sample spike or a run of stuck readings.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='offset' shape=TraceShape(readings=64, sample_seconds=30.0) rate=0.2 spike_share=0.6 magnitude=9.0 run_length=24

Field Type Default Description
field Literal['offset', 'flag'] 'offset' The value to add to the reading, or a flag saying a fault is in progress. Declare both: the flag is the label a detector is scored against.
shape TraceShape PydanticUndefined Device layout.
rate float 0.004 Share of readings inside a fault — which is what somebody means by 'four in a thousand are faulty'. The rate faults begin at is derived from it and the run length, because a stuck fault twenty-four readings long contributes twenty-four faulty readings per onset and not one.
spike_share float 0.6 Share of faults that are a single-sample spike rather than a stuck run.
magnitude float 9.0 How large a spike is, in the reading's own units.
run_length int 24 Readings a stuck fault lasts.

sensor.climate#

Relative humidity or barometric pressure.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='humidity' shape=TraceShape(readings=2880, sample_seconds=30.0) temperature=Reference(table='reading', column='temperature') mean=58.0 swing=9.0 persistence=0.97 reference_c=18.5 coupling=-1.9

Field Type Default Description
field Literal['humidity', 'pressure'] 'humidity' Percent relative humidity, or hectopascals.
shape TraceShape PydanticUndefined Device layout.
temperature Reference | None None A sensor.temperature column. Humidity falls as temperature rises, and a pair drawn independently is the give-away in a climate file.
mean float 58.0 Average humidity in percent, or pressure in hectopascals.
swing float 9.0 Standard deviation of the trace about its mean.
persistence float 0.97 Both quantities move slowly. Pressure especially: a front takes hours.
reference_c float 18.5 The temperature the coupling is measured about — the degrees at which the mean above is the answer. A parameter and not the column's own average, because an average is a property of the whole column and a chunk cannot see it.
coupling float -1.9 Units of this quantity per degree, when a temperature column is given. Negative for humidity, which is what the physics says.

sensor.device_id#

The identifier of the device a reading came from.

Priority P0 · dtype string · equivalence exact

Example params: shape=TraceShape(readings=2880, sample_seconds=30.0) prefix='dev-' width=6 first=1

Field Type Default Description
shape TraceShape PydanticUndefined Device layout.
prefix str 'dev-' Written before the number.
width int 6 Digits in the number.
first int 1 The number the first device gets.

sensor.drift#

The slow error a sensor accumulates over its life.

Priority P0 · dtype float64 · equivalence continuous

Example params: shape=TraceShape(readings=2880, sample_seconds=30.0) per_block=0.8 wander=0.15

Field Type Default Description
shape TraceShape PydanticUndefined Device layout.
per_block float 0.8 How far a typical device drifts across its whole run, in the reading's own units. Signed: some devices drift up and some down.
wander float 0.15 How much the drift wanders about its straight line, as a share of per_block. Zero is a perfectly linear drift, which no sensor has.

sensor.gps_track#

One coordinate of a GPS track.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='latitude' shape=TraceShape(readings=2880, sample_seconds=30.0) origin_latitude=51.5 origin_longitude=-0.12 scatter_km=8.0 speed_m_s=7.5 speed_spread=0.45 stopped_share=0.12 heading_persistence=0.98

Field Type Default Description
field Literal['latitude', 'longitude'] 'latitude' Which coordinate.
shape TraceShape PydanticUndefined Device layout.
origin_latitude float 51.5 Where tracks start.
origin_longitude float -0.12 Likewise.
scatter_km float 8.0 How far apart devices start.
speed_m_s float 7.5 Median speed while moving.
speed_spread float 0.45 Log-scale spread of the speed. Zero makes every step the same length, which is a track no vehicle has driven.
stopped_share float 0.12 Share of readings taken while stationary. A parked vehicle still reports, and a track without any is one no dwell-time analysis can be tested on.
heading_persistence float 0.98 How smoothly the track turns. This is what makes a track a track rather than a cloud: a vehicle keeps going the way it was going. At zero the heading is fresh every step; near one the track curves gently. How far it wanders from where it set off does not change with it.

sensor.schedule#

When each reading was taken - regular, jittered, or event-driven.

Priority P0 · dtype datetime64 · equivalence continuous

Example params: shape=TraceShape(readings=2880, sample_seconds=30.0) cadence='jittered' start='2026-01-01T00:00:00' jitter_seconds=0.4 gap_share=0.08 gap_multiple=25.0

Field Type Default Description
shape TraceShape PydanticUndefined Device layout.
cadence Literal['regular', 'jittered', 'event'] 'regular' Exactly on the second, jittered around it, or event-driven with gaps.
start str '2026-01-01T00:00:00' When each device's first reading is taken.
jitter_seconds float 0.4 Standard deviation of the jitter.
gap_share float 0.08 For the event cadence: share of intervals that are a long gap instead.
gap_multiple float 25.0 How much longer a gap is.

sensor.temperature#

An ambient temperature trace, with a day in it.

Priority P0 · dtype float64 · equivalence continuous

Example params: shape=TraceShape(readings=2880, sample_seconds=30.0) mean_c=18.5 daily_swing_c=6.0 peak_hour=15.0 noise_c=0.4 persistence=0.93 site_spread_c=2.5

Field Type Default Description
shape TraceShape PydanticUndefined Device layout.
mean_c float 18.5 The average.
daily_swing_c float 6.0 Peak-to-trough over a day, halved either way.
peak_hour float 15.0 Hour of the daily maximum.
noise_c float 0.4 Standard deviation of the residual.
persistence float 0.93 How much of one reading's residual carries into the next. This is what makes it a trace: at zero it is noise with a sine under it.
site_spread_c float 2.5 How much devices differ from each other.

sensor.vibration#

A vibration amplitude trace, in millimetres per second RMS.

Priority P0 · dtype float64 · equivalence continuous

Example params: shape=TraceShape(readings=2880, sample_seconds=30.0) baseline_mm_s=1.8 harmonic_ratio=0.35 cycles_per_block=48.0 spread=0.22 persistence=0.55

Field Type Default Description
shape TraceShape PydanticUndefined Device layout.
baseline_mm_s float 1.8 RMS amplitude of a healthy machine.
harmonic_ratio float 0.35 Amplitude of the running-speed harmonic, relative to the baseline.
cycles_per_block float 48.0 How many times the machine's duty cycle repeats across a device's rows.
spread float 0.22 Log-scale spread of the residual. Amplitudes are positive and skewed, so the noise is multiplicative rather than added.
persistence float 0.55 Vibration is far less persistent than temperature: the mass is small and the excitation changes with the load.

Software artefacts#

artefact.commit_hash#

A commit-hash-shaped string.

Priority P0 · dtype string · equivalence structural

Example params: form='sha1' uppercase=False

Field Type Default Description
form Literal['sha1', 'sha256', 'short'] 'sha1' Forty hex characters, sixty-four, or the seven a short hash has.
uppercase bool False Render the hex in capitals.

artefact.file_path#

A path to a file in a source tree, POSIX or Windows.

Priority P0 · dtype string · equivalence structural

Example params: separator='/' directories=('src', 'lib', 'tests', 'docs', 'assets', 'build', 'vendor', 'scripts', 'config', 'internal', 'public', 'migrations') stems=('index', 'main', 'utils', 'helpers', 'config', 'client', 'server', 'model', 'schema', 'handler', 'worker', 'parser', 'logger', 'router', 'adapter', 'fixture') depth_shares=(0.08, 0.31, 0.34, 0.19, 0.08) drive='C:'

Field Type Default Description
separator Literal['/', '\'] '/' POSIX or Windows. Windows paths also take a drive letter.
directories tuple[str, ...] ('src', 'lib', 'tests', 'docs', 'assets', 'build', 'vendor', 'scripts', 'config', 'internal', 'public', 'migrations') Directory names.
stems tuple[str, ...] ('index', 'main', 'utils', 'helpers', 'config', 'client', 'server', 'model', 'schema', 'handler', 'worker', 'parser', 'logger', 'router', 'adapter', 'fixture') File name stems.
depth_shares tuple[float, ...] (0.08, 0.31, 0.34, 0.19, 0.08) Share of paths one directory deep, two, and so on.
drive str 'C:' Windows drive prefix.

artefact.file_size#

How many bytes a file is, given what kind of file it is.

Priority P0 · dtype int64 · equivalence continuous

Example params: media_type=Reference(table='file', column='mime') p99_multiple=40.0 default_median=8000.0 minimum=1

Field Type Default Description
media_type Reference | None None An artefact.mime_type column. Without it every file is drawn from one distribution, which no file tree has.
p99_multiple float 40.0 How many times its median the 99th percentile of a type sits at.
default_median float 8000.0 Median for an unrecognised media type.
minimum int 1 Smallest file the tree holds.

artefact.firmware#

A firmware version - four components and no precedence rules.

Priority P0 · dtype string · equivalence structural

Example params: shape=ReleaseShape(layout='releases', projects=20000, releases_shares=(0.18, 0.14, 0.12, 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.04, 0.03)) prefix='' build_base=1000 highest_start=24

Field Type Default Description
shape ReleaseShape PydanticUndefined Project layout.
prefix str '' A vendor letter or word in front, as some use.
build_base int 1000 Where a device's build numbers start.
highest_start int 24 The largest minor a device's numbering can already be at when the run begins. Firmware numbering carries over from earlier product lines, so a device's first row in a file is rarely its first release ever — and a column whose minor never passes nine will never demonstrate that 2.14 sorts before 2.9 as text.

artefact.mime_type#

The media type a file's extension implies.

Priority P0 · dtype string · equivalence categorical

Example params: path=Reference(table='file', column='path') unknown_type='application/octet-stream'

Field Type Default Description
path Reference | None None An artefact.file_path column. Without it the type is drawn on its own, which means it will not agree with any path column in the table.
unknown_type str 'application/octet-stream' What an unregistered extension maps to.

artefact.semver#

A semantic version, ascending through a project's releases.

Priority P0 · dtype string · equivalence categorical

Example params: shape=ReleaseShape(layout='releases', projects=20000, releases_shares=(0.18, 0.14, 0.12, 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.04, 0.03)) start_major=1 prerelease_share=0.12 build_share=0.0

Field Type Default Description
shape ReleaseShape PydanticUndefined Project layout.
start_major int 1 The major version a project's first release is in.
prerelease_share float 0.12 Share of releases carrying a prerelease tag.
build_share float 0.0 Share carrying build metadata after a plus.

C.6 — Academic and educational#

Enrolment#

academic.attendance#

Sessions attended, out of the sessions the course held.

Priority P0 · dtype int64 · equivalence continuous

Example params: sessions=45 rate=0.86 concentration=8.0

Field Type Default Description
sessions int 45 Sessions the course held. A three-hour-a-week semester course is 45.
rate float 0.86 Mean fraction of sessions attended, over the whole column.
concentration float 8.0 How tightly attendance clusters around that mean. Larger is tighter; below about 2 the column is closer to all-or-nothing than to a typical class.

academic.course_code#

A catalogue number - a subject, a level, and a number inside that level.

Priority P0 · dtype string · equivalence structural

Example params: levels=4

Field Type Default Description
levels int 4 How many course levels the catalogue has. 4 gives 100- to 400-level.

academic.course_sequence#

A course code whose level cannot outrun the term it is taken in.

Priority P0 · dtype string · equivalence structural

Example params: courses_per_term=4 terms=8 max_level=4

Field Type Default Description
courses_per_term int 4 Consecutive rows belonging to one term of one student's programme.
terms int 8 Terms in a programme, after which the pattern starts again on the next student.
max_level int 4 Highest course level the catalogue has. Reached from term this many onwards.

academic.credit_hours#

Credits one course registration carries, as a whole number.

Priority P0 · dtype int64 · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

academic.instructor#

A key for the member of faculty teaching this section.

Priority P0 · dtype string · equivalence categorical

Example params: instructors=24 skew=0.5

Field Type Default Description
instructors int 24 Faculty available to teach.
skew float 0.5 How unevenly the teaching is shared. 0 gives every instructor the same load; 1 gives the first twice the second's.

academic.program#

A major, drawn as often as students in the United States actually graduate in it.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None representation='title'

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.
representation Literal['title', 'code', 'both'] 'title' Write the programme as its title, its six-digit CIP code, or the code then the title.

academic.registration_status#

What became of one registration - registered, completed, waitlisted, dropped, withdrawn or auditing.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

academic.section#

Which sitting of a course a registration is in, and how that section meets.

Priority P0 · dtype string · equivalence categorical

Example params: sections=4 modality=True

Field Type Default Description
sections int 4 How many sections a course runs. Drawn uniformly.
modality bool True Append the modality letter: H hybrid, W online, nothing for in person.

academic.student_id#

An institutional student number - letters, the year of entry, then a serial.

Priority P0 · dtype string · equivalence unique

Example params: prefix='S' first_cohort=2021 cohort_years=4 digits=8

Field Type Default Description
prefix str 'S' Institution letters in front of the number. Up to four capitals, or empty for none.
first_cohort int 2021 Earliest year of entry in the column.
cohort_years int 4 How many consecutive years of entry the column spans. Drawn uniformly.
digits int 8 Length of the zero-padded serial after the year.

academic.term#

The academic term a registration belongs to, spelled one way.

Priority P0 · dtype string · equivalence categorical

Example params: first_year=2021 years=4 seasons=('Fall', 'Spring', 'Summer') season_weights=None

Field Type Default Description
first_year int 2021 Calendar year of the earliest term.
years int 4 How many consecutive years the column spans. Drawn uniformly.
seasons tuple[Literal['Winter', 'Spring', 'Summer', 'Fall'], ...] ('Fall', 'Spring', 'Summer') Which terms the institution runs. Any of Winter, Spring, Summer, Fall.
season_weights tuple[float, ...] | None None Relative frequency per season, in the order seasons gives them. The published defaults when omitted.

Outcomes#

academic.gpa#

A running credit-weighted average of the grade column beside it, per student.

Priority P0 · dtype float64 · equivalence continuous

Example params: grade=Reference(table='transcript', column='grade') credit_hours=Reference(table='transcript', column='credits') registrations_per_student=32 decimals=2

Field Type Default Description
grade Reference — An academic.letter_grade column of the same table. Required: a cumulative GPA with no grades behind it is a number, not an average.
credit_hours Reference | None None An academic.credit_hours column. Given one, courses are weighted by it; given none, every course counts once.
registrations_per_student int 32 Consecutive rows belonging to one student, after which the average starts again. Set it to courses_per_term times terms.
decimals int 2 Places the average is reported to. Registrars publish two.

academic.letter_grade#

A letter grade on the 4.0 scale, from an average and a lean you set.

Priority P0 · dtype string · equivalence categorical

Example params: scale='plus_minus' mean=3.1 spread=0.9 skew=-0.8

Field Type Default Description
scale Literal['plus_minus', 'letter'] 'plus_minus' The twelve-point scale with pluses and minuses, or the five bare letters.
mean float 3.1 Mean grade point of the column. Realised exactly, not approached. Around 3.1 for US undergraduates.
spread float 0.9 Standard deviation of the attainment behind the letters, in grade points. Larger scatters the column further from its mean.
skew float -0.8 Lean of that attainment. Negative puts the long tail at the bottom, which is the shape a real grade column has.

academic.rubric_score#

A rubric total - so many criteria, each scored on a level scale, added up.

Priority P0 · dtype int64 · equivalence continuous

Example params: criteria=4 levels=4 mean_level=3.1 concentration=8.0

Field Type Default Description
criteria int 4 Rows the rubric has. Each is scored from 1 to levels.
levels int 4 Points on each row's scale. Four is beginning, developing, proficient, exemplary.
mean_level float 3.1 Mean score per criterion, between 1 and levels. The total averages this times criteria.
concentration float 8.0 How alike the candidates are. Larger clusters them around the mean; below about 2 the column is closer to all-or-nothing than to a marked class.

Persistence#

academic.attrition#

What became of a student at the end of a term - continued, stopped out, or withdrew.

Priority P0 · dtype string · equivalence categorical

Example params: hazard=Reference(table='transcript', column='hazard') rate=0.12 return_rate=0.45

Field Type Default Description
hazard Reference | None None An academic.persistence_hazard column of the same table. Given one, each row is drawn at its own hazard; given none, the whole column is drawn at rate.
rate float 0.12 Probability of leaving, for a column with no hazard beside it. Ignored when hazard is given.
return_rate float 0.45 The fraction of leavers who come back later, and so are stopping out rather than dropping out.

academic.financial_aid#

How a student is paying, as five mutually exclusive states.

Priority P0 · dtype string · equivalence categorical

Example params: weights=None

Field Type Default Description
weights tuple[float, ...] | None None Relative frequency per label, in the order labels() gives them. The published defaults when omitted.

academic.persistence_hazard#

The chance a student does not enrol next term, as a model of the columns you name.

Priority P0 · dtype float64 · equivalence continuous

Example params: baseline=0.12 covariates=(Covariate(column=Reference(table='transcript', column='gpa'), coefficient=-0.9, centre=3.0, scale=1.0),)

Field Type Default Description
baseline float 0.12 Probability of not returning next term for a student sitting at every covariate's centre. Around 0.12 a term is an ordinary four-year institution.
covariates tuple[Covariate, ...] — The columns that move the hazard, and by how much.

Psychometrics#

academic.growth_curve#

A score measured repeatedly, where people start differently and gain at different rates.

Priority P0 · dtype float64 · equivalence continuous

Example params: occasions=4 people=512 intercept_mean=200.0 intercept_sd=30.0 slope_mean=12.0 slope_sd=4.0 correlation=-0.3 residual_sd=8.0

Field Type Default Description
occasions int 4 Measurements per person, in consecutive rows. Two is the fewest that can show growth.
people int 512 Distinct people the intercepts and slopes are drawn for, after which the pattern repeats.
intercept_mean float 200.0 Score at the first occasion, averaged over people.
intercept_sd float 30.0 How much people differ at the first occasion.
slope_mean float 12.0 Score points gained per occasion, averaged over people.
slope_sd float 4.0 How much people differ in how fast they gain.
correlation float -0.3 Between a person's starting point and their rate of gain. Negative is the usual finding: those who start behind gain faster.
residual_sd float 8.0 Occasion-to-occasion noise around a person's own straight line.

academic.item_parameter#

One item's difficulty, discrimination or guessing rate, repeated for every row on it.

Priority P0 · dtype float64 · equivalence continuous

Example params: parameter='difficulty' items=30 difficulty_mean=0.0 difficulty_sd=1.0 discrimination_median=1.0 discrimination_dispersion=0.35 options=4 guessing_concentration=20.0

Field Type Default Description
parameter Literal['difficulty', 'discrimination', 'guessing'] 'difficulty' Which item parameter this column holds. Declare one column per parameter.
items int 30 Items in the instrument. Rows cycle through them, so row i is item i mod items.
difficulty_mean float 0.0 Centre of the difficulty distribution, on the ability scale. Zero means the instrument is centred on the population it is given to.
difficulty_sd float 1.0 Spread of the difficulties, in ability units.
discrimination_median float 1.0 Middle discrimination. One is the Rasch value; below 0.5 an item barely sorts candidates and above 2.5 it behaves like a switch.
discrimination_dispersion float 0.35 Spread of the discriminations, as the sigma of a lognormal, so they stay positive.
options int 4 Response options, which sets the centre of the guessing asymptote at one over this.
guessing_concentration float 20.0 How tightly the guessing asymptotes cluster on that centre. Larger is tighter.

academic.item_response_1pl#

Right or wrong under the Rasch model, where every item discriminates alike.

Priority P0 · dtype int64 · equivalence categorical

Example params: ability=Reference(table='responses', column='ability') difficulty=Reference(table='responses', column='difficulty') discrimination=1.0

Field Type Default Description
ability Reference — A standardised ability column of the same table.
difficulty Reference — An academic.item_parameter column holding difficulties.
discrimination float 1.0 The one discrimination every item shares. This being a number rather than a column is what makes the model a 1PL.

academic.item_response_2pl#

Right or wrong, with items differing in difficulty and in how sharply they sort.

Priority P0 · dtype int64 · equivalence categorical

Example params: ability=Reference(table='responses', column='ability') difficulty=Reference(table='responses', column='difficulty') discrimination=Reference(table='responses', column='discrimination')

Field Type Default Description
ability Reference — A standardised ability column of the same table.
difficulty Reference — An academic.item_parameter column holding difficulties.
discrimination Reference — An academic.item_parameter column holding discriminations.

academic.item_response_3pl#

Right or wrong, with a floor under the curve for candidates who guess.

Priority P0 · dtype int64 · equivalence categorical

Example params: ability=Reference(table='responses', column='ability') difficulty=Reference(table='responses', column='difficulty') discrimination=Reference(table='responses', column='discrimination') guessing=Reference(table='responses', column='guessing')

Field Type Default Description
ability Reference — A standardised ability column of the same table.
difficulty Reference — An academic.item_parameter column holding difficulties.
discrimination Reference — An academic.item_parameter column holding discriminations.
guessing Reference — An academic.item_parameter column holding guessing asymptotes.

academic.scale_score#

A reported test score - the candidate's ability, measured imperfectly and rounded.

Priority P0 · dtype int64 · equivalence continuous

Example params: mean=500.0 sd=100.0 minimum=200 maximum=800 granularity=10 ability=Reference(table='responses', column='ability') reliability=0.92

Field Type Default Description
mean float 500.0 Score of a candidate at the ability mean.
sd float 100.0 Score points per standard deviation of ability.
minimum int 200 Lowest score the scale reports.
maximum int 800 Highest score the scale reports.
granularity int 10 Step between reportable scores. Ten is the SAT's.
ability Reference — A standardised ability column of the same table.
reliability float 0.92 Share of the reported score's variance that is true score. Published for every testing programme as an alpha or KR-20; 0.92 is a good long test.

academic.subscore#

A reported part score, both less related to the whole and measured less reliably.

Priority P0 · dtype int64 · equivalence continuous

Example params: mean=25.0 sd=5.0 minimum=10 maximum=40 granularity=1 ability=Reference(table='responses', column='ability') correlation=0.85 reliability=0.75

Field Type Default Description
mean float 500.0 Score of a candidate at the ability mean.
sd float 100.0 Score points per standard deviation of ability.
minimum int 200 Lowest score the scale reports.
maximum int 800 Highest score the scale reports.
granularity int 10 Step between reportable scores. Ten is the SAT's.
ability Reference — The same standardised ability column the total is built from.
correlation float 0.85 True-score correlation between this subdomain and the overall construct. One means the subdomain measures nothing of its own.
reliability float 0.75 This subscore's own reliability, which is lower than a total's because it rests on fewer items.

C.7 — Additional domains#

Healthcare-adjacent#

clinical.diagnosis_code#

A diagnosis code shaped like ICD-10 and meaning nothing.

Priority P1 · dtype string · equivalence structural

Example params: categories=900 concentration=1.0 letters='ABCDEFGHIJKLMNOPQRSTVWXYZ' subdivided_shares=(0.18, 0.52, 0.3)

Field Type Default Description
categories int 900 How many three-character categories the file uses. A hospital's casemix is a few hundred; a single clinic's is a few dozen.
concentration float 1.0 How steeply the categories fall off in frequency. One is Zipf's law; zero makes every category equally common, which no extract has ever been.
letters str 'ABCDEFGHIJKLMNOPQRSTVWXYZ' Leading letters to draw from. Narrow it to model a specialty.
subdivided_shares tuple[float, float, float] (0.18, 0.52, 0.3) Share of codes with no subdivision, with one character after the point, and with two. Relative weights; they need not sum to one.

clinical.encounter#

How an admission started, or how it ended.

Priority P1 · dtype string · equivalence categorical

Example params: field='admission_type' weights=None

Field Type Default Description
field Literal['admission_type', 'disposition'] 'admission_type' How the encounter began, or how it ended. Declare one column of each.
weights tuple[float, ...] | None None Relative frequency per label of the chosen field, in the order labels() gives them. The defaults when omitted.

clinical.lab_result#

One analyte from a panel, with the interval it is read against.

Priority P1 · dtype float64 · equivalence continuous

Example params: analyte='potassium' field='value' low=None high=None abnormal_share=0.24 high_share=None abnormal_spread=1.6

Field Type Default Description
analyte Literal['sodium', 'potassium', 'creatinine', 'glucose', 'haemoglobin', 'platelets', 'wbc', 'alt', 'tsh'] 'potassium' Which analyte.
field Literal['value', 'reference_low', 'reference_high'] 'value' The result, or one end of the interval it is read against. Declare one column of each to reproduce what a report prints.
low float | None None Interval floor, or the analyte's default.
high float | None None Interval ceiling, or the analyte's default.
abnormal_share float 0.24 Share of results outside the interval, exactly. A healthy population gives 0.05 by the definition of the interval; a file of people who were tested for a reason gives far more, and which one is being built the type cannot guess.
high_share float | None None Share of the abnormal results that are above the interval rather than below, or the analyte's default.
abnormal_spread float 1.6 How far outside the interval an abnormal result goes, in multiples of the healthy log-scale spread.

clinical.length_of_stay#

How many days the admission lasted.

Priority P1 · dtype float64 · equivalence continuous

Example params: median=3.1 spread=0.86

Field Type Default Description
median float 3.1 Median stay in days.
spread float 0.86 Log-scale spread. The tail is the point: a small number of stays run to weeks and they are most of the bed-days.

clinical.medication#

A therapeutic class, the strength it was dispensed in, or how often it is taken.

Priority P1 · dtype string · equivalence categorical

Example params: field='class' weights=None frequency_weights=None

Field Type Default Description
field Literal['class', 'strength', 'frequency'] 'class' The class, its strength in milligrams, or the dosing frequency. Declare one column of each for a full prescription line.
weights tuple[float, ...] | None None Relative frequency per class, in the order labels() gives them. The defaults when omitted.
frequency_weights tuple[float, ...] | None None Relative frequency per dosing frequency.

clinical.procedure_code#

A procedure code shaped like HCPCS Level II or ICD-10-PCS.

Priority P1 · dtype string · equivalence structural

Example params: scheme='hcpcs' letters='ABCDEGHJKLMPQRSTV' procedures=400 concentration=1.0

Field Type Default Description
scheme Literal['hcpcs', 'icd10pcs'] 'hcpcs' HCPCS Level II — a letter and four digits — or the seven-character ICD-10-PCS. CPT is not offered; see the docstring.
letters str 'ABCDEGHJKLMPQRSTV' HCPCS leading letters to draw from.
procedures int 400 How many distinct procedures the file uses.
concentration float 1.0 How steeply the procedures fall off in frequency, as for a diagnosis.

clinical.provider_id#

The clinician or organisation on the encounter.

Priority P1 · dtype string · equivalence structural

Example params: providers=240 concentration=0.7

Field Type Default Description
providers int 240 How many distinct providers the file has. A department is dozens; a health system is thousands.
concentration float 0.7 How unevenly the encounters fall across providers. Zero gives every provider the same caseload, which no rota has ever produced.

clinical.vital_sign#

A recorded vital sign - heart rate, blood pressure, temperature, saturation.

Priority P1 · dtype float64 · equivalence continuous

Example params: measure='heart_rate' systolic=None centre=None spread=None terminal_digit_share=0.35

Field Type Default Description
measure Literal['heart_rate', 'systolic', 'diastolic', 'respiratory_rate', 'temperature_c', 'spo2'] 'heart_rate' Which vital sign.
systolic Reference | None None A clinical.vital_sign systolic column. Required by the diastolic measure, which subtracts a pulse pressure from it.
centre float | None None Middle of the distribution, or the measure's default. For diastolic this is the pulse pressure, not the pressure.
spread float | None None Standard deviation, or the measure's default.
terminal_digit_share float 0.35 Share of readings rounded to the nearest ten rather than to one. Digit preference is what a manual cuff and a wall clock do to a chart; how much of it there is depends on who was measuring, so this is a visible default and not a finding. Zero gives a clean file.

HR and payroll#

hr.benefits_election#

Which benefits package somebody elected.

Priority P0 · dtype string · equivalence categorical

Example params: plans=('None', 'Core', 'Core plus dental', 'Family', 'Family plus dental') shares=(0.07, 0.31, 0.18, 0.29, 0.15)

Field Type Default Description
plans tuple[str, ...] ('None', 'Core', 'Core plus dental', 'Family', 'Family plus dental') Plan names.
shares tuple[float, ...] (0.07, 0.31, 0.18, 0.29, 0.15) Take-up of each. Normalised.

hr.employee_id#

An employee number, issued in hire order.

Priority P0 · dtype string · equivalence exact

Example params: prefix='E' width=7 first=100001

Field Type Default Description
prefix str 'E' Written before the number.
width int 7 Digits in the number.
first int 100001 The number the first hire got.

hr.job_level#

The job level or band somebody sits at.

Priority P0 · dtype string · equivalence categorical

Example params: levels=('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') shares=(0.09, 0.19, 0.24, 0.17, 0.09, 0.03, 0.11, 0.05, 0.02, 0.01)

Field Type Default Description
levels tuple[str, ...] ('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') Band names.
shares tuple[float, ...] (0.09, 0.19, 0.24, 0.17, 0.09, 0.03, 0.11, 0.05, 0.02, 0.01) Share of headcount at each. Normalised.

hr.org_unit#

The cost centre somebody is charged to.

Priority P0 · dtype string · equivalence categorical

Example params: divisions=('ENG', 'SALES', 'GA', 'MKT', 'OPS') departments=(('PLATFORM', 'PRODUCT', 'INFRA', 'QUALITY'), ('ENTERPRISE', 'MIDMARKET', 'PARTNERS'), ('FINANCE', 'LEGAL', 'PEOPLE', 'FACILITIES'), ('BRAND', 'DEMAND', 'CONTENT'), ('SUPPORT', 'LOGISTICS')) shares=(0.42, 0.19, 0.14, 0.12, 0.13) separator='-'

Field Type Default Description
divisions tuple[str, ...] ('ENG', 'SALES', 'GA', 'MKT', 'OPS') Division codes.
departments tuple[tuple[str, ...], ...] (('PLATFORM', 'PRODUCT', 'INFRA', 'QUALITY'), ('ENTERPRISE', 'MIDMARKET', 'PARTNERS'), ('FINANCE', 'LEGAL', 'PEOPLE', 'FACILITIES'), ('BRAND', 'DEMAND', 'CONTENT'), ('SUPPORT', 'LOGISTICS')) Departments inside each division, in the same order.
shares tuple[float, ...] (0.42, 0.19, 0.14, 0.12, 0.13) Share of headcount in each division. Engineering is the big one.
separator str '-' Joins the two.

hr.payroll_deduction#

A monthly payroll deduction, or one part of it.

Priority P0 · dtype float64 · equivalence continuous

Example params: field='total' salary=Reference(table='staff', column='salary') plan=Reference(table='staff', column='benefits') plans=('None', 'Core', 'Core plus dental', 'Family', 'Family plus dental') costs=(0.0, 92.0, 138.0, 296.0, 361.0) tax_rate=0.24 pension_rate=0.05

Field Type Default Description
field Literal['total', 'tax', 'pension', 'benefits'] 'total' The whole deduction, or one of the three parts of it.
salary Reference — An hr.salary column, annual.
plan Reference | None None An hr.benefits_election column.
plans tuple[str, ...] ('None', 'Core', 'Core plus dental', 'Family', 'Family plus dental') Plan names.
costs tuple[float, ...] (0.0, 92.0, 138.0, 296.0, 361.0) Monthly employee contribution for each plan.
tax_rate float 0.24 Effective tax and social-insurance rate.
pension_rate float 0.05 Employee pension contribution rate.

hr.performance_rating#

The performance rating somebody was given.

Priority P0 · dtype string · equivalence categorical

Example params: ratings=('Below expectations', 'Meets some', 'Meets', 'Exceeds', 'Outstanding') shares=(0.03, 0.11, 0.58, 0.23, 0.05)

Field Type Default Description
ratings tuple[str, ...] ('Below expectations', 'Meets some', 'Meets', 'Exceeds', 'Outstanding') Rating names, worst first.
shares tuple[float, ...] (0.03, 0.11, 0.58, 0.23, 0.05) Share at each. Normalised.

hr.salary#

Annual salary, lognormal within a band.

Priority P0 · dtype float64 · equivalence continuous

Example params: level=Reference(table='staff', column='level') levels=('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') medians=(42000.0, 55000.0, 71000.0, 92000.0, 118000.0, 149000.0, 98000.0, 126000.0, 165000.0, 214000.0) spread=0.19 rounding=100 default_median=60000.0

Field Type Default Description
level Reference | None None An hr.job_level column. Without it every salary comes from one distribution, which is not a pay structure.
levels tuple[str, ...] ('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') Band names, in the level column's order.
medians tuple[float, ...] (42000.0, 55000.0, 71000.0, 92000.0, 118000.0, 149000.0, 98000.0, 126000.0, 165000.0, 214000.0) Median salary in each band.
spread float 0.19 Log-scale spread within a band. This is what makes the bands overlap, which is the property a pay-equity review exists to look at.
rounding int 100 Salaries are set to round numbers.
default_median float 60000.0 Median for a band not on the list.

hr.tenure#

Years of service.

Priority P0 · dtype float64 · equivalence continuous

Example params: level=Reference(table='staff', column='level') levels=('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') years_per_level=1.9 median_years=2.6 external_share=0.28 spread=0.7

Field Type Default Description
level Reference | None None An hr.job_level column. Promotion takes time, so tenure climbs with the level — except for the external hires, who are the interesting case.
levels tuple[str, ...] ('IC1', 'IC2', 'IC3', 'IC4', 'IC5', 'IC6', 'M1', 'M2', 'M3', 'M4') Band names, in the level column's order.
years_per_level float 1.9 Typical years between promotions.
median_years float 2.6 Median tenure of somebody at the first band.
external_share float 0.28 Share hired straight into their band from outside, whose tenure owes nothing to it. A ladder without them cannot test a time-to-promotion model.
spread float 0.7 Log-scale spread of tenure within a band.

hr.time_off_balance#

Days of leave left in the balance.

Priority P0 · dtype float64 · equivalence continuous

Example params: tenure=Reference(table='staff', column='tenure') base_days=25.0 days_per_year=0.5 maximum_days=32.0 taken_share=0.68 spread=0.34

Field Type Default Description
tenure Reference | None None An hr.tenure column, in years. Entitlement grows with service, which is what a leave policy says and what a liability calculation needs.
base_days float 25.0 Days a first-year employee is entitled to.
days_per_year float 0.5 Extra days earned per year of service.
maximum_days float 32.0 Where the entitlement stops growing.
taken_share float 0.68 Share of the entitlement typically used.
spread float 0.34 How much people differ in what they take.

Marketing and analytics#

marketing.ab_arm#

Which variant this session saw.

Priority P1 · dtype string · equivalence categorical

Example params: arms=('control', 'treatment') shares=(0.5, 0.5)

Field Type Default Description
arms tuple[str, ...] ('control', 'treatment') Arm names. The first is the control, by the convention marketing.conversion_event reads.
shares tuple[float, ...] (0.5, 0.5) Share of sessions assigned to each arm.

marketing.ad_performance#

Ad spend, impressions or click-through rate for one campaign in one period.

Priority P1 · dtype float64 · equivalence continuous

Example params: field='impressions' spend=None median_spend=850.0 spend_spread=0.85 median_cpm=12.5 cpm_spread=0.35 median_impressions=68000.0 ctr_mean=0.021 ctr_concentration=60.0

Field Type Default Description
field Literal['spend', 'impressions', 'ctr'] 'spend' Which metric this column carries.
spend Reference | None None A marketing.ad_performance spend column, read by the impressions field. Ties impressions to what the spend actually bought; left unset, impressions draws its own count instead.
median_spend float 850.0 Median spend per campaign for the period.
spend_spread float 0.85 Log-scale spread of spend across the book.
median_cpm float 12.5 Median cost per thousand impressions.
cpm_spread float 0.35 Log-scale spread of the CPM across campaigns.
median_impressions float 68000.0 Median impressions, when spend is not set.
ctr_mean float 0.021 Mean click-through rate.
ctr_concentration float 60.0 Beta concentration. Higher is tighter.

marketing.attribution_channel#

How this session found the site.

Priority P1 · dtype string · equivalence categorical

Example params: channels=('organic_search', 'paid_search', 'paid_social', 'email', 'direct', 'referral', 'affiliate') shares=(0.28, 0.2, 0.14, 0.09, 0.18, 0.08, 0.03)

Field Type Default Description
channels tuple[str, ...] ('organic_search', 'paid_search', 'paid_social', 'email', 'direct', 'referral', 'affiliate') Channel names.
shares tuple[float, ...] (0.28, 0.2, 0.14, 0.09, 0.18, 0.08, 0.03) Share of sessions from each. Normalised.

marketing.campaign#

Which of the live campaigns this session's traffic belongs to.

Priority P1 · dtype string · equivalence categorical

Example params: platforms=('SEARCH', 'SOCIAL', 'DISPLAY', 'EMAIL', 'AFFIL') campaigns=40 width=5 concentration=1.1

Field Type Default Description
platforms tuple[str, ...] ('SEARCH', 'SOCIAL', 'DISPLAY', 'EMAIL', 'AFFIL') Ad-platform codes.
campaigns int 40 How many campaigns are concurrently live.
width int 5 Digits in the campaign number.
concentration float 1.1 How steeply traffic favours the busiest campaigns. One is Zipf's law; higher concentrates further, lower flattens towards even shares.

marketing.conversion_event#

Whether this session converted.

Priority P1 · dtype bool · equivalence categorical

Example params: arm=Reference(table='session', column='arm') control_label='control' baseline=0.04 lift=0.012

Field Type Default Description
arm Reference | None None A marketing.ab_arm column. Every arm but the control gets the lift.
control_label str 'control' Which arm value is the control. Read only when arm is set.
baseline float 0.04 Conversion rate at the control arm, or for every session when arm is not set.
lift float 0.012 Added to the baseline for every arm that is not the control. Negative is a real answer an experiment can give.

marketing.funnel_stage#

The furthest stage this session reached before it left.

Priority P1 · dtype string · equivalence categorical

Example params: stages=('view', 'product', 'cart', 'checkout', 'purchase') continue_rates=(0.55, 0.32, 0.58, 0.72)

Field Type Default Description
stages tuple[str, ...] ('view', 'product', 'cart', 'checkout', 'purchase') Steps in order, from first to last.
continue_rates tuple[float, ...] (0.55, 0.32, 0.58, 0.72) Probability of continuing to the next stage, given the current one was reached. One fewer than the stage count.

marketing.utm_session#

The UTM query-string parameters this session arrived with, if any.

Priority P1 · dtype string · equivalence structural

Example params: channel=Reference(table='session', column='channel') campaign=Reference(table='session', column='campaign')

Field Type Default Description
channel Reference — A marketing.attribution_channel column.
campaign Reference — A marketing.campaign column.

Real estate#

real_estate.characteristics#

The size, bedroom count or bathroom count of the listing.

Priority P1 · dtype float64 · equivalence continuous

Example params: field='bedrooms' size=Reference(table='listing', column='size') median_size=1850.0 size_spread=0.42 sqft_per_bedroom=550.0 bedroom_noise=0.6 max_bedrooms=6 sqft_per_bathroom=700.0 bathroom_noise=0.5 max_bathrooms=5.0

Field Type Default Description
field Literal['size', 'bedrooms', 'bathrooms'] 'size' Which characteristic this column carries.
size Reference | None None A real_estate.characteristics size column. Required by the bedrooms and bathrooms fields — there is no bedroom count independent of the square footage it sits inside.
median_size float 1850.0 Median size in square feet.
size_spread float 0.42 Log-scale spread of size across the book.
sqft_per_bedroom float 550.0 Square feet a bedroom count is scaled against.
bedroom_noise float 0.6 Spread, in bedrooms, around what the size alone implies.
max_bedrooms int 6 The most bedrooms a listing may have.
sqft_per_bathroom float 700.0 Square feet a bathroom count is scaled against.
bathroom_noise float 0.5 Spread, in bathrooms, around what the size alone implies.
max_bathrooms float 5.0 The most bathrooms a listing may have.

real_estate.days_on_market#

How long the listing sat before it sold or was withdrawn.

Priority P1 · dtype float64 · equivalence continuous

Example params: median_days=28.0 spread=0.65

Field Type Default Description
median_days float 28.0 Median days listed.
spread float 0.65 Log-scale spread. The tail is the point: most listings sell inside a couple of months and a few sit for a year.

real_estate.listing#

A listing number, issued in order.

Priority P1 · dtype string · equivalence exact

Example params: prefix='MLS-' width=7 first=100000

Field Type Default Description
prefix str 'MLS-' Text before the digits.
width int 7 Digits in the number.
first int 100000 The first number issued.

real_estate.price#

The listing price, a hedonic function of its features plus noise.

Priority P1 · dtype float64 · equivalence continuous

Example params: size=Reference(table='listing', column='size') bedrooms=Reference(table='listing', column='bedrooms') bathrooms=Reference(table='listing', column='bathrooms') property_type=Reference(table='listing', column='property_type') price_per_sqft=210.0 reference_bedrooms=3.0 bedroom_premium=0.03 reference_bathrooms=2.0 bathroom_premium=0.04 type_categories=('single_family', 'condo', 'townhouse', 'multi_family') type_multipliers=(1.0, 0.9, 0.94, 0.82) spread=0.28

Field Type Default Description
size Reference — A real_estate.characteristics size column.
bedrooms Reference | None None A real_estate.characteristics bedrooms column.
bathrooms Reference | None None A real_estate.characteristics bathrooms column.
property_type Reference | None None A real_estate.property_type column.
price_per_sqft float 210.0 Price per square foot at every reference.
reference_bedrooms float 3.0 The bedroom count price_per_sqft is quoted at.
bedroom_premium float 0.03 Share added to the price for every bedroom above the reference count.
reference_bathrooms float 2.0 The bathroom count price_per_sqft is quoted at.
bathroom_premium float 0.04 Share added to the price for every bathroom above the reference count.
type_categories tuple[str, ...] ('single_family', 'condo', 'townhouse', 'multi_family') Property type names.
type_multipliers tuple[float, ...] (1.0, 0.9, 0.94, 0.82) Price-per-square-foot multiplier for each of type_categories, in order.
spread float 0.28 Log-scale spread from every rating factor this column does not model — location, condition, the rest of an appraisal.

real_estate.property_type#

What kind of home the listing is.

Priority P1 · dtype string · equivalence categorical

Example params: types=('single_family', 'condo', 'townhouse', 'multi_family') shares=(0.58, 0.2, 0.12, 0.1)

Field Type Default Description
types tuple[str, ...] ('single_family', 'condo', 'townhouse', 'multi_family') Property type names.
shares tuple[float, ...] (0.58, 0.2, 0.12, 0.1) Share of listings of each type. Normalised.

Insurance#

insurance.claim_frequency#

How many claims a policy made in the period.

Priority P1 · dtype int64 · equivalence categorical

Example params: counting='negative-binomial' mean=0.19 dispersion=0.6

Field Type Default Description
counting Literal['poisson', 'negative-binomial'] 'negative-binomial' Poisson, or the negative binomial that real claim counts need.
mean float 0.19 Claims per policy per period.
dispersion float 0.6 Negative-binomial shape. Low means a few policyholders claim far more than the rest; high approaches the Poisson from above.

insurance.claim_severity#

What a claim cost, or what a policy's claims cost together.

Priority P1 · dtype float64 · equivalence continuous

Example params: field='incurred' family='lognormal' frequency=Reference(table='policy', column='claims') deductible=Reference(table='policy', column='deductible') limit=Reference(table='policy', column='limit') median=1450.0 max_claims=12 tail=2.1

Field Type Default Description
field Literal['per_claim', 'incurred'] 'incurred' One claim's size, for a claim-level table; or a policy's total, for a policy-level one. The subgroup's rows are policies, so the total is the default.
family Literal['gamma', 'lognormal', 'pareto'] 'lognormal' Gamma, lognormal or Pareto.
frequency Reference | None None An insurance.claim_frequency column. Required by the incurred field, which sums that many claims.
deductible Reference | None None An insurance.coverage deductible column.
limit Reference | None None An insurance.coverage limit column.
median float 1450.0 Median cost of one claim.
max_claims int 12 Most claims one policy may have. It fixes how many words a row spends, which is what keeps the column row-addressed — derived from the data instead, a chunk holding a busier policy would shift the whole stream.
tail float 2.1 Pareto shape, or the log-scale spread for the other two families. Below two a Pareto has no finite variance, which is the case reinsurance exists for.

insurance.coverage#

The deductible or the limit on a policy.

Priority P1 · dtype float64 · equivalence categorical

Example params: field='deductible' deductibles=(100.0, 250.0, 500.0, 1000.0, 2500.0) deductible_shares=(0.21, 0.34, 0.26, 0.14, 0.05) limit_multiples=(20.0, 50.0, 100.0, 250.0) limit_shares=(0.18, 0.37, 0.31, 0.14) base=1000.0

Field Type Default Description
field Literal['deductible', 'limit'] 'deductible' The excess the holder bears, or the cap on a claim.
deductibles tuple[float, ...] (100.0, 250.0, 500.0, 1000.0, 2500.0) The excesses on offer.
deductible_shares tuple[float, ...] (0.21, 0.34, 0.26, 0.14, 0.05) Take-up of each.
limit_multiples tuple[float, ...] (20.0, 50.0, 100.0, 250.0) Cap on a claim, as a multiple of the base deductible.
limit_shares tuple[float, ...] (0.18, 0.37, 0.31, 0.14) Take-up of each.
base float 1000.0 What the limit multiples are multiples of.

insurance.loss_ratio#

Incurred loss over premium.

Priority P1 · dtype float64 · equivalence continuous

Example params: incurred=Reference(table='policy', column='incurred') premium=Reference(table='policy', column='premium')

Field Type Default Description
incurred Reference — An insurance.claim_severity incurred column.
premium Reference — An insurance.premium column.

insurance.policy#

A policy number, one row per policy.

Priority P1 · dtype string · equivalence structural

Example params: lines=('MOT', 'HOM', 'TRV', 'PET', 'LIA', 'PRO') shares=(0.38, 0.24, 0.13, 0.09, 0.08, 0.08) width=8 first=1

Field Type Default Description
lines tuple[str, ...] ('MOT', 'HOM', 'TRV', 'PET', 'LIA', 'PRO') Line-of-business codes.
shares tuple[float, ...] (0.38, 0.24, 0.13, 0.09, 0.08, 0.08) Share of the book in each line. Motor is the big one.
width int 8 Digits in the number.
first int 1 The first policy number issued.

insurance.premium#

What the policy was charged for the period.

Priority P1 · dtype float64 · equivalence continuous

Example params: deductible=Reference(table='policy', column='deductible') base=420.0 reference_deductible=250.0 discount_per_doubling=0.11 spread=0.31 minimum=60.0

Field Type Default Description
deductible Reference | None None An insurance.coverage deductible column. A higher excess buys a discount, which is the whole reason a holder is offered the choice.
base float 420.0 Premium at the reference deductible.
reference_deductible float 250.0 The excess the base premium is quoted at.
discount_per_doubling float 0.11 Share taken off the premium each time the excess doubles.
spread float 0.31 Log-scale spread across the book, from every rating factor this column does not model. Real books are far more spread than one factor explains.
minimum float 60.0 The smallest premium written.

Supply chain#

supply_chain.carrier#

The carrier, or the tracking number it issued.

Priority P1 · dtype string · equivalence structural

Example params: field='tracking_number' carrier=Reference(table='shipment', column='carrier') carriers=('UPS', 'FedEx', 'USPS', 'DHL') shares=(0.34, 0.3, 0.26, 0.1)

Field Type Default Description
field Literal['carrier', 'tracking_number'] 'carrier' Which of the two this column carries.
carrier Reference | None None A supply_chain.carrier carrier column. Required by the tracking_number field, which renders digits at that carrier's own length.
carriers tuple[str, ...] ('UPS', 'FedEx', 'USPS', 'DHL') Carrier names.
shares tuple[float, ...] (0.34, 0.3, 0.26, 0.1) Share of shipments on each carrier.

supply_chain.inventory_level#

How much of the item is on hand.

Priority P1 · dtype float64 · equivalence continuous

Example params: reorder_point=Reference(table='item', column='reorder_point') buffer_median=180.0 buffer_spread=0.9

Field Type Default Description
reorder_point Reference — A supply_chain.reorder_point column.
buffer_median float 180.0 Median stock held above the reorder point.
buffer_spread float 0.9 Log-scale spread of the buffer — wide, because a healthy item sitting just after a delivery and one about to trigger a reorder are both realistic rows.

supply_chain.lead_time#

How many days between the order and the delivery.

Priority P1 · dtype float64 · equivalence continuous

Example params: warehouse=Reference(table='shipment', column='warehouse') median_days=4.5 spread=0.5 warehouses=('DC-EAST', 'DC-CENTRAL', 'DC-WEST', 'DC-SOUTH') warehouse_multipliers=(0.85, 1.0, 1.15, 1.05)

Field Type Default Description
warehouse Reference | None None A supply_chain.warehouse column.
median_days float 4.5 Median lead time.
spread float 0.5 Log-scale spread across shipments.
warehouses tuple[str, ...] ('DC-EAST', 'DC-CENTRAL', 'DC-WEST', 'DC-SOUTH') Warehouse codes.
warehouse_multipliers tuple[float, ...] (0.85, 1.0, 1.15, 1.05) Median multiplier for each of warehouses, in order. Read only when warehouse is set.

supply_chain.reorder_point#

The stock level at which an item is reordered.

Priority P1 · dtype float64 · equivalence continuous

Example params: median=120.0 spread=0.4

Field Type Default Description
median float 120.0 Median reorder point.
spread float 0.4 Log-scale spread across items.

supply_chain.shipment#

A shipment number, issued in order.

Priority P1 · dtype string · equivalence exact

Example params: prefix='SHP-' width=8 first=1

Field Type Default Description
prefix str 'SHP-' Text before the digits.
width int 8 Digits in the number.
first int 1 The first number issued.

supply_chain.supplier_scorecard#

An on-time rate, a defect rate or a fill rate for one supplier.

Priority P1 · dtype float64 · equivalence continuous

Example params: field='on_time_rate' on_time_mean=0.93 on_time_concentration=80.0 defect_mean=0.02 defect_concentration=150.0 fill_mean=0.96 fill_concentration=100.0

Field Type Default Description
field Literal['on_time_rate', 'defect_rate', 'fill_rate'] 'on_time_rate' Which metric this column carries.
on_time_mean float 0.93 Mean on-time rate.
on_time_concentration float 80.0 Beta concentration for on_time_rate.
defect_mean float 0.02 Mean defect rate.
defect_concentration float 150.0 Beta concentration for defect_rate.
fill_mean float 0.96 Mean order fill rate.
fill_concentration float 100.0 Beta concentration for fill_rate.

supply_chain.warehouse#

Which distribution centre fulfilled the shipment.

Priority P1 · dtype string · equivalence categorical

Example params: warehouses=('DC-EAST', 'DC-CENTRAL', 'DC-WEST', 'DC-SOUTH') shares=(0.32, 0.28, 0.24, 0.16)

Field Type Default Description
warehouses tuple[str, ...] ('DC-EAST', 'DC-CENTRAL', 'DC-WEST', 'DC-SOUTH') Warehouse codes.
shares tuple[float, ...] (0.32, 0.28, 0.24, 0.16) Share of shipments fulfilled by each. Normalised.

Text corpora#

text_corpora.chat_transcript_turn#

One turn of a support chat.

Priority P1 · dtype string · equivalence structural

Example params: sentiment=Reference(table='chat', column='sentiment')

Field Type Default Description
sentiment Reference | None None A text_corpora.sentiment_label column. Read only for the customer's own turns; an agent's turns are always drawn from a fixed, neutral pool.

text_corpora.email_message#

A one-line business email.

Priority P1 · dtype string · equivalence structural

Example params: ``

text_corpora.job_posting#

A one-line job posting.

Priority P1 · dtype string · equivalence structural

Example params: ``

text_corpora.news_article#

A headline and a one-sentence lede.

Priority P1 · dtype string · equivalence structural

Example params: ``

text_corpora.product_description#

A one-line piece of marketing copy.

Priority P1 · dtype string · equivalence structural

Example params: ``

text_corpora.sentiment_label#

The register a document was written in.

Priority P1 · dtype string · equivalence categorical

Example params: sentiments=('negative', 'neutral', 'positive') shares=(0.25, 0.45, 0.3)

Field Type Default Description
sentiments tuple[str, ...] ('negative', 'neutral', 'positive') Label names.
shares tuple[float, ...] (0.25, 0.45, 0.3) Share of documents at each. Normalised.

text_corpora.social_post#

A one-line social media post.

Priority P1 · dtype string · equivalence structural

Example params: sentiment=Reference(table='post', column='sentiment')

Field Type Default Description
sentiment Reference | None None A text_corpora.sentiment_label column. Left unset, every post is drawn in a neutral register.

text_corpora.support_ticket#

A one-line customer support ticket.

Priority P1 · dtype string · equivalence structural

Example params: sentiment=Reference(table='ticket', column='sentiment')

Field Type Default Description
sentiment Reference | None None A text_corpora.sentiment_label column. Left unset, every ticket is drawn in a neutral register.

Scientific#

scientific.allele_frequency#

The population frequency of the alternate allele at one locus.

Priority P2 · dtype float64 · equivalence continuous

Example params: mean=0.15 concentration=3.0

Field Type Default Description
mean float 0.15 Mean frequency across loci. Real variants are mostly rare, so the default sits well under one half.
concentration float 3.0 Beta concentration. Higher is tighter.

scientific.assay_plate_reading#

One well's reading off an assay plate.

Priority P2 · dtype float64 · equivalence continuous

Example params: median=0.42 spread=0.35 detection_limit=0.01

Field Type Default Description
median float 0.42 Median reading.
spread float 0.35 Log-scale spread across wells.
detection_limit float 0.01 The instrument's floor. A reading cannot come back below what it can actually detect.

scientific.chemical_formula#

A Hill-system-shaped chemical formula - carbon, hydrogen, then one heteroatom.

Priority P2 · dtype string · equivalence structural

Example params: carbon_max=20 hydrogen_max=20 heteroatom_share=0.7 heteroatom_max=9

Field Type Default Description
carbon_max int 20 Most carbon atoms.
hydrogen_max int 20 Most hydrogen atoms.
heteroatom_share float 0.7 Share of formulas that carry a heteroatom at all. The rest are plain hydrocarbons.
heteroatom_max int 9 Most heteroatom atoms.

scientific.measurement#

A value, or the uncertainty stated alongside it.

Priority P2 · dtype float64 · equivalence continuous

Example params: field='uncertainty' value=Reference(table='sample', column='value') median_value=25.0 value_spread=0.3 relative_uncertainty=0.05 uncertainty_spread=0.3

Field Type Default Description
field Literal['value', 'uncertainty'] 'value' The measurement, or its stated uncertainty.
value Reference | None None A scientific.measurement value column. Required by the uncertainty field, which is stated as a share of it.
median_value float 25.0 Median of the value field.
value_spread float 0.3 Log-scale spread of the value across rows.
relative_uncertainty float 0.05 Typical uncertainty as a share of the value — an instrument's stated precision.
uncertainty_spread float 0.3 Log-scale spread of the relative uncertainty itself, across rows.

scientific.snp_matrix_column#

One locus's genotype for every sample - 0, 1 or 2 copies of the alternate allele.

Priority P2 · dtype int64 · equivalence categorical

Example params: allele_frequency=0.3

Field Type Default Description
allele_frequency float 0.3 This locus's own alternate-allele frequency. A parameter, not a reference — one registered column is one locus.

scientific.spectrum_curve#

A peak-normalised spectrum, one intensity per bin, as a comma-joined string.

Priority P2 · dtype string · equivalence structural

Example params: bins=12 width_bins=1.5 noise_amplitude=0.03

Field Type Default Description
bins int 12 Points in the discretised curve.
width_bins float 1.5 Peak width, in bins (a Gaussian sigma).
noise_amplitude float 0.03 Baseline noise added to every bin.

scientific.taxon#

An invented genus with a genuine, common Latin species epithet.

Priority P2 · dtype string · equivalence structural

Example params: ``

C.8 — Signal, audio, and image#

Signals#

signal.waveform#

An audio clip per row, written as a WAV file the column points at.

Priority P0 · dtype string · equivalence exact

Example params: components=(Component(shape='sine', frequency=440.0, end_frequency=None, amplitude=0.6, phase=0.0, colour=None), Component(shape='sine', frequency=660.0, end_frequency=None, amplitude=0.3, phase=0.0, colour=None)) sample_rate=8000 duration=1.0 snr_db=20.0 random_phase=True peak=0.95

Field Type Default Description
components tuple[Component, ...] — The terms of the mixture.
sample_rate int 8000 Samples per second.
duration float 1.0 Seconds of audio per row.
snr_db float | None None Signal-to-noise ratio in decibels. Omit for a noiseless clip.
random_phase bool True Give each row an independent phase per component, so the rows are examples of one signal rather than copies of it.
peak float 0.95 What the loudest sample of every row is scaled to, where 1.0 is full scale. Below 1.0 by default, so a resampler's overshoot has somewhere to go.

Speech#

speech.speaker_variant#

The same text narrated again, in a randomly drawn voice and speed.

Priority P0 · dtype string · equivalence exact

Example params: source=Reference(table='table1', column='text') voices=('af_heart', 'am_adam') speed_jitter=0.15

Field Type Default Description
source Reference — The text column this row narrates.
voices tuple[Literal['af_heart', 'am_adam'], ...] ('af_heart', 'am_adam') The pool a voice is drawn from, uniformly, per row.
speed_jitter float 0.15 Speed varies per row within ±this fraction of 1.0 — a stand-in for prosody variation between speakers, since the model itself takes no other knob for it.

speech.utterance#

A real spoken narration of another text column, one WAV file per row.

Priority P0 · dtype string · equivalence exact

Example params: source=Reference(table='table1', column='text') voice='af_heart' speed=1.0

Field Type Default Description
source Reference — The text column this row narrates.
voice Literal['af_heart', 'am_adam'] 'af_heart' Which installed Kokoro voice reads every row.
speed float 1.0 Playback speed multiplier.

Music#

music.symbolic_composition#

A short melody per row, written as a MIDI file the column points at.

Priority P1 · dtype string · equivalence exact

Example params: length=16 scale='major' root=60 octaves=2 step_range=2 note_ticks=240 ticks_per_quarter=480 tempo_bpm=120.0 velocity=80 program=0

Field Type Default Description
length int 16 Notes in the melody.
scale str 'major' One of ('major', 'natural_minor', 'major_pentatonic', 'minor_pentatonic', 'chromatic').
root int 60 MIDI note number of scale degree 0.
octaves int 2 Octaves above the root the walk may reach.
step_range int 2 Largest step the walk may take, in scale degrees, per note.
note_ticks int 240 Duration of each note, in ticks.
ticks_per_quarter int 480 MIDI division: ticks in one quarter note.
tempo_bpm float 120.0 Tempo.
velocity int 80 Note velocity, fixed for all.
program int 0 MIDI instrument program number.

Images#

image.barcode#

An EAN-13 barcode per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: height=48 module_width=2 quiet_zone=10

Field Type Default Description
height int 48 Image height in pixels.
module_width int 2 Pixels per narrowest bar or space.
quiet_zone int 10 Blank modules of margin on each side.

image.chart#

A bar chart per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: width=96 height=64 bars=6 gridlines=3 margin=6 bar_width_share=0.7

Field Type Default Description
width int 96 Image width in pixels.
height int 64 Image height in pixels.
bars int 6 Bars per chart.
gridlines int 3 Evenly spaced horizontal gridlines. 0 draws none.
margin int 6 Blank pixels of margin on every side of the plot.
bar_width_share float 0.7 Share of each bar's own slot the bar itself fills; the rest is the gap to its neighbours.

image.geometric_shape#

A filled circle, square or triangle per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: width=64 height=64 classes=('circle', 'square', 'triangle') shares=(0.34, 0.33, 0.33) radius=(7.0, 11.0, 23.0) radius_jitter=0.1

Field Type Default Description
width int 64 Image width in pixels.
height int 64 Image height in pixels.
classes tuple[Literal['circle', 'square', 'triangle'], ...] ('circle', 'square', 'triangle') Shape classes.
shares tuple[float, ...] (0.34, 0.33, 0.33) Share of images of each class.
radius tuple[float, ...] (7.0, 11.0, 23.0) Median radius per class, in pixels. The default three are chosen so the filled area comes out well separated, not just the radius — a circle, a square and a triangle cover very different areas at the same radius (roughly pi r^2, 4 r^2 and 1.6 r^2), so equal radii would not give equal-looking sizes and unequal-but-close radii would not reliably order the areas either.
radius_jitter float 0.1 Radius spread within a class, as a share of that class's own radius.

image.identicon#

A small symmetric avatar per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: grid=5 scale=32 source=None

Field Type Default Description
grid int 5 Cells per side. Must be odd, for a centre column.
scale int 32 Pixels per cell.
source Reference | None None A string column to hash. Left unset, the pattern is drawn like any other media type's own randomness; set, the same value always draws the same identicon.

image.noise_field#

A noise or texture image per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: width=64 height=64 colour='grey' mean=128.0 sd=40.0 exponent=1.0

Field Type Default Description
width int 64 Image width in pixels.
height int 64 Image height in pixels.
colour Literal['grey', 'rgb'] 'grey' One channel or three.
mean float 128.0 Average pixel level, on the 0-255 scale the file stores.
sd float 32.0 Standard deviation of the pixel levels, in the same units as mean.
exponent float 0.0 Power-spectrum slope: 0 white, 1 pink, 2 Brownian. Higher is smoother.

image.qr_code#

A QR symbol per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: scale=6 payload_length=10 mask=0

Field Type Default Description
scale int 6 Pixels per module.
payload_length int 10 Characters in the encoded ID.
mask int 0 Fixed mask pattern (0-7) — not the penalty-optimal one a full encoder would choose; see sensym_df.qr's module docstring.

Document image#

document.field_boxes#

The exact bounding box of every field of a sibling document, as a JSON file per row.

Priority P0 · dtype string · equivalence exact

Example params: source=Reference(table='table1', column='invoice') template='invoice'

Field Type Default Description
source Reference — The document-image column to recompute and label.
template Literal['invoice', 'receipt', 'form', 'id_card'] — Which template source was drawn as.

document.form#

A synthetic application form per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: ``

document.id_card#

A synthetic ID-style card per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: ``

document.invoice#

A synthetic invoice per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: ``

document.receipt#

A synthetic receipt per row, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: ``

document.scan_artefact#

A degraded scan of a sibling document, written as a PNG the column points at.

Priority P0 · dtype string · equivalence exact

Example params: source=Reference(table='table1', column='invoice') template='invoice' skew_degrees=3.0 blur_sigma=0.6 jpeg_quality=70 stain_probability=0.3 fold_probability=0.3

Field Type Default Description
source Reference — The document-image column to recompute and degrade.
template Literal['invoice', 'receipt', 'form', 'id_card'] — Which template source was drawn as.
skew_degrees float 3.0 Maximum rotation, drawn per row.
blur_sigma float 0.6 Maximum Gaussian blur radius, drawn per row.
jpeg_quality int 70 JPEG re-encoding quality — lower is noisier.
stain_probability float 0.3 Chance a row gets a coffee-stain overlay.
fold_probability float 0.3 Chance a row gets a fold-line overlay.
← PreviousSpeech narrationNext →CLI reference