Resquites High-Range Tests

THETIS · engine 2.2 · bank 1.5

THETIS technical manual

How THETIS is built and scored, and how precisely it measures: the item bank, the item response model, the adaptive algorithm, the stopping rule, the score report, and a simulation study of 10,000 virtual candidates on the live bank. Updated 2026-11-02.

1. Purpose and intended use

THETIS estimates an adult's fluid reasoning ability on the deviation-IQ scale (mean 100, SD 15), for self-knowledge, research and admission to the Resquites high-IQ societies. It is an online reasoning test, not a clinical or diagnostic assessment, and it should not be used for clinical, legal, educational-placement or employment decisions. It is intended for ages 16 and up; norms are being built for ages 16 to 75.

2. Construct

THETIS measures abductive–dynamic reasoning: inferring a hidden rule from evidence (abduction), testing it against every observation, and updating it when the evidence shows the rule has changed (dynamic updating). Every item has one keyed answer, the prediction of the simplest explanation of everything shown, verified by computer search of the item's full hypothesis space; every distractor is the prediction of a modeled reasoning error. The eighth family, Moorings (bank 1.4), measures deduction instead: its rules are given in full, nothing is hidden or changes, and the keyed answer is the one option the rules force, verified by enumerating every arrangement they allow; its facet is Deductive Reasoning. See About THETIS.

3. Item bank

Bank 1.5 holds 20,923 puzzles in eight families, of which 19,345 are scored: the 749 retired after the audits below and the 829 withdrawn for a repeated entry are never given. Since 20 October 2026 every family is scored, Moorings included (section 5). (The 7 bank puzzles shown in the public practice module are kept apart and never scored; they are not among the 20,923.) Items have 4 options (4,787 items), 5 options (1,440 items), 6 options (14,536 items), 8 options (160 items). The table counts all 20,923 puzzles by family and by the IQ at which each is most informative: its a-priori difficulty b from the item model, on the IQ scale.

The bank is a bell curve of difficulty, deepest at IQ 130–14519,345 scored puzzles of bank 1.5, by the IQ at which each measures best, in eight families
The bank is a bell curve of difficulty, deepest at IQ 130–145Columns for 13 IQ bands from 50–70 to 235+, each split by puzzle family. The bank rises from 585 puzzles in the lowest band to 3,107 at IQ 130–145 and falls to 612 in the highest.01,0002,0003,0004,000Puzzles58550–7090570–8553085–1001,458100–1151,770115–1303,107130–1453,101145–1602,569160–1752,270175–190914190–2051,127205–220397220–235612235+Difficulty of the puzzle, IQ band (the item model's, SD 15)
Table view
IQ bandMutable MatricesDrifting StringsThe Oracle's LedgerTranslationsLevers and LampsTurnstilesOrbitsMooringsTotal
50–7011090159001688122585
70–8513120038240162498158905
85–10019676009600162530
100–115118237290150150242492221,458
115–1302582132073001501512332581,770
130–1454966293493623005162941613,107
145–1604067283792602603884342463,101
160–1754801602932853734013202572,569
175–190161320912634293224162682,270
190–2050004836616041299914
205–220000321522923582931,127
220–2350000275342167397
235+0000612000612

Retired puzzles (749) are left out. Difficulties are the item model's (the anchor calibration is on hold), so a puzzle's band may change once the calibration from first attempts of examinees with known scores is in force.

IQ bandMSCTLGODTotal
50–70110902150016110122663
70–85131200522401624100158921
85–10019676009600162530
100–115118237473150150242492221,641
115–1302582134013001501512582581,989
130–1455106326874503005163111613,567
145–1604067285962602603884432463,327
160–1754801604842854204013232572,810
175–1901613201503204313224242682,396
190–2050004836916041299917
205–220000321722923582931,147
220–2350000281342167403
235+0000612000612

Families: M = Mutable Matrices; S = Drifting Strings; C = The Oracle's Ledger; T = Translations; L = Levers and Lamps; G = Turnstiles; O = Orbits; D = Moorings.

The difficulties form a bell curve. The bank is deepest from IQ 100 to 190 (15,730 puzzles; all eight families have puzzles up to about IQ 180, and five of them beyond IQ 190) and holds 2,114 puzzles below IQ 100, down to about IQ 50. Above IQ 190 it holds 3,079 puzzles in five families (80 Translations, 1,434 Levers and Lamps, 486 Turnstiles, 420 Orbits and 659 Moorings), up to about IQ 257; above IQ 205, 2,162 (32 Translations, 1,065 Levers and Lamps, 326 Turnstiles, 379 Orbits and 360 Moorings). The routing block draws its first five items from IQ 85 to 115 (2,171 puzzles). The table includes the 749 retired puzzles (428 The Oracle's Ledger, 145 Translations, 86 Orbits, 73 Levers and Lamps, 14 Mutable Matrices and 3 Drifting Strings). The generators behind the bank have produced over 100,000 machine-verified candidate items.

Bank 1.5 (20 October 2026) corrects a fault and raises the top of the scale. Repeated entries. The Oracle's Ledger generator had drawn its cards with replacement, so 824 ledgers showed the same card under the same key card twice. Every key was right, but a repeat tells the examinee nothing new and reads as a mistake. These puzzles are withdrawn: never drawn again, while the answers already given to them still count. Each level gets as many new puzzles, at the same difficulties, from the corrected generator, which never repeats a trial; each passed the generator's own check and the separate audit program under every reading (952 of 8,500 candidates failed one and were left out). The 5 Levers and Lamps puzzles that showed two runs alike are withdrawn too. A new check refuses any puzzle whose evidence repeats an example or a statement (a ledger entry, a run, a rule card) when the bank is encrypted and when the server loads it. In the families whose pictures are successive states of one machine, a state that comes back is evidence and stays. New top levels. Levers and Lamps levels 15 to 19 add 745 puzzles from IQ 234 to 257: six lamps, one hidden, a lever held in secret during the last run, and the board asked four to eight ticks ahead as it would have been with no lever. Their wrong options are modeled errors wherever possible (the board one tick late, the hidden lamp left out, the lever kept). Every puzzle passed the generator's check of every explanation with the fewest wires and the separate audit program under its six readings, one of which leaves open whether the secret lever acts on its first tick, so that the key does not depend on how it is read (3,162 of 4,500 candidates failed a reading and were left out), and fits a phone screen. With them the reporting ceiling rises to IQ 230+ (section 6).

Bank 1.4 (16 October 2026) adds an eighth family and closes the gaps left at the top of three others. Moorings is a family of deduction. Boats, drawn as shapes, are to be moored at numbered berths along a pier (from level 13, 7 boats at two piers), and rule cards drawn as pictograms say where they may go: one boat before or after another, next to it or not, not at an end, at a berth or not, and from level 9 if–then and either–or rules. A puzzle asks where a boat must be, which berth must, could or cannot hold a boat, or what follows from one more rule. The rules are given in full and never change, and the key is the one option they force. The generator proves this by enumerating every arrangement (up to 5,040), and a separate audit program, written from the rules text alone by an auditor who never saw the generator or the keys, agreed on all 34,678 candidates. Each wrong option is what a known mistake gives: an either–or rule read as "or both", "before" read as "just before", an if–then rule read backward, a rule forgotten, berths counted from the other end, "could" taken for "must", and so on. No rule card states the asked boat's berth and no option repeats a rule card, so nothing can be read off. A puzzle's difficulty comes from a complexity index (the propagation steps and case splits a solver needs, the rules it uses, how many arrangements the rules leave, the load of the rules and the kind of question), set on the item model's scale so that levels 1 to 3 fall at IQ 60 to 100, 4 to 7 at 100 to 145, 8 to 12 at 145 to 190 and 13 to 16 above 190: 2,513 puzzles in sixteen levels, about 160 each, from IQ 65 to 227. The practice module has two Moorings puzzles, and the rules show a legend of the cards in English, Spanish and Mandarin. From 16 to 20 October 2026 the Moorings puzzles were tried out unscored, two in each session; since 20 October 2026 they are drawn and scored like the puzzles of every other family. Turnstiles (levels 21 and 22, IQ 208 to 225; 320 puzzles) and Orbits (levels 18 and 19, IQ 211 to 212; 319 puzzles) now reach well beyond IQ 205, and two levels of Translations fill the gap between IQ 158 and 181 (levels 22 and 23, at IQ 164 and 175; 285 puzzles, in which no earlier run of the final machine shows its answer). Every new puzzle passed the verifiers written from the published rules (for Orbits, the strict check that a single simplest motion fits; for Turnstiles, every kind of stuck, slow and broken gate) and the separate audit programs under every reading used in the key audits; 56% of the Translations candidates of level 23 failed a check and were left out. Mutable Matrices, Drifting Strings and The Oracle's Ledger will be extended above IQ 190 only after the calibration has shown where their present levels lie.

Bank 1.3 (15 October 2026) extends the three families that stopped lowest, so that every family measures into the high range: 2,560 puzzles in sixteen new levels, built by the same generators with new structures. The Oracle's Ledger (levels 11 to 15: two changes among plain and offset sorts, an if–then rule with an offset branch, three rules in turn one of which is an if–then rule, and longer ledgers with fewer approvals) now reaches IQ 186, where it stopped at 139; Mutable Matrices (levels 12 to 16: the black and the white tokens each follow a schedule of their own, one color's law growing step by step or following a cue socket while the other's stays, alternates or changes) IQ 181, from 159; and Drifting Strings (levels 13 to 18: the cells in even and in odd places each follow a schedule of their own, alternating, changing, depending on a feature of the row or growing) IQ 185, from 151. Each new puzzle was verified twice: by its generator, which tests every explanation in its family's hypothesis space (for the matrices and strings now including every pair of schedules for the two colors or the two sets of places), and by the separate audit programs, extended to the same schedules, under every reading used in the key audit; for the Oracle's Ledger they also try two changes over the whole rule library, if–then rules included. 6% of the candidates failed one of these checks and were left out, and every puzzle kept passes the answer-shown checks. Within each new level the puzzles were chosen for the quality of their wrong options and spread over the level's whole range of difficulty. The levels of Levers and Lamps above IQ 180 were also topped up from the same verified pool as their puzzles: 480 more, each confirmed by the audit program under all six of its readings, so that fewer puzzles are shown again to the examinees who reach them. The rules of Mutable Matrices and Drifting Strings now say that each color, or each set of places, may follow a schedule of its own.

Bank 1.2 (9 October 2026) added 280 puzzles above IQ 190 in three more families, so that the highest scores no longer rest on one kind of puzzle: 100 Orbits, 100 Turnstiles and 80 Translations puzzles, from IQ 191 to 225. Each was solved again by separate programs written from the rules text alone, and samples of them by blind auditors who were not told the keys; all came out with exactly one answer, the key. The Translations puzzles were rebuilt after those audits until none of the shortcuts the auditors looked for worked: every machine's colors have to be worked out (no earlier run of the final machine shows the answer, and whether an earlier run's drawing must be copied or color-swapped varies from puzzle to puzzle); the six options are three laws of the cycle, each drawn in both colorings (the answer's law, the last law as if it stayed, and another law of the cycle), so that each pair of wrong options stands for one step left undone: seeing that the law changes, finding the place in the cycle, or working out the colors; the key is the only answer under every reading of the rules the auditors could construct; and no simple way of picking among the options (the option never drawn before, the one drawn most often, the one farthest from the others, the darkest, one whose negative is offered, and so on) finds the key more than about two percentage points above chance (one in six) in any band of 16 puzzles. A Turnstiles option is now a bucket the marble can reach.

Key audit (7 October 2026). Every puzzle was solved again by a separate program for each family, written from the rules text alone, without the item generators. Of 12,599 puzzles, 12,464 came out with exactly one answer, the key: all Levers and Lamps, Turnstiles and Orbits puzzles, 1,693 of 1,696 Drifting Strings, 1,556 of 1,570 Mutable Matrices, 1,422 of 1,434 Oracle's Ledger and 1,614 of 1,720 Translations puzzles. The other 135 were retired: 18 Translations puzzles had a wrong key (a four-run cycle shown twice in full predicts another option), and 117 had two equally simple answers. Where keys depended on a reading the rules text left open (whether a slow gate may be partway through its count when the record starts, how a changing Oracle rule continues, how a machine's opposite coding applies, that exchanging speeds also exchanges directions, how a cue socket acts), the rules now say so.

Shortcut audit (8 October 2026). A puzzle must be solved by inferring its hidden rule, not by recognizing something it already shows. Every puzzle was checked for an answer that can be read off or copied from the puzzle itself, and 576 were retired: 416 Oracle's Ledger puzzles whose next card already appears in the ledger, 73 Levers and Lamps puzzles in which a shown run passes through the asked situation, 86 Orbits puzzles whose asked picture repeats a snapshot, and 1 Translations puzzle whose final run repeats an earlier run's inputs. The same checks now run before any bank can be encrypted and again when the server loads it.

Translations rules check (9 October 2026). The Translations rules now say what every puzzle assumed: a law is any fixed rule from the two input cells to the output cell; a machine's opposite colors apply to its inputs and output alike and count as one exception, added to the number of laws; a cycle seen twice in full goes on, and its repeats are not changes. Every Translations puzzle was solved again under that wording with every law allowed or only the ten laws of the bank, and with a cycle counted or not as a rule of its own; 38 puzzles of level 11 gave another answer under one of those readings and were retired. The Orbits rules now say that tokens resting on one spoke do not meet again.

4. Item model

The probability of a right answer is the four-parameter logistic model: P(θ) = c + (d − c) / (1 + e−a(θ − b)), with θ the ability on the logit scale (IQ = 100 + 15θ under the current norms). For every item a = 1.8 and c = 1 / (number of options) (0.125, 0.167, 0.200, 0.250); b is the item's a-priori difficulty from THETIS's complexity model of the rules (section 10). The upper asymptote d = 1 − slip allows for lapses: slip = .10 on the first five items, while the format is new, and .02 afterwards (Barton & Lord, 1981; Rulison & Loken, 2009). The discrimination, guessing and slip parameters are a priori, and for now so are the difficulties: their calibration on examinees with known IQ scores waits for enough first attempts (section 10).

5. Adaptive algorithm

  • Routing block. Items 1 to 5 come from the IQ 85–115 band (b between −1 and +1).
  • Mastery gate. No item above IQ 130 until three right answers to items at IQ 85 or harder.
  • Crash protection. The routing estimate may fall at most 0.5 logit per item; the reported score always uses the full estimate.
  • Ladder (engine 2.2, 3 November 2026; the author's rule). From IQ 180 up, the test climbs one level of 10 points at a time (180–189, 190–199, and so on; an item's level is its difficulty in IQ points, 100 + 15b, rounded). A level is cleared by at least three right answers to its items, from at least two families, with at least two in three of its items right. Levels are cleared in order, as the answers come, and a cleared level stays cleared. Only the lowest level not yet cleared is open: no item above its top is given, a routing estimate above it is served at its top, the test does not stop while the estimate stands above it (section 6), and the score is held at its top. Engine 2.1 aimed every item at the routing estimate, which near the top climbs 10 to 20 points with each right answer: a run of right answers could go from the 180s straight to IQ 200 and beyond, so a result of IQ 230+ could rest on three items in the 180s, none in the 190s and a long run of Levers and Lamps items above IQ 230, the only family the bank has there. Now a score in the 200s rests on consistent success in the 180s and the 190s, each level from more than one family. Only Levers and Lamps has items in the 230s, so that level cannot be cleared, and no item above IQ 239 is given (Levers and Lamps' levels above it stay in the bank, unused): a session that opens the 230s ends at most at IQ 239, which is reported as IQ 230+.
  • Selection. Maximum Fisher information at the routing estimate, chosen at random among the 16 (first three items) or 8 most informative eligible items (randomesque exposure control), joined by every item that gives at least 97% of the information of the 8th, up to 16 in all (the near-best draw, 14 October 2026). Items generated at one level share their parameters, so many tie in information; the places that tied items take among the most informative are filled by a random draw from all of them, so every item of a level can be used.
  • Exposure control (14 October 2026). The server counts how often each item has been shown to the examinees for whom it was within reach: those whose final estimate lies within 1.5 logits of its difficulty (finished sessions; author test sessions left out). An item shown to more than a quarter of them is drawn less often, with weight (0.25 / rate)²; an item shown to half of them or more (and at least 8 times) is rotated out while its family has three other items within reach (Sympson & Hetter, 1985; Stocking & Lewis, 1998). Rates count once 25 examinees have been within reach of the item, and the weights are brought up to date as sessions finish, at most every 30 minutes.
  • Content balance. Families compete on information; among those whose best item gives at least 40% of the information of the best item anywhere, the family furthest below its equal share is chosen (Kingsbury & Zara, 1989), never the same family twice in a row when another is eligible.
  • No repeats. A person never sees an item twice, across all their sessions, nor two items built on the same structure: not in one session and, from 14 October 2026, not in a later session either.
  • Moorings (bank 1.4). From 16 to 20 October 2026, two puzzles of every session, at random places from the 6th to the 18th, were Moorings puzzles that did not count (seeded puzzles), meant to calibrate its levels one by one before scoring them. Since 20 October 2026 every Moorings level is scored, by the author's decision: its puzzles are drawn and scored like those of every other family, and nothing is seeded. A session begun before keeps its seeded answers unscored, and its seeded places still to come receive scored puzzles. The calibration study checks every Moorings level on first attempts like the levels of the other families (section 10); the author can retire a level that misfits, and cap Moorings' share of a session above IQ 190 if answers relayed from outside help cluster there.
Every family takes about an equal share from IQ 75 to 190Share of each scored puzzle family in 400 simulated sessions at each true IQ (tools/family-mix.mjs)
Every family takes about an equal share from IQ 75 to 190Stacked columns of each family's share of the puzzles at 16 true IQ levels from 75 to 220. The eight families take about one eighth each from IQ 75 to 190; at IQ 220 the largest share is 17%.0%25%50%75%100%Share7580859095100105110115120130145160190205220True IQ of the simulated examinee
Table view
True IQMutable MatricesDrifting StringsThe Oracle's LedgerTranslationsLevers and LampsTurnstilesOrbitsMooringsPuzzles
7513%13%13%11%13%13%13%13%33.7
8013%13%13%12%13%12%13%13%33.1
8513%13%12%12%13%13%13%13%32.4
9013%12%12%13%13%12%12%13%32.5
9513%13%12%13%13%12%12%13%32.5
10013%13%12%12%13%13%13%12%31.8
10513%12%12%13%12%12%13%13%30.6
11012%13%13%12%12%13%13%12%30.1
11513%13%12%13%12%12%13%12%30.6
12013%13%12%12%12%13%12%13%31.5
13012%13%12%12%13%12%13%13%31.4
14512%13%12%13%12%13%12%13%31.6
16012%12%12%13%13%13%13%12%32.0
19012%12%12%12%12%13%13%13%34.9
2058%10%10%14%14%14%15%15%39.3
2205%6%6%17%16%17%17%17%42.3

The engine balances the families wherever each has puzzles near the examinee's level. Where a family has few puzzles that hard, the others take its place, so the share shifts at the top of the scale.

6. Stopping rule and scoring

Ability is the expected a-posteriori (EAP) estimate on a grid from θ = −6 to 14 (step .05) with a normal prior N(0, 3²), wide enough that very high scores are reachable. The test stops when the posterior standard deviation reaches 0.30 (4.5 IQ points) after at least 20 items, or after 45 items. From engine 2.2 (3 November 2026) it does not stop on the standard error while the estimate stands above the top of the ladder's open level (section 5) and that top is below the reporting ceiling: there is a level still to climb. The report gives the IQ with its 95% range (±1.96 SE), the percentile and, above IQ 130, the rarity. Held scores. An estimate above the top of the open level is reported at that top (IQ 199 when the 190s are open, for example), with a range from the bottom of the open level, or lower, to its top in place of the 95% range, and a note on the report, the certificate and the Verify page that a higher score needs consistent success at that level; the model's estimate is kept with the result. A result held at 239, the top of the 230s, is at the reporting ceiling and is reported as IQ 230+. Response times never enter the score; a puzzle not answered within its clock (section 12) scores as wrong. Reporting ceiling. An estimate at or above IQ 230 is reported as IQ 230+, with the lower end of its range, never as a single figure. The ceiling sits where the simulation stops measuring as precisely as it does just below: from 9 to 20 October 2026 it was IQ 220, near the hardest puzzles of bank 1.4; bank 1.5's Levers and Lamps levels reach about IQ 257, and with them engine 2.1 measured IQ 230 with an actual error of 10.4 IQ points, about as precisely as IQ 220 was with bank 1.4 (9.8), and IQ 240 to 250 only with 12.2 to 15.7, even with up to 60 puzzles. With the ladder (engine 2.2) no item above IQ 239 is given, and IQ 230 is measured with an actual error of 11.9 IQ points (section 9). Certificates keep the ceiling they were issued under (IQ 220+ before 20 October 2026), and those issued before the ceiling existed are shown at IQ 220+.

7. Score report

Besides the IQ and its range, the report gives eight facet scores (augmented subscores from the families' items; the eighth, Deductive Reasoning, comes from Moorings, scored since 20 October 2026; reports issued before read "not measured"), a confidence-calibration index from the optional confidence rating given with each answer, estimated equivalents on other scales (for orientation only), a note on what THETIS measures, and the provisional-norms disclaimer; scores above IQ 195 and below IQ 55, where the bank stops measuring with its usual precision, carry the extreme-range and floor disclaimers. Results computed with the anchor calibration of 7 to 8 October 2026 carry a note that it was withdrawn and that the author can rescore them (section 10). Each finished session has a PDF certificate with an ID and QR code, verifiable at /verify/.

Rule complexity (from 20 October 2026). Next to the IQ, the report, the certificate and the Verify page give the result's rule complexity: the complexity c of the rules the examinee solves half the time beyond chance. The item model sets every puzzle's difficulty from the complexity of its rules (section 4): b = 1.35 + 1.0115 (c − 1.909) logits from the model's anchor at c = 1.909, and b = 1.35 − 3.10 (1.909 − c) below it. Read backward at the estimate θ, the same relation gives c, and θ ± 1.96 SE gives its 95% range; for example Rule complexity 2.6 (95% range 2.0–3.1) at IQ 130, 4.5 (95% range 3.9–5.1) at IQ 160 and 6.5 (95% range 5.9–7.1) at IQ 190. The bank's puzzles run from c = 0.4 to 10.9, and at the reporting ceiling the score is given as 9.1+. The IQ ranks a result in a population and needs norms; the rule complexity describes what was solved, so no norm enters it, and it keeps its meaning above the range where population norms can exist (above about IQ 190). It is the same estimate in other units: it adds no precision, and it is only as good as the item model's complexity coding, which the calibration on first attempts will check. Certificates issued before 20 October 2026 do not carry it.

8. Simulation study

10,000 virtual candidates in the specification's seven tiers, each with a personal attention-slip rate drawn between .02 and .10, took THETIS through the server's own engine on the live bank: its 19,345 scored puzzles in eight families, Moorings included (practice and retired puzzles left out, as on the server), each with its own difficulty, discrimination, guessing parameter and options (seed 20261006). Their answers follow the item model with their own slip rate, which is larger than the engine assumes after item 5: the results include that misfit.

TierTrue IQNItems (min–max)BiasRMSEMean SE95% coverageStopped at 45
Clinical40–7050039.5 (28–45)−0.34.214.8896.4%32.4%
Low average70–851,00033.2 (27–45)−1.14.924.4592.3%2.7%
Average85–1154,00031.7 (24–45)−1.04.924.4592.5%1.8%
High average115–1302,00031.4 (24–45)−1.14.994.4492.7%1.1%
Gifted130–1601,50031.6 (27–45)−1.85.164.4691.7%1.5%
Profoundly gifted160–20080033.9 (28–45)−2.76.594.4889.6%4.9%
Beyond200–24020041.8 (33–45)−5.510.435.0579.5%42.5%

Bias and RMSE in IQ points (estimate minus true). Coverage: share of 95% ranges containing the true IQ. The routing rule held in 100.0% of sessions (all first five items within IQ 85–115).

Across all tiers the correlation between true and estimated IQ is .99 and the root-mean-square error 5.25 IQ points, with 32.5 items on average. Estimates run slightly low (1.0 to 2.7 points from the Average to the Profoundly gifted tier) because the virtual candidates slip more often than the engine's .02 assumes; human data will show which slip rate fits real candidates. For the same reason the 95% range holds the true IQ in 90% to 93% of sessions in those tiers rather than 95%.

9. Reliability and precision

  • Marginal reliability in a normal population sample of 2,000 virtual candidates: .92 (correlation of true and estimated IQ .95), with 31.8 items on average.
  • Retest with entirely new items (1,000 candidates, two sessions each): r = .91; mean change −0.49 IQ points.
  • Standard error at the end of a session: 4.5 IQ points or less unless the 45-item limit is reached, so the 95% range is about ±9 points.
  • Precise range. At fixed true abilities (1,600 sessions at each 5-point step, and 6,400 from IQ 180 to 210, where the range ends and a smaller run cannot tell its last level from the next) the error stays near its target (RMSE 7 or less, averaged with the neighboring levels) from about IQ 55 to 200: RMSE 4.6 to 6.9 IQ points at single levels, 5.3 on average. Above IQ 200 it grows, as more and more sessions reach the 45-item limit before the estimate has climbed all the way (RMSE 9.9 at IQ 220, 13.9 at IQ 240); scores there carry that wider margin, up to the reporting ceiling of IQ 230+ (section 6), which 29% of sessions reach at IQ 230, 78% at IQ 240 and 88% at IQ 250. The ladder (section 5) spends puzzles on every level from IQ 180 up, below the examinee's own, so the climb is surer and the measurement near the top a little less precise than with engine 2.1, which could pass over whole levels: from IQ 215 to 230 the error was 8.5 to 10.4 points with engine 2.1, and is 9.7 to 11.9 with the ladder.
  • Item exposure. In the population sample of the simulation (2,000 sessions on the scored puzzles of bank 1.5), the most used item appeared in 6.3% of sessions, 6,306 different items were used, and no item was used in more than 20% of sessions. Exposure matters most among examinees of one level, who meet the same items: at fixed true abilities (400 sessions each, tools/exposure-study.mjs), the most used item appeared in 10% of sessions at IQ 160, 13% at IQ 190 and 17% at IQ 205 with the near-best draw (12%, 16% and 24% without it), with RMSE 5.9 at IQ 190. In a stream of 3,000 sessions from a high-range population (mean IQ 135), the highest rate at which an item was shown to the examinees within its reach was 18% without the exposure control and 21% with it; no item passed the 25% limit either way, so none was rotated out, at the same precision (RMSE 5.1 and 5.3). With bank 1.2 the most used item reached 24% of the sessions at IQ 175, and one item passed the limit without the control.
  • Retakes. A retake never repeats an item or its structural twin, so each attempt meets new items. Simulated people at IQ 150, 175 and 200 taking THETIS four times keep their precision: RMSE 5.6, 5.0, 5.1, 5.2 at IQ 150; 5.6, 5.3, 5.8, 5.9 at IQ 175; 6.8, 7.0, 5.8, 8.8 at IQ 200 on attempts 1 to 4. Real retakes are studied as they come in (section 10).
Actual error stays between 4.6 and 6.9 points from IQ 55 to 200, then widens1,600 simulated sessions at each 5-point step of true IQ (6,400 from IQ 180 to 210, where the precise range ends) · engine 2.2 on the 19,345 scored puzzles
Actual error stays between 4.6 and 6.9 points from IQ 55 to 200, then widensTwo lines over true IQ from 55 to 250. The reported standard error stays near 4.5 IQ points across the range; the actual error stays between 4.6 and 6.9 from IQ 55 to 200, then rises to 18.2 at IQ 250.Precise range: IQ 55 to 20005101520IQ pointsStopping target: SE 4.5918.2557085100115130145160175190205220235250True IQ of the simulated examinee
Table view
True IQReported SEActual error (RMSE)BiasPuzzlesStopped at 4595% range held the true IQ
554.64.6−0.540.125%96%
604.54.6−0.437.510%95%
654.54.6−0.535.45%95%
704.54.8−1.034.53%93%
754.54.7−0.933.62%94%
804.44.8−0.932.72%92%
854.55.0−0.932.22%92%
904.55.1−1.132.42%92%
954.55.0−0.932.52%91%
1004.45.1−1.031.92%92%
1054.44.9−0.930.72%92%
1104.44.9−0.830.21%93%
1154.44.7−0.831.11%94%
1204.45.0−1.231.52%92%
1254.44.9−1.031.62%93%
1304.45.3−1.431.42%90%
1354.44.9−1.331.31%94%
1404.45.0−1.531.52%92%
1454.45.1−1.731.61%92%
1504.45.2−1.831.81%90%
1554.45.4−2.031.82%91%
1604.45.4−2.032.22%91%
1654.45.4−2.032.21%90%
1704.45.6−2.333.13%90%
1754.56.2−2.533.34%89%
1804.56.2−2.533.74%89%
1854.56.3−2.634.15%88%
1904.56.5−2.734.97%89%
1954.56.5−3.036.39%87%
2004.66.9−3.238.114%86%
2054.67.5−3.439.218%84%
2104.78.5−3.940.124%83%
2154.89.7−4.441.131%81%
2204.99.9−4.442.037%82%
2255.011.5−4.843.252%84%
2305.311.9−5.344.174%80%
2355.812.1−6.044.789%79%
2406.713.9−7.844.997%80%
2457.915.0−10.644.999%87%
2509.018.2−14.844.999%89%

The simulated examinees slip on 2 to 10% of answers, more often than the engine assumes after the first five puzzles, so the actual error runs a little above the reported one. Above the precise range the bank holds fewer well-targeted puzzles, so more sessions run to 45 puzzles and the estimate widens.

Most sessions end after about 32 puzzlesThe test stops when the score is precise enough (SE 4.5 IQ points), after 20 to 45 puzzles
Most sessions end after about 32 puzzlesOne line over true IQ from 55 to 250: the average number of puzzles, lowest at 30.2 and rising toward 45 at the ends of the scale, where well-targeted puzzles are fewer.01020304050PuzzlesAt least 20 puzzlesAt most 45 puzzles44.9557085100115130145160175190205220235250True IQ of the simulated examinee
Table view
True IQPuzzles on averageSessions that ran to 45
5540.125%
6037.510%
6535.45%
7034.53%
7533.62%
8032.72%
8532.22%
9032.42%
9532.52%
10031.92%
10530.72%
11030.21%
11531.11%
12031.52%
12531.62%
13031.42%
13531.31%
14031.52%
14531.61%
15031.81%
15531.82%
16032.22%
16532.21%
17033.13%
17533.34%
18033.74%
18534.15%
19034.97%
19536.39%
20038.114%
20539.218%
21040.124%
21541.131%
22042.037%
22543.252%
23044.174%
23544.789%
24044.997%
24544.999%
25044.999%

10. Norms

The norms are provisional: IQ = 100 + 15θ, with θ the engine's estimate on the item model's a-priori difficulties. Those difficulties come from THETIS's complexity model, which scores the rules of each puzzle (its laws, their changes, what is hidden and how much must be held in mind), extended downward so that the simplest puzzles sit near IQ 50; no human data have calibrated them yet.

Anchor calibration, 7 to 8 October 2026 (on hold). Two examinees with known scores of IQ 135 to 147 (Raven's Advanced Progressive Matrices 139, Mensa 135, Glia Society 147) scored 50 to 99 on the a-priori difficulties. Holding them at their known scores, one difficulty shift per family was estimated by maximum a posteriori under the engine's own model, partially pooled around the common mean: Mutable Matrices +3.64, Drifting Strings +3.27, The Oracle's Ledger +3.74, Translations +2.04, Levers and Lamps +4.19, Turnstiles +3.96, Orbits +2.26 logits, where one logit is 15 IQ points (7 sessions, 209 answers). It was put on hold on 8 October 2026: it rested on two examinees' retakes and author test sessions rather than first attempts, and carrying shifts measured on the easiest puzzles to every puzzle inflated the highest scores. Sessions in progress moved back to the a-priori difficulties with their answers kept. Results computed with the calibration keep their stored scores, carry a note that it was withdrawn, and can be rescored by the author on the current norms, one certificate at a time or all at once.

Calibration from first attempts (the procedure from bank 1.4). The calibration will be estimated again from first attempts only (a retake meets puzzles of the same families a second time, and an author test session is not a real attempt), with each anchor examinee's ability held at a score from a supervised, professionally normed test. Anchors come from two sources: examinees who report such a score in the questionnaire after the test and tick that the author may contact them to confirm it (only they are shown to the author with their name and email, and the author adds the score once its proof is in hand), and people the author invites with a code that records their known score, counted on their first attempt only. The aim is 30 anchors spread over IQ 85 to 115, 115 to 130 and above 130; an estimate needs at least 10. Before anything is applied, the author's tools show the result for every anchor session, including the score calibrated without that person (the honest check of how well it predicts someone it was not fitted to), and a calibration can be applied only when these left-out scores lie closer to the known scores than the current difficulties' do. Once one is applied, the author lists every certificate it would change and rescores them together. The calibration is estimated again at 30 anchors and at each study snapshot, but a snapshot by itself never moves a score. A person's own reported scores never enter their own THETIS score.

Studies on real data (from 14 October 2026). The author's tools run three studies on the stored sessions, and the server keeps a snapshot each time the number of first attempts passes 30, 50, 100, 200, 500 and 1,000. None of them changes a score. Calibration: with each examinee's estimate held fixed, the difficulty shift of every family and of every family level that fits the first attempts best (fixed-ability online calibration; Stocking, 1988; Ban et al., 2001), each family's relative to the mean of the families, with the standardized residual of observed against expected right answers and infit and outfit mean squares (Wright & Masters, 1982); a level is flagged when it misfits clearly (|z| of 3 or more with at least 30 answers, or infit outside .75 to 1.33 with at least 90). The levels added in banks 1.3 and 1.4 are calibrated the same way as their answers come in, and a level that keeps misfitting is retired. Moorings levels: each level's answers so far (the seeded answers of 16 to 20 October 2026 included), how many come from people near its difficulty, and its fit. Integrity by family: the share of each family's answers given just after a return to the window, after a screenshot shortcut or unusually fast, so that a family that draws outside help shows. Retakes: the test–retest correlation of each person's first two sessions, the practice gain (second minus first, with its 95% interval) by the time between them, and the correlation the measurement error alone would allow. Fairness: first attempts by first language (English or another) and by device (phone, tablet or computer, as the page reports it): mean scores, integrity and person-fit flag rates, and differential item functioning per family and per level by the Mantel–Haenszel procedure, matching on the ability estimate (Holland & Thayer, 1988; variance after Robins, Breslow & Greenland, 1986), classified A (negligible), B (moderate) or C (large). On simulated sessions with effects built in, the studies find a misplaced level, a practice gain and a family that is harder for one language group, and report no difference by device where there is none.

Facet check (bank 1.4). At 500 first attempts, and again at 1,000, the author's tools test whether the facets measure anything beyond the overall IQ, before the score report changes for good. A bifactor item response model (Gibbons & Hedeker, 1992) with a general factor and one specific factor per facet is fitted by marginal maximum likelihood to the answers themselves (not to the facet scores), each family level an item, split by question type where a family asks more than one kind. Haberman's (2008) test then asks of each facet whether its own score predicts the facet's true score better than the overall score does (proportional reduction in mean squared error), and the facet scores are compared by first language and by device. Each facet is kept, merged with another or reported no more, and if none adds value the report gives the overall IQ alone. With about four scored answers per facet in a session, simulations suggest that few facets will pass; until the check has run, the facet scores are labeled as a profile, not as separate scores.

Results so far

From 30 first attempts on, this section shows how first-attempt scores are spread (mean, median, spread, devices, length), and how they compare with the earlier scores examinees report in the optional questionnaire after their result, each kind of earlier test once 30 people have reported one. A first attempt counts a day after it ends, and one whose integrity record is for review is left out. Only totals are shown: no single result can be picked out.

Three corrections named in the THETIS specification remain in the scoring module, switched off: a difficulty shift of the bank (Δb = 1.2), a linear link to Raven's Advanced Progressive Matrices (a = −0.15, b = 1.35) and a Pearson Type IV score distribution (skewness 0.5, kurtosis 4.0). They change scores substantially and rest on no human data, so from bank 1.4 the author's tools no longer offer them: THETIS reports on the plain scale, which moves only through the calibration from first attempts above.

11. Data quality and integrity

Each first session carries four flags that never change a score: person fit (the standardized log-likelihood lz, Drasgow, Levine & Williams, 1985, judged against model-consistent sessions of the same ability), timing consistency, rapid wrong answers, and leaving the test page. Flagged sessions can be examined or set aside when norms are built.

Integrity record (engine 2.1, from 9 October 2026). THETIS is taken online without a supervisor, so every session also carries an integrity record, marked clean or for review with its reasons, which the author reviews before any society admission. It never changes a score. It looks for the traces outside help leaves (another person, a program, an AI system): an unlikely answer pattern (lz below the 1st percentile of honest sessions); answer times that do not follow the person's own pace under the lognormal response-time model (van der Linden, 2006; timing index above the 99th percentile); answer times that do not rise with the puzzles' expected solution times (the slope of log time on log expected time below 0.3 over at least 20 answers: in simulated sessions on bank 1.5 with all eight families scored, the current clocks and the ladder of engine 2.2 this flags 0.07% of honest examinees, whose slope is near 1, 97% of sessions whose every answer was relayed from elsewhere, which take about the same time on every puzzle, and 48% of sessions relayed only on the puzzles above IQ 145); hard puzzles (above IQ 145) answered right in under 15% of their expected time, measured on the server's own clock as well as the page's; leaving the test window while a puzzle is on screen, twice or for 20 seconds in all; copy, print or right-click attempts; and a perfect run at the reporting ceiling.

Per-puzzle evidence and the sealed record (from 10 October 2026). With each answer the test page sends what it saw while the puzzle was on screen: departures from the window with their times, screenshot shortcuts (Print Screen, Win+Shift+S, Shift-Command-3/4/5), copy, print and drag attempts, and AI assistant add-ons that insert themselves into the page (no other add-on is recorded). Three more checks follow: screenshot, relay (away 10 seconds or more, then the right answer within 15 seconds of coming back, on two puzzles or one above IQ 145) and addon. The server keeps its own time for every puzzle and extends a SHA-256 hash chain over the answers, times and evidence; the chain's fingerprint is printed on the certificate and shown on the verification page, so the record cannot be changed afterwards without the change showing. Each puzzle carries a faint watermark of the examinee's name, certificate ID, puzzle number and time. No website can see a screenshot itself, or a photo of the screen; these are the traces that come with them. The author's integrity report (PDF) gives every puzzle's timeline and the check of the seal. RUFIT carries the same evidence and seal from 13 October 2026.

Per-puzzle evidence and the sealed record (10 October 2026). No website can see a screenshot itself, or a photo of the screen taken with a phone, so THETIS records the traces that come with them, puzzle by puzzle: every departure from the test window (when, for how long, and whether to another tab or to another window or app); screenshot shortcuts (Print Screen; Win+Shift+S, Shift-Command-3, 4 or 5 and Ctrl+Shift+S; or Win or Command with Shift held as the window loses focus, the sign of a snipping overlay); copy, print, right-click and drag attempts; elements that browser add-ons insert into the page; and the longest pause in input. The server adds its own times: when it sent each puzzle and when the answer arrived. Three more reasons for review follow from this evidence: screenshot shortcuts; the right answer within 15 seconds of coming back after 10 seconds or more away from the window (the trace of a puzzle carried to outside help and back), on two puzzles or on one above IQ 145; and an AI assistant add-on in the page. The watermark shows the examinee's name, certificate ID, puzzle number and the time the puzzle was served, so a copy found elsewhere can be traced to its session and puzzle. As each answer is recorded, the server extends a hash chain (SHA-256) over the answer, its times and its evidence; when the session ends, the chain's fingerprint is printed on the certificate and the Verify page, so the record cannot be changed afterwards without the change showing. The author's integrity report (PDF) sets out the result, the integrity record, every puzzle's timeline and the check of the seal. The page's evidence is what the examinee's browser reports: a second device leaves no trace in it, and a tampered page can report nothing, which is why the answers and their times are checked as well.

Camera check (required from 3 November 2026). No web page can see a phone photo of the screen; a camera can. Every session now turns on the camera check before its first puzzle: an object detector running in the examinee's own browser (COCO-SSD on TensorFlow.js; the video never leaves the device) looks every two seconds for a phone or another person in view (two checks in a row) and for an empty seat (nobody in view for a minute or more), and counts the time the camera was off or covered. A puzzle is covered on screen while the camera is not watching (its clock keeps running). Each answer carries the camera's summary of its puzzle, sealed with the record; when the detector sees one of these, the page keeps a small still picture (160 by 120 pixels) for the author's review, deleted after 90 days. A phone, another person, an empty seat during a timed puzzle, and puzzles the check did not cover are reasons for review in the integrity record; they never change the score. A result whose check covered every puzzle is marked so on its certificate and Verify page. The check cannot see a device or a helper out of the camera's view: like the rest of the record, it is a trace, not proof.

Supervised verification (optional). Unproctored internet testing cannot by itself support the rarest claims (International Test Commission, 2006). From 9 to 17 October 2026 a THETIS result of IQ 190 or more counted for society admission only after a supervised verification; since 17 October 2026 no result needs one, and every application is decided on the certificate and its integrity record. The author can still arrange a supervised verification of any result: a one-time code for the certificate's holder, who takes a second THETIS session (no practice, puzzles not seen before, the same clocks) on a video call with camera and whole screen shared. The result is confirmed when the second estimate is not significantly below the first (their difference within 1.96 standard errors of the difference); the certificate and the Verify page then show Verified, and otherwise Not confirmed with the supervised result. The supervised session keeps its own certificate, marked as supervised. The author can also withdraw a certificate.

12. Session delivery

Clocks (engine 2.1). Every puzzle has its own time limit: eight times the puzzle's expected solution time (the item model's), from 3 minutes for the easiest to 20 minutes for the hardest. THETIS is a power test, so the clock must almost never decide a score: answer times spread around their median with a log-scale SD of about 0.5 to 0.7 between people, so eight times the expected time leaves out about one honest answer in 500, even for slow and careful examinees. (From 9 to 13 October 2026 the limit was about four and a half times the expected time, from 1½ to 8 minutes; the 8-minute cap gave the hardest puzzles barely three times their expected time.) The server keeps the clock from the moment it sends the puzzle and accepts an answer up to 5 seconds after it ends (network delay); after that the puzzle counts as unanswered. The page shows the time left, sends the selected answer (or none) when it reaches zero, and resumes with the time left after a reload. Between puzzles the examinee may take a break: the next puzzle, and its clock, start only when they continue. The limits leave THETIS a power test for honest examinees while ending unlimited time for outside help, and they make answer times usable evidence for the integrity record. Sessions started before 9 October 2026 finish untimed.

Each answer is saved on the server as it is given, in Redis with a durable copy in Netlify Blobs; every write is a compare-and-set on a revision number, and each item carries a one-time token, so a retried or doubled request never counts an answer twice. The page retries every request with growing pauses, keeps an unsent answer in the browser, and resumes a reloaded or reopened session on the same item. A connection failure can therefore never change a score.

13. Limitations

  • The puzzle difficulties are the item model's: no human data have calibrated them yet. Two examinees with known scores of IQ 135 to 147 scored far lower on them (retakes and author test sessions), and a first attempt by an examinee with supervised scores near IQ 145 scored far higher on the anchor calibration. Which way, and how far, the item model errs at each level will be settled by first attempts of examinees with known scores. The simulation shows the engine's precision if the item model is right, not the test's validity.
  • Norms are provisional; above about IQ 190 a score is a position on the THETIS scale, not a population rarity.
  • An unsupervised online test cannot rule out outside help. The clocks, the watermark, the per-puzzle evidence and the integrity record make it harder and more visible, but a phone or a second device used beside the test leaves no trace in the browser. An online result is accepted with its integrity record, which the author reviews before any society admission, and the author can ask for a supervised verification of any result in dispute.
  • Human validity data are limited. Concordance with earlier proctored scores is being collected through the optional questionnaire; the comparisons so far are in section 10 (Results so far). They rest on scores people report themselves, from tests taken under different conditions and in different years, and on the people who chose to report one, so they describe this sample rather than prove an equivalence.
  • The engine keeps the eight scored families in balance, about one eighth of the puzzles each, up to IQ 190 in the simulation (400 sessions per level; up to IQ 145 before bank 1.3). Above it the families with the hardest puzzles take a larger share: at IQ 205, Orbits 15%, Moorings 15%, Levers and Lamps 14%, Translations 14%, Turnstiles 14%, Drifting Strings 10%, The Oracle's Ledger 10% and Mutable Matrices 8%; at IQ 220, Turnstiles 17%, Orbits 17%, Translations 17%, Moorings 17%, Levers and Lamps 16%, Drifting Strings 6% and The Oracle's Ledger 6%. The highest scores rest mostly on the five families whose puzzles reach beyond IQ 205, and sessions there are longer (42 puzzles on average at IQ 220).
  • The difficulties of the bank 1.3 and 1.4 levels come from the same item model as the rest of the bank, extended to structures it had not been used on; until first attempts calibrate them (section 10), they carry the same caution as every a-priori difficulty, and more. Moorings' difficulties rest on a complexity index of its own; its puzzles count from 20 October 2026 on those a-priori difficulties, like the other families' on theirs, and the study of first attempts checks each of its levels (section 10).

14. Version history

  • Engine 2.2 (3 November 2026): the ladder above IQ 180 (section 5): one level of 10 points at a time, a level cleared by three right answers from at least two families with at least two in three of its items right, no item above the open level, no stop on the standard error while the estimate stands above it, and the score held at its top (section 6); no item above IQ 239; held scores shown as such on the report, the certificate, the result email and the Verify page; the integrity report lists every session's levels above IQ 180 (and names Moorings in its timeline); the author's rescoring applies the ladder to sessions of engines 2.0 and 2.1, and their certificates keep their scores until the author rescores them; the simulation, the family mix, the exposure study and the person-fit cut-offs run again.
  • Results so far (22 October 2026): the distribution of first-attempt scores and the comparisons with earlier scores reported in the questionnaire, published from totals kept by the server (section 10).
  • Bank 1.5 (20 October 2026): the 824 Oracle's Ledger puzzles that showed a card under the same key card twice withdrawn (never drawn again; answers already given still count) and replaced level by level by the corrected generator, and the 5 Levers and Lamps puzzles that showed two runs alike withdrawn; a check against repeated entries when the bank is encrypted and loaded; Levers and Lamps levels 15 to 19 (745 puzzles, IQ 234 to 257); the reporting ceiling raised to IQ 230+; the simulation, family mix, exposure study and person-fit cut-offs run again.
  • Rule complexity (20 October 2026): a reading of each result that no norm enters, the complexity of the rules solved half the time beyond chance, on the score report, the certificate and the Verify page; a sample certificate on the THETIS page.
  • Moorings scored (20 October 2026): every Moorings level scored, by the author's decision, without waiting for the level-by-level calibration; nothing is seeded any more (sessions begun before keep their seeded answers unscored); the Deductive Reasoning facet measured; the simulation, the family mix, the exposure study and the person-fit cut-offs run again on the eight scored families.
  • Bank 1.4 (16 October 2026): Moorings, a family of deduction (2,513 puzzles in sixteen levels, IQ 65 to 227), seeded unscored, two puzzles a session, and scored level by level from the author's tools once each level is ready; the eighth facet, Deductive Reasoning; two Moorings puzzles in the practice module (16 puzzles) and a legend of the rule cards in three languages; 320 Turnstiles and 319 Orbits puzzles beyond IQ 205 and 285 Translations puzzles at IQ 164 and 175; the calibration procedure (anchors from the questionnaire with consent to be contacted, and from invited examinees; applied only when it predicts left-out anchors better); the facet check at 500 and 1,000 first attempts; the simulation, the family mix, the exposure study and the person-fit cut-offs run again, with a projection for eight scored families.
  • Bank 1.3 (15 October 2026): 2,560 puzzles in sixteen new levels of The Oracle's Ledger (to IQ 186), Mutable Matrices (to IQ 181) and Drifting Strings (to IQ 185), verified by their generators and by the audit programs extended to the new schedules; 480 more Levers and Lamps puzzles above IQ 180; the rules of Mutable Matrices and Drifting Strings made explicit about schedules of their own for each color or set of places; the simulation, the family mix, the exposure study and the person-fit cut-offs run again on the new bank.
  • Exposure control and studies of real data (14 October 2026): the near-best draw; exposure weights and rotation by the rate at which an item is shown to the examinees within its reach; retakes that never repeat an item or its structural twin; the device and, optionally, the first language recorded; calibration, retake and fairness studies in the author's tools, with snapshots as first attempts accumulate.
  • Evidence and seal (10 October 2026): per-puzzle evidence from the test page (departures, screenshot shortcuts, add-ons, input pauses) with the server's own times; three more reasons for review (screenshot shortcuts, right answers just after coming back, AI assistant add-ons); the watermark with puzzle number and time; each record sealed with a hash chain whose fingerprint is on the certificate; the integrity report for the author.
  • Engine 2.1 (9 October 2026): a clock on every puzzle, kept by the server, with breaks between puzzles; the integrity record; reporting ceiling IQ 220+; supervised verification from IQ 190 (no longer required since 17 October 2026); the watermark. Estimation unchanged.
  • Bank 1.2 (9 October 2026): 280 puzzles above IQ 190 in Orbits, Turnstiles and Translations, verified by separate programs and blind audits; Translations and Orbits rules made explicit; 38 Translations puzzles retired; the answer-shown checks enforced before encryption and at load; the simulation run on the live bank's 12,128 scored puzzles, with 1,600 sessions per ability level in its precision profile, and the person-fit cut-offs recomputed on them.
  • Shortcut audit (8 October 2026): 576 puzzles that showed their own answer retired.
  • Norms of 8 October 2026: the item model's difficulties again (the bank's bell curve); the anchor calibration on hold until it can be estimated from first attempts of at least 10 examinees with known scores; sessions in progress moved back with their answers kept; results computed with the calibration marked, and rescorable in the author's tools; disclaimers at IQ 55 and 195; simulation on a model of the bank.
  • Exposure control (7 October 2026): items that tie in information are drawn at random from all of them instead of in the bank's order, which spreads use over about four times as many items with no change in precision; the simulation leaves out the retired and practice items, as the server does, and its precision profile uses more sessions per ability level.
  • Anchor calibration (7 October 2026; on hold since 8 October): item difficulties shifted by family on two examinees with known IQ scores (+2.04 to +4.19 logits); sessions in progress moved onto it; tools to re-estimate it and rescore finished sessions; disclaimers at IQ 80 and 200.
  • Engine 2.0 (6 October 2026): untimed (to 9 October 2026); four-parameter model with slip; routing block, mastery gate and crash protection; stop at SE 0.30 after 20 to 45 items; mandatory 14-item practice module (16 from bank 1.4); facet scores, confidence calibration and concordance in the report; Redis session store.
  • Earlier versions (October 2026): each item had a generous time limit; engine 1.0 also reported a processing-speed index; bank 1.1 made THETIS a power test scored on reasoning alone and added the Turnstiles and Orbits families; the test stopped at SE 0.28 after 18 to 34 items.

15. References

  • Ban, J.-C., Hanson, B. A., Wang, T., Yi, Q., & Harris, D. J. (2001). A comparative study of on-line pretest item calibration/scaling methods in computerized adaptive testing. Journal of Educational Measurement, 38(3), 191–212.
  • Barton, M. A., & Lord, F. M. (1981). An upper asymptote for the three-parameter logistic item-response model (RR-81-20). Educational Testing Service.
  • Drasgow, F., Levine, M. V., & Williams, E. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67–86.
  • Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57(3), 423–436.
  • Haberman, S. J. (2008). When can subscores have value? Journal of Educational and Behavioral Statistics, 33(2), 204–229.
  • Heinrich, J. (2004). A guide to the Pearson Type IV distribution (CDF/MEMO/STATISTICS/PUBLIC/6820). University of Pennsylvania.
  • Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel–Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Lawrence Erlbaum.
  • International Test Commission. (2006). International guidelines on computer-based and internet-delivered testing. International Journal of Testing, 6(2), 143–171.
  • Kingsbury, G. G., & Zara, A. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2(4), 359–375.
  • Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel–Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42(2), 311–323.
  • Rulison, K. L., & Loken, E. (2009). I've fallen and I can't get up: Can high-ability students recover from early mistakes in CAT? Applied Psychological Measurement, 33(2), 83–101.
  • Stocking, M. L. (1988). Scale drift in on-line calibration (RR-88-28). Educational Testing Service.
  • Stocking, M. L., & Lewis, C. (1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics, 23(1), 57–75.
  • Sympson, J. B., & Hetter, R. D. (1985). Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th annual meeting of the Military Testing Association (pp. 973–977). Navy Personnel Research and Development Center.
  • van der Linden, W. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31(2), 181–204.
  • Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. MESA Press.
  • Wright, B. D., & Stone, M. H. (1979). Best test design. MESA Press.