Skip to content

Validation and tolerances ​

Every algorithm is compared with Crystal-Builder or sci-form golden data. The tolerances below are the initial contract; a test that needs a wider tolerance must record the reason in the algorithm page.

FeatureComparisonTolerancePhase
Element reference dataexact0 (copied from Crystal-Builder)1
Cell parametersabsolute1e-6 Angstrom / 1e-6 degree1
Cell volumerelative1e-91
Fractional coordinatesabsolute1e-61
Atom setsup to translation and permutation1e-6 fractional1
Bond lengthabsolute1e-4 Angstrom1
Bond orderabsolute1e-31
Bond setcanonical (i, j, image) after P1 atom mappingexact1
Bond length (full corpus)absolute0.05 Angstrom (inherits the P1 mapping tolerance)1
Coordination numbersexact01
P1 atom mapping (golden)absolute periodic distance0.05 Angstrom (symmetry-standardized sites)1
Ring size/countexact01
Minimum-image distanceabsolute1e-9 Angstrom1
CIF cell parametersabsolute1e-6 Angstrom / degree1
CIF atom countexact01
POSCAR/XYZ round tripabsolute1e-9 Angstrom1
Neighbor distanceabsolute1e-9 Angstrom1
Slab atoms per layerexact01
Ring pathsatom indices and imagesexact after atom mapping, translation home and direction1
Ring budgettoo-dense refusalexact parity with the golden error1
Functional groupsatom setsexact after atom mapping1
Space group number / Hall numberexactexact2
Wyckoff lettersexactexact2
Standardized cellabsolute1e-5 Angstrom / 1e-4 degree2
K-pathpoints and labels1e-6 fractional2
PXRD positionsabsolute0.01 degree 2-theta5
PXRD intensitiesabsolute after normalization1e-35
UFF energyabsolute1e-6 eV per atom4
UFF forcesabsolute1e-4 eV/Angstrom4
UFF stressabsolute1e-3 GPa4
Phonon q frequenciesabsolute0.5 cm^-1 (phonopy spring golden)4
Indexing figures of meritrelative to the Python reference1e-47
Indexed lattice (golden)Q-spectrum agreement5e-4 relative7
Peak fit centreabsolute2e-3 degree (synthetic Gaussian)7
Pawley cellabsolute2e-3 Angstrom (golden pattern)7
Pawley zero shiftabsolute5e-3 degree (golden pattern)7
Pawley intensityabsolute3 esds7
Pawley Rwp / chi^2absolute0.02 (golden pattern)7
Le Bail intensityfour esds90% of a noisy synthetic pattern7
Rietveld positionabsolute3e-3 fractional (rutile golden)7
Rietveld cellabsolute2e-3 Angstrom (rutile golden)7
Rietveld zero shiftabsolute0.02 degree (rutile golden)7
Rietveld Rwpabsolute0.25 (rutile golden, single-Gaussian model)7
Space-group screenexactrutile P4_2/mnm among the candidates; 001 removes it7
Space-group screenexactdiamond Fd-3m among the candidates; 200 removes it7
Extinction classrankingrutile P4_2/mnm is the first golden row7
EQeq chargesabsolute1e-3 e against the authors' program on MOF-54
UFF energyabsolute1e-4 kcal/mol against sci-form on methane and water4
Chargesabsolute1e-6 e4
Porosityabsolute0.01 Angstrom / 0.1 percent5
Topologycanonical graphexact5
SMILES round tripgraph and chargesexact after atom mapping6
Embedding bond lengthabsolute0.3 Angstrom for ring closure6
MOF assemblyatom count and finite coordinatesexact/deterministic6
Powder positionabsolute0.1 degree 2-theta7
Rietveld scale/backgroundrelative1e-37
Structural refinementRwpnon-increasing over accepted steps7
Instrument doubletratio1e-97

Rules ​

Validation flow

  • The comparison helpers live in xtalforge-testkit (Task 0.9.4).
  • Golden files are generated by scripts/golden/ and never by CI.
  • The independent end-to-end battery (tests/e2e/, scripts/e2e.sh) compares the WASM surface with gemmi, spglib, ASE and NumPy references and with committed 3D screenshots; its scope, findings and known gaps are in the audit page.
  • A deliberate deviation from Crystal-Builder is recorded in the algorithm page with the reason and the affected structures.
  • Tightening a tolerance is a normal change; widening one requires a note in the changelog and in the algorithm page.