xtalforge

Appearance

End-to-end audit and visual validation ​

The 1.1.0 API is verified against independent reference implementations and against deterministic 3D frames. The battery lives in tests/e2e/ and runs through Playwright against a built VitePress site; it is also the regression gate for the 3D playground.

Reference implementations ​

DomainReferenceWhat it provides
CIF parsing, symmetry expansion, cell and densitygemmi 0.7.5Expanded P1 sites, cell parameters and the atom list used for the position comparison
Space group and primitive cellspglib 2.8.0International number and primitive volume
Lattice matrix, periodic neighbours, nanotubesASE 3.29.0Cartesian cell, shortest periodic distance and the (5, 5) tube geometry
Bragg anglesNumPyReciprocal metric, d spacing and Cu Kα 2θ positions
Visual framesCommitted PNG goldensPixel comparison of the deterministic canvas

The reference JSON is generated by tests/e2e/reference/generate.py and is committed; the browser never reads it.

What the battery exercises ​

The suite has 68 checks (49 from the first audit, plus the porosity, MOF and refinement-panel checks against Python, the Wyckoff letter checks against spglib for the eight samples, the PXRD intensity checks for Si, NaCl, TiO2 and Fe and the guided tour). For every sample in Si, NaCl, diamond, graphene, TiO2, MFI, MOF-5 and Fe it verifies:

  • formula, cell volume and density against gemmi;
  • P1 atom positions against gemmi modulo lattice translations (maximum error is reported in Ångström and required to be below 5 mÅ);
  • detected space group against spglib;
  • the 12 cell-edge segments against the ASE lattice;
  • the projected atom centres against an independent reimplementation of the camera used by the renderer;
  • the shortest perceived bond against the ASE periodic neighbour list (metal-metal contacts are intentionally absent from the bond graph).

It also covers:

  • 2 x 2 x 2 supercells (volume, atom count and scene geometry);
  • (1 1 1) slabs with two layers and 10 Å of vacuum (surface area, thickness, cell height, atom count and position range);
  • primitive cell, standardized cell, P1 expansion, asymmetric unit and Niggli reduction;
  • graphene, (5, 5) nanotubes, C60, C20 and a pcu MOF;
  • Cu Kα PXRD peak positions for Si, NaCl, TiO2, Fe and graphene;
  • relative Cu Kα PXRD intensities for Si, NaCl, TiO2 and Fe against |F|^2 m LP with gemmi's IT92 form factors (within 1 % of the strongest line);
  • the seventeen-step playground tour (zone counts, the tetrahedral angle, the NaCl supercell, the twisted-bilayer cell and every builder step);
  • Brillouin-zone vertex, face and edge counts for fcc silicon and rock salt, bcc iron, the graphene hexagonal prism and the orthorhombic MFI cell;
  • the playground tabs, Brillouin view, atom selection distances/angles/ torsions, nanoribbon/polymer/Schwarz P builders, powder plot, UFF energy, POSCAR export and fullscreen;
  • deterministic screenshots for eight samples, six operations and the space-fill style;
  • the cell-volume, CIF-inspector and document-editor demos.

Findings fixed by the audit ​

The battery exposed four real defects, all fixed in the same cycle:

  1. Supercells dropped the symmetry expansion. Structure::supercell copied the asymmetric sites, so xtalforge supercell Si.cif 2 2 2 wrote 8 atoms instead of 64 (and the WASM playground showed the same cell). The core now expands to P1 first through Structure::as_p1; the CLI atom-count estimate uses the P1 count and the CLI tests expect the full cell.
  2. is_p1 trusted only the Hall symbol. A CIF with an explicit _symmetry_equiv_pos_as_xyz loop but no space-group name was labelled P 1, so formula, density and charge ignored the operations. is_p1 now requires every operation to be the identity as well.
  3. The CIF-inspector demo shipped an incomplete silicon CIF. Its eight point operations had no centring translations, so the reader correctly produced one atom (Si, 0.29 g/cm³) while the page claimed silicon. The demo now uses the Hall symbol F 4d 2 3 -1d and reports Si8 and 2.329 g/cm³.
  4. The WASM surface could not feed a viewer. It only returned scene counts, summaries had no P1 atom count, and transforms/builders returned no structure text. The audit added scene_data, n_atoms and text-returning variants, plus Python scene_data and the CLI mesh --scene geometry dump, so the browser draws the same Rust scene instead of reimplementing geometry.
  5. The Brillouin zone of a conventional centred cell was a cube. The zone was built from structure.lattice(), so silicon (fcc), rock salt (fcc) and bcc iron all returned the reciprocal cell of the conventional cube instead of the primitive zone. brillouin_zone_for_structure now detects the primitive cell first, and the clipping uses an initial cube large enough for anisotropic cells (graphene returned a box instead of its hexagonal prism). The zone counts are now checked for fcc, bcc, simple cubic, orthorhombic and hexagonal lattices.

Functional audit of 2026-11-11 ​

A second pass ran every CLI subcommand on the samples and compared the results with independent calculations (gemmi IT92 structure factors, spglib, ASE neighbour lists and geometry checks). It found and fixed:

  1. PXRD relative intensities. The form factor was the Z-based Gaussian for every element; rutile (1 1 1) came out at 28 % of (1 1 0) instead of 19 % and silicon (2 2 0) at 77 % instead of 67 %. The International Tables (1992) coefficients are now embedded and the Si, NaCl, Fe and rutile lines agree with the independent |F|^2 m LP to 0.1 %.
  2. Twisted bilayers ignored the angle. build twist 2 1 21.79 returned an untwisted AA bilayer (any angle was rounded to 0 or 60 degrees). The builder now constructs the commensurate moire cell; every carbon keeps three 1.42 Angstrom neighbours in its layer.
  3. Polymers overlapped themselves. A six-unit polyethylene chain was packed into a 6.2 Angstrom cube, giving C-C contacts of 0.67 Angstrom. The chain is now periodic along a, joined head to tail.
  4. SMILES rings collapsed. The embedder had no angle term; benzene came out non-planar with 1.17 Angstrom 1-3 contacts. Hybridization-based angle targets fix rings, chains and alkynes.
  5. Disordered carbon contacts. build carbon produced carbons 0.15 Angstrom apart and terminations colliding across narrow gaps. The dual graph is relaxed to 1.42 Angstrom/120 degrees, terminations follow the free in-plane valence, overlapping terminations are dropped and every bond that crosses the cell boundary carries its lattice image (71 bonds of the default build were drawn across the whole cell).
  6. Playground defaults. The twisted-bilayer form called the builder with m = 0, which is rejected.
  7. Vibrations of crystals were nonsense. ff vibrations Si.cif reported modes of -13 847 and +16 528 cm^-1. The P1 expansion turned a coordinate of -1e-32 (a Cartesian round trip of 0) into exactly 1.0, which duplicated an atom in the UFF view (+1028 eV). Coordinates now stay in [0, 1) (P1Cell::expand, Lattice::wrap) and silicon has three zero acoustic modes and triply degenerate optical modes (347-749 cm^-1 with UFF).
  8. Le Bail and Pawley ignored the background. With a 50-count baseline Le Bail reached R = 0.69 even from the true cell, because the partition inflated the peaks to absorb the baseline. Both now subtract a SNIP envelope (about ten peak widths) and Le Bail uses the standard profile-weighted partition; R drops to 0.27 on the same data.
  9. MOF node orientation. The greedy rule left the in-plane angle of planar nodes arbitrary, so srs linkers were stretched 0.55-1.26 times. A Kabsch fit over every connection-to-edge assignment now orients the nodes.
  10. ZTC and Schwarz P carbons were not sp2. They are now the relaxed dual of the remeshed surface (see below); the sampled grid also gained the closing plane that the marching tetrahedra need for a closed surface.
  11. Wyckoff letters used moyo's origin. Silicon at 0, 0, 0 in origin choice 1 was reported as 8b. When the input basis is the conventional one (or a signed axis permutation in a cubic group) the letter is now matched on the input coordinates; all eight samples agree atom by atom with spglib.
  12. EEQ was QEq. eeq_charges reused the QEq table and returned the QEq charges. It is now EEQ2019 and matches the sci-form/dftd4 reference for water and methanol to 2e-3.
  13. Rietveld cell refinement was local. From a rutile cell 1 % off it stopped at Rwp 0.47 with c = 2.932 (true 2.989). A Pawley cell stage (pawley_cell) now runs first; the same start recovers a = 4.6399 and c = 2.9890 (Rwp 0.12).
  14. Incompatible MOF blocks were built. Two metal-free carboxylate blocks (benzene_tri with benzene) placed oxygens 0.6 Angstrom apart; the builder now refuses them with an invalid_input error.
  15. ZTC/Schwarz P below their minimum cell. Below 12 Angstrom (gyroid) or 8 Angstrom (Schwarz P) fewer than 90 % of the carbons are three-coordinate; the builders now refuse such cells, and still reject fewer than two samples (the new WASM suite run caught that regression).
  16. Surface differences. A field-by-field comparison of CLI, Python and WASM on the eight samples (scripts/check-surfaces.sh) found: eeq was missing from Python and WASM; the CLI info JSON had no n_atoms; twelve WASM errors were thrown as bare strings without Error.code; and the PXRD reflections with equal two-theta came out in a different order in WASM (last-bit differences in the sort key). All four are fixed and the 91 compared fields now agree.
  17. Periodic EEQ was not converged. The real-space Coulomb cutoff gave MOF-5 Zn at +4.4. EEQ now uses an Ewald sum for the Gaussian charges; a rutile 2 x 1 x 1 supercell repeats the unit-cell charges to 1e-4 and MOF-5 ranges from -0.58 to +0.79.
  18. The Rietveld phase fit ignored the background. fit_phase documents a scale and background fit but kept the background at its start value (0), so the automatic run reported Rwp 1.05 on the reference rutile pattern. Scale and background are now a weighted linear least-squares pair; the same fit gives Rwp 0.33 (the rest is the fixed width).
  19. Le Bail existed only in the CLI. Python powder_lebail and WASM powder_lebail_info were added, with a parity-matrix row; WASM gained powder_pawley_width_info because its Pawley could not refine the size and strain widths that Python can (Rwp 0.31 to 0.21 on rutile).
  20. CLI and bindings disagreed on Rietveld. The CLI forced 30 iterations where Python and WASM use the library default; it now uses the default and the three surfaces agree.
  21. The CLI panicked on a closed pipe. xtalforge ... | head printed a panic and a backtrace hint; it now exits quietly with status 141.
  22. The refinement panel showed the indexed cell as the Pawley cell. The automatic result exposed only the quick cubic-index edge as cell, and the panel printed the leading row's indexing candidate (a = 4.5902 Angstrom for rutile) while the Pawley fit had refined it. CLI, Python and WASM now return best_cell from the Pawley stage, falling back to the leading indexed cell when Pawley is skipped; rutile reports a = 4.5993 Angstrom, and the panel and CLI print it.
  23. Playground state after a transform. Running a builder or transform left the sample selector on a stale sample id; it now shows "Built or transformed". The nanotube tour step looks from +x so the tube lies across the view instead of end-on.

Verified as correct in the same pass: formula, volume and density of the eight samples; space groups; bond counts, ring census and Si-Si distances; C60, C20, graphene, ribbon and (5, 5) nanotube bond lengths; QEq/EQeq charge neutrality; supercells, slabs and the Brillouin zones; analytic UFF forces and the Gamma-point Hessian against central differences of the energy (crates/xtalforge-ff/tests/uff_gradient.rs); the general indexing of a synthetic rutile pattern (a = 4.6403, c = 2.9870 against 4.6400 and 2.9890); Pawley cell refinement from a 1 % expanded start (a = 4.6399); Rietveld from a near cell (a = 4.6399, c = 2.9890, Rwp 0.12) and the automatic powder run; MOF assembly of pcu/sql/Cr3O frameworks (C-C 1.35-1.48 Angstrom).

Known gaps ​

  • Polymers carry the heavy-atom backbone only; hydrogens are added with hydrogens afterwards.

  • The battery covers the structure, symmetry, builder, view and powder-position surfaces. scripts/check-surfaces.sh adds UFF energies, QEq/EQeq/EEQ charges, MFI porosity, the pcu/Zn4O/benzene MOF and Le Bail, Pawley, Rietveld and the automatic run on the reference rutile pattern, on the CLI, Python and WASM surfaces (97 fields). The browser runner checks the porosity panel, the MOF builder and the refinement panel (all four methods) against the Python results.

  • Wyckoff multiplicities are compared per atom with the spglib orbit sizes, divided by the centring factor when a sample is stored in its primitive cell (MOF-5); the earlier MOF-5 exclusion is gone.

  • Screenshot goldens are renderer- and platform-specific. They catch changes in the scene, the camera and the layout, not differences between browser GPUs; the numeric projection check is the platform-independent evidence for the 3D geometry.

  • The WASM bundle measured 3943 KiB against a 3968 KiB budget at 1.1.0. The Phase 9 release measures 4603 KiB against the 4736 KiB budget set by this release; the earlier 3904 KiB budget was raised by the 2026-11-11 audit for the fixes listed above; any further surface helper needs a deliberate size review.

  • The CLI, Python and WASM surfaces have their own integration suites; the Playwright battery targets the WASM surface because it is the one a browser user sees. scripts/check-parity.sh keeps the four surfaces aligned.

Run it ​

bash
scripts/e2e.sh                  # rebuild WASM + docs and run all checks
scripts/e2e.sh --skip-build     # reuse the current dist
scripts/e2e.sh --update-goldens # review and accept new screenshots

Captured PNGs and the assertion output are written to tests/e2e/test-results/, which is ignored by git. The committed goldens live in tests/e2e/golden/.