Testing

The unit tests cover the engineering computations, checked against the books' worked examples: the tolerance stack-up (Fischer’s pin and ground plate, Drake’s MRSS factor, Cogorno’s fastener and boundary formulas), ISO 286 limits and fits (against the ISO 286-1 formulas and published values) and ISO 2768, the PMI semantics (ISO 8015 and ASME Y14.5 checks, inspection plan, gauge sizes), the release package manifest and its verification, the clearance classification, the AP242 mapping of the PMI (and the correction of OpenCASCADE’s IT grade), sheet metal bend allowance and flat length (Shih’s actuator bracket), pin location (Drake ch. 24, Fischer’s lever), composite position frames and the pattern evaluation (its fit checked against a brute-force search), measurement capability (Drake’s gauge study example, 42.6 %; TUR and ISO 14253-1 acceptance limits), and the title block values and revision block. They cover the selection rules too: the scope verdict on occurrence paths (a look-alike sibling outside the work part is refused), collector counts and refusal reasons, the tangent and connected chains, and the select-through choice. They cover the command controller’s decisions as well (nxstyle/command_core.py). Typed lengths are read with their units and the field’s decimal separator; 3.175 in a field whose separator is a comma is refused, not guessed. Every Extrude Boolean is either built (None for a new body, Unite, Subtract) or refused with a reason (Intersect), and the Boolean NX would infer is checked. So are Revolve’s: its Boolean, its axis (a vector through the origin is the Body’s axis, elsewhere a datum line) and its Start and End angles, refused where FreeCAD cannot revolve them. A section’s picked curves must close: the impeller casing’s three loops do, a loop with a curve missing is open, and the whole casing branches. Picked geometry is matched to the sketch shape’s edge names, which follow another order. They cover ISO 10303 too: Part 21 strings, reals, binaries, comments, line breaks inside tokens and several data sections (with the standard’s own examples), the EXPRESS reader and the checker (attribute order of internal mapping, * for re-derived attributes, ONEOF, abstract entities, SELECT and aggregate values), the bundled dictionary against ISO 10303-108 Cor.1 annex F, and a parametric model written, checked without an error, read back and compared, including rules that must catch broken data; a Part 112 sketch of every kind of command evaluated against hand-computed geometry, and sketches written as Part 112 commands that reproduce them; lengths and angles in inches and degrees; the application protocol named in a header; and the product structure of an assembly with its quantities. The EXPRESS interpreter has its own tests: the operators and built-in functions with indeterminate values, a schema FUNCTION with loops and a SET local, derived and inverse attributes, SELECT types in TYPEOF, the complex entity constructor, and a small schema whose every kind of rule (an entity’s WHERE rule, a defined type’s, an aggregate bound given by an attribute, UNIQUE and a global RULE) is broken once and met once, with a rule left undecided when an unknown entity may use the instance. In a real FreeCAD, freecadcmd tests/fc_parametric_step.py exports a constrained, parameter-driven plate and a CSG part as STEP with parameters, imports them into new documents and compares the solids, before and after the same parameter change (NXSTYLE_OUT keeps the files). They cover the Carbon client (Manufacturing with Carbon): every flow against a stand-in Carbon server on this computer that checks each request against Carbon’s published OpenAPI schemas, with a negative control (a request the schema rejects is refused); publishing a part with its model, documents and inspection plan, an assembly’s BOM from its STEP, a second publish that writes nothing, a review that writes nothing, a BOM change on an Active method going to a new Draft, a new revision created inactive, an unreleased or tampered package refused, the API key’s sources and that it is never printed. In a real FreeCAD, freecadcmd tests/fc_carbon.py builds an assembly of links to other documents (a link array among them), reads its BOM, makes its release package and publishes it to the stand-in server. They also cover the launcher: the profile folders and their reset, refusing a profile inside or above the normal FreeCAD folder, the environment and command it builds, which Strata, Katydid, SheetMetal, LCInterlocking and DFM it loads (DFM only with its OCP, whose folder goes first, installed by a pinned --no-deps pip command) and where it reads nxstyle.json, that every application’s ribbon has the Application tab, that the Laser Cut ribbon names only commands LCInterlocking registers, and the memory ceiling (a runaway is stopped, a second concurrent run refused). FreeCAD is not needed. GitHub Actions runs both, and checks that the ribbon files are valid JSON, on every push (Windows and Ubuntu).

This starts FreeCAD with NX-Style in a throwaway profile and drives the real UI with synthetic mouse and key events. It ends with the count of checks passed, failed and skipped (171 checks on FreeCAD 1.1.4 with Strata, Katydid, SheetMetal and LCInterlocking, 2026-10-27). Among them:

A check that needs something this FreeCAD build lacks (pythonocc for AP242, the FEM workbench, QtTest) is counted as skipped, never as passed; --require-all makes a skipped check fail the run. It writes out/smoke.json and screenshots. While developing, NXSTYLE_SMOKE_ONLY=s_move,s_thread (step names from `smoke.py’s plan) runs only those steps after Strata’s start; a full run is still the check.

A result file counts only when it is this run’s and the run finished. The launcher passes each run a new ID, and the result (written atomically by nxstyle/harness_result.py) must carry that ID, the schema nxstyle-harness/1, status complete, and every planned step listed as run. A tutorial that fails part way lists the steps it then skipped. So a passing smoke.json left in the folder by an earlier run cannot pass a run that crashed, and neither can {}, empty lists, a truncated file or a run that stopped half way. A FreeCAD that exits with an error after writing a good result fails too. tests/test_harness_result.py covers each of these, through launch_nx.main with the FreeCAD run replaced by a stub.

The replay works through the tutorials the way the books tell an NX user to: the same keys, the NX-named ribbon buttons, MB1 clicks on sketch points in the graphics window, MB2 to end tools and accept dialogs. On the Katydid canvas (tutorials O to R) it double-clicks the empty canvas and types each component’s name, drags wires from grip to grip (Shift adds), sets Graft, Flatten and input values from the grips' right-click menu, and types slider values into the slider’s dialog; after each step it saves a screenshot with the floating canvas laid over the lower half of the graphics window. Dimension values are applied through FreeCAD’s API, the equivalent of typing them into Rapid Dimension; everything else is real UI input.

Each feature is checked by its volume against the same operation done directly with FreeCAD’s Part API. Volume alone does not prove the shape: a slot cut in the wrong place has the same volume. Tutorial A therefore also compares the part with a reference solid built with the Part API: their symmetric difference and their bounding boxes must agree (nxstyle/geomcheck.py). The boxes are exact ones (optimalBoundingBox without the display mesh): Shape.BoundBox is the box of the mesh once the 3D view has drawn a curved solid, 0.02 to 0.09 mm off for a disc of R60. The final part must also be drawn in the graphics window. That is an occupancy check (something is drawn), not a check of the image. Save steps press the real Ctrl+S and check that the file was rewritten; only a new part’s first file name is given by script, because the replay cannot drive the system file dialog. Before the key the part’s Comment gets a marker unique to that save; the step passes when the file on disk holds the marker and the part is unmodified. The file’s time is no witness: FreeCAD saves to a temporary file and renames it over the old one, and in replays on Windows the new file kept the replaced file’s last-write time in 4 of 10 saves, with the change in it. Tutorials S to X check engineering results: the stack-up against Fischer’s figures, the fit against ISO 286, the STEP file’s PMI and what OpenCASCADE reads back, the inherited dimensions on the sheet, the clearance kinds and penetration depth, the release states in Strata and the release packages against their manifests, the flat pattern against Shih’s hand calculation, and the pin location against Drake’s formulas. Dialogs that Strata opens are answered by the replay the way a user fills them in. Tutorials AA to AC check the composite frame’s plan rows and gauges, the evaluation of measured holes, the TUR and acceptance limits, Drake’s gauge study result, and the title block against the Strata release. Tutorial AE is the acceptance case for the command controller (Extrude). It does New, Sketch, then Extrude four times through the real UI: None (a new body), Unite, Subtract chosen in the Boolean list, and a separate new body. It also checks what must not happen:

Then it edits the first extrude. From the Part Navigator’s MB3 menu it previews a new length and presses Cancel. The value must come back, with no undo step added and nothing left to recompute. It edits again from the ribbon and presses OK, which must add one undo step. Ctrl+Z twice takes back the edit and then the whole New Body extrude, and Ctrl+Y twice restores both. Extrude (Subtract) is then started from the Command Finder and cancelled at the plane pick, which must leave no object and no undo step. Finally it saves, reopens and edits again, and Esc cancels. The radial menu’s Edit Parameters runs the same command as the ribbon’s, but the replay does not drive the radial menu itself. Every solid is compared with nxstyle/geomcheck.py, and every command must add exactly one undo step. --tutorials G,H picks some; N runs M first, V runs T first, AC and BC run T and V first. A run may take 5 minutes plus 5 per tutorial chosen before it counts as timed out; the canvas tutorials O to R take about 20 minutes together.

The replay writes out/tutorial.json, a progress.log and screenshots. Exit code 1 means a step FAILED. The same, differs and missing outcomes are the replay’s own judgement of each step against the book; they are not an independently measured equivalence with NX.

On Windows both runs use a hidden desktop, so nothing appears on your screen; --visible shows FreeCAD instead.

Every run is capped at 4 GB of memory (--max-memory-gb) inside a Windows job object. FreeCAD’s allocations beyond the ceiling are refused and the run ends with exit code 4, so a runaway cannot take the machine down. Run one at a time.

Each run uses a fresh throwaway profile. To test what a second start sees, give both runs the same --profile DIR; it is kept between runs.

The tutorial pages' step tables come from docs/results/tutorials.json: a replay’s steps and the FreeCAD version, nothing else. After a full replay that should be published:

It refuses results that name a folder in your home directory, since they are published. python tools/tutorial_table.py out/tutorial.json prints the result counts per tutorial and updates the table in docs/TUTORIAL-REPLAY.md. See Writing these docs for building the site.