Metering and Performance Analysis on Lisp Machines

MIT CADR System 46

The historical source is the public System 46 snapshot at Git revision 8e978d7d1704096a63edd4386a3b8326a2e584af.

The recovered public System 46 on-line help contains the two Qmisc docstrings but no operator documentation for LMETER or PTRAC. That omission is consistent with two contemporary manual-planning notes: one says to add LMETER to the programs chapter, and another lists “metering features” as an unwritten system topic. ptrace.notes likewise ends with an unfinished request for a brief document.

Maintained LM-3 System 303

The later public source is the maintained LM-3 System repository at Fossil check-in 4df393c68d7f083ce42d5c377039d26043cc18a9031ace28258dc97f4137eb91, tag system-303. This is maintained restoration work, not silently treated as an untouched historical distribution.

The current files were independently compared byte-for-byte with the named Fossil check-in. The maintained microcode sources were also inspected where a rough documentation claim conflicted with meter.lisp.

Licensed Genera 8.5 material

The Genera source observations use the locally purchased Open Genera archive and the Genera 8.5 world it supplies. The source, world, and decoded help remain ignored and are not linked or redistributed. The descriptions below are original evidence summaries.

The local system directories identify both METERING and METERING-INTERFACE component patch levels as 444. Recovered installed help is from documentation system 440, so a difference between help and source is reported as a versioned difference, not silently treated as a typo.

The public manual cross-check is the Metering Interface and PC Metering material in Symbolics's Program Development Utilities, Genera 8. The optional-system loading instruction is also present in the Genera 8.3 Software Installation Guide for the NXP1000. Both were verified 2026-07-18.

Raw meters are not the later Meter application

The name collision persists into System 303. Its maintained subsystem manual explicitly distinguishes the READ-METER/WRITE-METER microcode counters from the Lisp metering tools. LMETER uses those counters directly; LM-3 METER instead records event frames. Calling both simply “the meter” obscures a real architecture change.

Record layout and lifecycle

PAGE-TRACE-ON allocates a wired, static ART-32B array, by default large enough for 2,000 four-word records. Each record contains:

1. a clock sample;
2. the referenced virtual address;
3. flags for swap-out, stack-group switch, transport, scavenging, and asynchronous activity, together with a low micro-PC field; and
4. a sampled current function taken from the machine's M-AP register.

The recorder validates that loaded microcode symbols match the running microcode version, clears the array, wires it, and enables the %PAGE-TRACE microcode hook. PAGE-TRACE-OFF disables the hook, unwires the array, and releases the GC arrest. GC is arrested because collection or flipping could change address and region meanings; the source also warns that flipping invalidates a trace.

The “current function” is explicitly heuristic: the microcode samples M-AP at the page event and hopes it is representative. The decoder stores area and space names, function, and micro-PC symbol information in list records before a later flip can invalidate those interpretations.

Complete user-oriented PTRAC surface

The textual trace heading says that entries are newest first and identifies the columns as delta milliseconds, address, flags, micro-PC, and macro function. Each decoded line can show the area and region space type plus OUT, SCAV, TRAN, and ASYN flags.

Two shortcuts are pushed into TV:*ESCAPE-KEYS*:

• the character named HAND-UP calls PAGE-TRACE-ON; and
• HAND-DOWN sends the current trace to an editor buffer named PAGE TRACE.

These are entries in the historical TV escape-key table. Without a compatible System 46 runtime, this page does not guess how a particular modern keyboard mapping labels those characters.

Source-visible limits and unfinished work

• A proposed 512-by-512 scatter plot of space interval against time interval is present only as commented code in revision 57. It is not a shipped command.
• ptrace.notes says the filters, evaluated address fields, micro-PC timing investigation, and GC arrest were completed, but a brief user document was still outstanding.
• Revision 57 improves wiring, adds more region-space decodes, handles the special free-memory address, and reformats output relative to revision 54. It comments out the scatter plot that was live in revision 54.
• Several simulations describe their own approximation limits. For example, the group-swap model does not fully model memory residency or induced paging, and the deferred-cache simulation omits some verification references.

Recording controls and event mask

SYS:%METER-MICRO-ENABLES uses bit 1 for page faults, 2 for consing, 4 for function entry/exit, and 8 for stack-group switches. The default octal 14 therefore records function boundaries and stack switches. Stack-switch records are essential to avoid charging time spent in another stack group to the function that happened to be active before the switch.

The maintained microcode implements page-in and page-out recording when bit 1 is enabled. A cons event number and analyzer slot exist, but meter.lisp marks the cons event unimplemented and no corresponding microcode recording routine was found.

Complete METER:ANALYZE option and report surface

General keywords are:

The :TREE analyzer accepts:

The three tree reports answer different questions:

• TREE-PRINT prints every reconstructed call-tree node with indentation, function, own real/run time and faults, and descendant real/run time and faults.
• SUMMARIZE-TREE merges nodes by function and prints Functions, # calls, Run T, Faults, and Real T, followed by totals.
• TREE-FIND-CALLERS prints caller, callee, and call count for the requested callees.

:LIST-EVENTS prints one row per event with real time, run time, cumulative page faults, stack-group name, current function, stack depth, event kind, and event-specific data. Function records show calls and returns; stack-switch records name the old stack group; page records show page-in/page-out, address, area, and decoded microcode position.

Documentation drift is evidence

The older rough doc/meter.text calls the analyzer option :FILTER; the maintained source parser and formal manual call it :ANALYZER. Passing :FILTER to the inspected implementation does not select an analyzer: it falls through into analyzer-specific options and is rejected. The rough note also says page-fault recording is unimplemented, whereas the maintained microcode contains explicit page-in and page-out recording paths. The consing statement remains accurate.

This is an example of why the manual layers cannot be collapsed. The rough note captures an earlier state and design intent; the maintained source and formal manual describe a later interface.

1. The older PC Meter

PC Meter predates the Metering Interface, which the installed documentation dates to Genera 7.2. It statistically samples the current macro-PC into a wired array of address-range buckets. On 3600-family hardware the sampling source can be an audio microtask or a FEP task; on Ivory/IMach the inspected source uses a periodic interrupt handler. Samples outside the configured address range are counted as misses.

The recovered dictionary documents the following complete operation set:

REPORT first prints total, ignored, and listed sample counts and the percentage listed. Each subsequent row identifies a bucket, percentage, and either one function or a truncated indication of multiple functions.

Source adds user-relevant operations omitted from the recovered dictionary: UNMAKE-PC-ARRAY, PC-METERING-SUPPORTED-P, sampling-rate query/setters, FUNCTION-LIST with a threshold, and ESCAPE-MONITOR. The apparent escape-key registration for ESCAPE-MONITOR is commented out, so it is not counted as a live key binding.

2. The optional Metering Substrate

METERING-SUBSTRATE is a general instrumentation framework, not a screen. Its system declaration is marked optional and not advertised. The module order was deliberately flattened because the source says the system-construction tool had dependency bugs; that comment is restoration-relevant evidence, not a claim that the conceptual architecture is random.

The architecture separates:

• triggers, which define observable events or boundaries;
• regions, which pair an entry and exit and can compute differences;
• controls, which decide scope and when instrumentation is switched on;
• generators, which create values at a trigger;
• collectors, including records, histograms, and backtraces, which retain generated values; and
• generated/inserted/wirable code, which installs fast instrumentation into function and machine paths.

The command layer exposes composition operations rather than a fixed profiler: attach a collector or generator to a trigger, name generated values, filter events, meter conditionally, set a control's scope/switch, turn a function into a metering region, remove function metering, display/reset collectors, show active metering state, and reset or disable the utility. The Metering Interface uses this substrate to construct its standard run types.

Function-call metering calibrates entry and exit overhead. The first interface selection computes repeated “fudge factors” so this overhead can be subtracted and an error estimate displayed. The source default on VLM is to ignore inconsistent trials rather than prompt, while the recovered general documentation describes the interactive Retry once, Retry, and Ignore choices. That machine-conditional source behavior is another reason not to generalize hardware instructions to Open Genera without qualification.

3. The optional Metering Interface

Loading system METERING loads METERING-SUBSTRATE and then the Dynamic Windows interface. The site system file deliberately maps the name METERING to the interface system, while METERING-SUBSTRATE maps to the lower layer. After loading, the program's Select key is %.

The source layout has a title, an interactor, a command menu, a History of Metering Runs pane, a separate field-header pane, and a Metering Results pane. Runs remain in the Lisp world until explicitly deleted or a cold boot. The six top command-menu items, in source order, are:

Complete Meter Form arguments

The documentation-440 command page does not list :SAMPLING-INTERVAL, but the inspected 8.5 source accepts it. It is therefore labeled source-visible in the later component rather than retroactively inserted into the older dictionary.

Before the collected trial, the implementation executes the form COUNT - 1 times. It compiles forms whose operator or arguments cannot safely be applied directly and asks whether to continue if compilation produced warnings.

Complete Meter in Process arguments and controls

With Function keys, Function-( starts and Function-) finishes the run. Mode Lock can gate either start/stop scheme; with Function keys both conditions must be true. With the default scheme, the current source presents a completion choice in the Metering Interface. Installed help additionally says the END key ends the run. Runtime verification of the exact Open Genera key path remains pending below.

Starting or stopping in the middle of a function has visible consequences. A Call Tree run can show multiple apparent roots because the real root was entered before recording. A Function Call run counts only completed entry/exit pairs; an incomplete final invocation can be invisible as such after earlier completed calls have already contributed data.

Keyboard accelerators and asynchronous controls

Mouse gestures by presentation

There is a direct help/source disagreement. Documentation system 440 assigns Show All Node Descendants to s-Left. The component-444 source assigns that command s-c-Left and uses s-Left for the undocumented thresholded ladder. Both facts are preserved; runtime must determine which bindings are installed in the specific world used by the harness.

Complete major fields and visible subfield families

Time is wall-clock time and can include other processes. Process Time counts only time in the metered process. Incl includes callees or descendants; Excl does not. Slash-prefixed subfields are bar graphs, while percent-prefixed subfields are numeric percentages. RAvg is the average of the main histogram mode, a heuristic noise filter rather than a guaranteed corrected value.

Complete display-parameter choices

The 12 Page Fault event-type choices are Replace, Write, Fetch, Prefetch, Prefetch Mark, Fetch Load, Prefetch Load, Prefetch Load Mark, Create Page, Flushable Page Fault, Prefetched Page Fault, and Flush.

Call-tree filtering ORs the two node thresholds and function-name matches, then ANDs that result with maximum depth. A displayed matching descendant retains its calling path. Hoisting changes the display root without losing the run's whole-run totals; /Root and %Root subfields make the distinction explicit.

Measurement caveats represented by the interface

• Percentages can exceed 100 when overlapping inclusive values are summed or a denominator represents a different scope.
• Deterministic entry/exit metering adds substantial overhead; calibrated fudge factors and Error subfields estimate, but cannot eliminate, uncertainty.
• Statistical results require enough samples and are not expected to be exactly repeatable.
• Statistical Function Call and Statistical Call Tree sampling runs in another process, so it cannot sample code while scheduling is inhibited. Statistical Program Counter sampling is lower-level and can observe such code.
• Metering code and collectors enlarge the working set, so the act of measuring can increase paging.
• Paging-system time is based on a coarsely ticking clock; small sample counts have correspondingly large uncertainty.
• “Within Functions” uses encapsulation. On very short runs, its own boundary functions and several hundred microseconds of overhead can become visible.

Genera runtime observation: optional system not resident

A fresh isolated session named d15-metering-genera-20260718, generation 1, booted the known Genera 8.5 VLOD and reached Dynamic Lisp Listener 1. The exact command intent Load System Metering was dispatched there. The Command Processor said that METERING was unknown, searched the site pathname DIS-SYS-HOST:>sys>site>metering.system.newest, and requested a login. This is positive runtime evidence that the optional system was neither known to this fresh world nor reachable through its unconfigured site/file-service context; it is not evidence that the purchased archive lacks the source, which the static audit above directly inspected. A subsequent Select-% attempt did not open a Metering Interface frame.

No host file service, site configuration, or credential was added merely to make the screen reachable. Consequently the six menu items, live field layout, fudge-factor interaction, and the disputed Super-Left bindings remain unverified in this world. The ignored capture build/genera-computer-use/d15-metering-genera-20260718/screenshots/0002-load-metering.png records the minimal blocker at 1,200 by 900 pixels (PNG SHA-256 3c9a5c3c24363f7700adf37cd1de0f31ede50c6e0fa2506e9391793b624ae98b, pixel SHA-256 b1dac302e707cef17c2eed22a32bda4385ce43515d9d0781c9708296bfdb458d). It is not curated into the museum: it shows a failed load in the Listener, not the visible application this article is about. This explicit blocker therefore stands in place of an application screenshot under the repository policy.

The generation record retains the full harness provenance. In portable form:

The one-shot RFC 868 responder also completed successfully, with responder evidence SHA-256 095a9399505c0f38ec9df869e2f1227e8d9e3b0d7551c2a80ac2fe5ee5739645 and completion-record SHA-256 4cb87fe34076cc7d91b468e722fabb44ca82f3892220a29cff85b3b82e84b792.

TODO: after a separately authorized, isolated, and reproducible way to expose the licensed Metering system to the private world exists, load it, select %, wait for fudge-factor work to settle, verify the six-item command menu and the actual Super-Left/Super-Control-Left call-tree bindings, and capture one sparse initial interface state. Review that exact image and use under the repository's screenshot policy before embedding it. Do not capture a full Help chapter or a bulk command sequence.

TODO: when the CADR harness is available, inspect the System 303 disk label for METR. If present, run a harmless small form through METER:TEST, display both summary and :LIST-EVENTS output, resume GC, and capture only the minimal report needed to establish actual behavior. If absent, record that exact runtime blocker rather than mutating the preservation disk layout.