PyTyche

The high-level API covers the design, execution, and analysis of adaptive experiments aimed at discovering stable segments of a test population with heterogeneous treatment response. Operators, PMs, and agentic systems alike can run robust experimentation loops end to end without hand-rolling the statistics—moving the question from "does this work" to "who does this work for, and why."

PyTyche is the optimized extraction of real experimentation programs I've run and developed over many years and across companies—programs responsible for delivering millions of dollars of incremental revenue across a range of settings. Batched adaptive experimentation delivers most of the live-optimization wins of an online bandit at a fraction of the operational cost, and the segments it finds are robust to drift and targetable across the whole business—not just the moment and regime they were learned in. This is the system behind programs that have consistently delivered double-digit lift across a range of verticals.

Underneath is a novel Bayesian causal forest implementation, inspired by the shared-forest construction of Linero, Sinha & Lipsitz (2020), with a hand-rolled Python/JAX MCMC sampler built for easily available commercial GPUs—not a cluster.

BART-family models, BCF included, have a well-known problem at large n: posteriors over-concentrate, and coverage quietly degrades. PyTyche ships a previously unpublished empirical survey of the shapes this miscalibration takes across diverse data-generating processes and scales, along with default and customizable corrections for your specific data.

The sampler's speed is what makes this practical: simulation-based calibration requires many full fits, which was prohibitive at 10–12 hours each and is routine at 20 minutes.

SBC corrections treat the symptom. Two directions in the library target the root cause—samplers that explore the actual posterior instead of getting stuck in it:

• rotate/perturb — an implementation of the Pratola (2016) tree operators alongside the standard Grow/Prune moves, letting tree structures escape local modes and represent the full posterior uncertainty.
• particle Gibbs — a particle-Gibbs sampler to replace the default Metropolis-Hastings used in most BART fitting. PyMC's PGBART is the closest equivalent, but isn't optimized for the target scale of several million users on consumer hardware.

The first public release (v0.2) lands Friday, June 12. PyTyche began life inside the Stoa Stack and spun out as a standalone library—Stoa remains its end-to-end production implementation, from assignment and serving through feature generation.

The project has also served as a working test bed for agentic-assisted development of statistical software—an experience that shifted my own practice toward TDD and spec- and doc-driven development.

Related: TycheJS, the browser-native side of the same project—conjugate priors, variational inference, and causal tree discovery running entirely client-side.