Articles

Product teams can use counterfactual estimation to answer retrospective questions about the effects of AI prompt engineering, despite the lack of random treatment assignment in logged production data.

Product experimentation teams often struggle with causal inference on LLM-based features due to the opt-in trap, where users who click to opt in differ systematically from those who don't. Propensity score methods can help reweight comparison groups to approximate a randomized experiment.

When measuring model quality, standard regression approaches can be flawed due to confounding variables. This tutorial shows how to use instrumental variables to extract a clean causal estimate from LLM routing decisions in Python.

Standard product experiments answer a binary question about average efficacy, but uplift modeling turns that binary into a nuanced spectrum, estimating a conditional average treatment effect for each user based on their specific features.

Discover how to avoid p-hacking and stop early in product experiments using the mixture Sequential Probability Ratio Test (mSPRT) and sequential testing. This tutorial provides a step-by-step guide on implementing mSPRT in Python and applying it to a synthetic LLM product dataset.

Collaborator contamination occurs when users in a control group are influenced by treated users in a collaborative workspace, leading to biased results in product experiments. Cluster randomization is a technique that can help overcome this issue by flipping the coin at the workspace level.

The Global Rollout Problem occurs when a team ships a model upgrade to the entire user base simultaneously, eliminating the control group. Synthetic control is a solution that builds a weighted combination of untreated units to estimate causal effects.

The 'any' type in TypeScript may seem like a convenient solution, but it can actually break the safety net you installed the language to get.

TypeScript builds on JavaScript's runtime methods and adds its own compile-time utilities. Learn the essential methods that every developer should know.

You've seen decorators in Angular, but do you know what happens behind the scenes? Decorators are a powerful TypeScript feature that can transform your code when used effectively.

Discover the world of TypeScript, a JavaScript framework that adds superpowers to your coding skills. Learn how to get started with TypeScript and understand its features.

Discover how to improve error handling in your Dart codebase with a complete, modern approach, including records, result types, monads, and freezed exceptions.