This notebook is a quick demo of how to how to measure jailbreak robustness. I plan to build off this post in future posts.

In this post I want to make a few points about \(r\), the Pearson correlation coefficient, and \(R^2\), the coefficient of determination. They’re so widely used that I think some of the implicit assumptions behind them can become lost.

Many robot arms use 6 or 7 actuators to control where the hand ends up. This might seem like overkill for positioning a hand in 3D space, but in this post I want to dig into the math behind how robot arms work.

This post provides a quick guide on the basics of how to work with Amazon CloudWatch and, in particular, CloudWatch Logs Insights. CloudWatch Logs Insights is what lets you run SQL-like queries over your logs.

In this post, I’ll talk about how to extract utility functions from a large language model (LLM). This works by generating pairwise preferences, then fitting a Thurstonian model to the data. This approach allows us to quantify how much LLMs value different outcomes. In addition, we can give the LLMs probabilistic questions in the form of lotteries (an X% chance of A or a Y% chance of B) and use that to help us quantify the LLM’s preferences. This post uses the techniques from the paper Utility Engineering by Mazeika et al.

This post shows how Bayesian updating works when we observe new evidence. We compare four different prior beliefs about the probability of success of some event. Priors vary along two dimensions: Location – what probability do we believe a priori? Strength – how strongly do we believe it? (number of pseudo-observations encoded in the Beta prior)

DeepSeek-R1 is an interesting model. It’s generally quite willing to discuss a range of topics, although because it’s a thinking model, the outputs can be a little harder to get into a structured format. In this post, I show how to use langchain and Pydantic to get structured output from DeepSeek-R1.