Articles

2024

• 
  Eradicating N+1s: The Two-phase Data Load and Render Pattern in Go May 28, 2024

  Using a two-phase data load and render pattern to prevent N+1 queries in a generalized way. Especially useful in Go, but applicable in any language.

• The Notifier Pattern for Applications That Use Postgres May 6, 2024

  Maximizing Postgres connection economy by using a single connection per program to receive and distribute all listen/notify notifications.

• Web APIs: Enriched DX By Disallowing Unknown Fields May 5, 2024

  Using Go’s DisallowUnknownFields option to improve an API’s integration experience by making paramter naming mistakes faster to resolve.

2023

• 
  River: a Fast, Robust Job Queue for Go + Postgres Nov 20, 2023

  Introducing River, a Postgres-based job queue designed for resilience and correctness through strong transactional guarantees.

• On Using Go's `t.Parallel()` Aug 26, 2023

  How and why we started annotating all our Go tests with t.Parallel(), and why you might want to consider doing so too.

2022

• Soft Deletion Probably Isn't Worth It Jul 19, 2022

  The traditional soft deletion pattern using deleted_at columns leaks into code, curbs the usefulness of foreign keys, and makes data removal difficult. Luckily, there’s an alternative.

2021

• 
  Postgres: Boundless `text` and Back Again Sep 10, 2021

  The text type in Postgres, why it’s awesome, and why you might want to use varchar anyway. Also, a story about trying to get string parameters bounded at Stripe.

• How We Went All In on sqlc/pgx for Postgres + Go Sep 8, 2021

  Touring the ORM and Postgres landscape in Go, and why sqlc is today’s top pick.

• 
  Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL Feb 16, 2021

  A privacy and GDPR-friendly approach to producing basic and complex site analytics, and one which is more accurate in an age when adblockers reign supreme.

2020

• Feature Casualties of Large Databases Dec 1, 2020

  Databases shed important RDMS features as they get big. Examining why this tends to be the case, and some ideas for preventing it.

2019

• Doubling the Sorting Speed of Postgres Network Types with Abbreviated Keys Aug 7, 2019

  Making the sorting speed of network types in Postgres twice as fast by designing SortSupport abbreviated keys compatible with their existing sort semantics.

• 
  Sequences: A Modest, Contra-garden Travel Project Jun 4, 2019

  In the spirit of the continued production of independent content, I’m running a two-week photography and writing project in Berlin called Sequences.

• Building a Robust Live Reloader with WebSockets and Go May 28, 2019

  A walkthrough of the design of a live reload feature for the static site generator that builds this site, touching on fsnotify, WebSockets, and the curious case of file 4913.

• 
  SortSupport: Sorting in Postgres at Speed Feb 4, 2019

  How Postgres makes sorting really fast by comparing small, memory-friendly abbreviated keys as proxies for arbitrarily large values on the heap.

2018

• How to Manage Connections Efficiently in Postgres, or Any Database Oct 15, 2018

  Hitting the limit for maximum allowed connections is a common operational problem in Postgres. Here we look at a few techniques for managing connections and making efficient use of those that are available.

• A Missing Link in Postgres 11: Fast Column Creation with Defaults Aug 28, 2018

  How a seemingly minor enhancement in Postgres 11 fills one of the system’s biggest operational holes.

• 
  Tweeting for 10,000 Years: An Experiment in Autonomous Software Jul 20, 2018

  Using the design principles of a 10,000 year clock to build a program that’s intended to run on a macro timescale.

• 
  Living APIs, and the Case for GraphQL Jun 8, 2018

  Why it makes sense to model APIs as graphs, and what GraphQL can do for us to help with discovery, batch operations, and gradual enhancement.

• 
  Scaling a High-traffic Rate Limiting Stack with Redis Cluster Apr 26, 2018

  Flattening a single very hot vertical Redis node into a horizontal Redis Cluster at Stripe.

• 
  Touring a Fast, Safe, and Complete(ish) Web Service in Rust Mar 27, 2018

  A detailed look at the frameworks, concurrency model, error handling, middleware constructs, and testing strategies of a web service written in Rust.

• 
  Speed and Stability: Why Go is a Great Fit for Lambda Jan 17, 2018

  Why Go’s stability and simple deployments is a good fit for a serverless environment.

2017

• 
  Scaling Postgres with Read Replicas & Using WAL to Counter Stale Reads Nov 17, 2017

  Scaling out operation with read replicas and avoiding the downside of stale reads by observing replication progress.

• 
  Redis Streams and the Unified Log Nov 8, 2017

  Building a log-based architecture that’s fast, efficient, and resilient on the new stream data structure in Redis.

• 
  Implementing Stripe-like Idempotency Keys in Postgres Oct 27, 2017

  Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.

• 
  Should You Build a Webhooks API? Sep 28, 2017

  When it comes to streaming APIs, there’s now a lot of great options like SSE, GraphQL subscriptions, and GRPC streams. Let’s examine whether webhooks are still a good choice in 2017.

• Transactionally Staged Job Drains in Postgres Sep 20, 2017

  Building a robust background worker system that leverages transactional isolation to never work a job too early, and transactional durability to never let one drop.

• Using Atomic Transactions to Power an Idempotent API Sep 6, 2017

  Part one of a series on getting easy data correctness by building APIs on the transactional machinery of Postgres.

• The Limits of Copy-on-write: How Ruby Allocates Memory Aug 28, 2017

  Why Ruby’s scheme for memory allocation doesn’t play nicely with copy-on-write, and how a compacting garbage collector will help.

• 
  How Postgres Makes Transactions Atomic Aug 16, 2017

  A dive into the mechanics that allow Postgres to provide strong atomic guarantees despite the chaotic entropy of production.

• 
  Pseudo-HTML and Pidgin CSS: Building an Email Newsletter Aug 2, 2017

  Building a toolchain for sending a newsletter, and the dismal state of HTML and CSS in email.

• The Accidental Evangelist May 28, 2017

  Lets talk about the bad aspects of the technology we use as well as the good. Better information will lead to better decisions in the next generation of technical architecture.

• A Comparison of Advanced, Modern Cloud Databases May 22, 2017

  A non-exhaustive primer of modern cloud database solutions like Aurora, Cosmos, and Spanner.

• 
  Building Robust Systems with ACID and Constraints May 16, 2017

  On ensuring system integrity, operability, and correctness through a solid foundational database, and how ACID transactions and strong constraints work in your favor. Why to prefer Postgres over MongoDB.

• 
  In Pursuit of Production Minimalism May 10, 2017

  Practicing minimalism with the lofty goal of total ephemeralization to build coherent, stable, and operable stacks.

• 
  Is GraphQL the Next Frontier for Web APIs? Mar 29, 2017

  Musings on the next API technology, and whether REST-ish JSON over HTTP is just “good enough” to never be displaced in a significant way.

• Why Doesn't Stripe Automatically Upgrade API Versions? Mar 17, 2017

  Given a wide API surface area are small changes, most API upgrades are safe for most users. Could we upgrade their fixed API version automatically?

• 
  Learning From Terminals to Design the Future of User Interfaces Jan 28, 2017

  How we overvalue the wrong technology and novel aspects of interface design at the expense of substantial gains to our productivity.

2016

• Using Canonical Log Lines for Online Visibility Nov 26, 2016

  A lightweight and technology agnostic operational technique for easy and near realtime visibility and live querying into production systems.

• Service Limits Nov 8, 2016

  Why and how to put limits and rate limits on your web services. Introducing redis-cell.

• The Case For A Go Worker Pool Aug 19, 2016

  Error handling and fragility; or why a worker pool belongs in Go’s standard library.

• The Missing Manual for Hacking Postgres Aug 17, 2016

  The basics of Postgres development: how to build the project, run the test suite, and format a patch for submission.

• AWS Islands Jun 15, 2016

  The case for a concerted effort to build a powerful, but streamlined, platformon AWS.

• The Intrinsic Static Site Apr 9, 2016

  Building a static site on AWS with a global CDN, free HTTPS with automatic certificate renewal, and a CI-based deployment process powered by GitHub pull requests.

• Notes on Go Mar 31, 2016

  Notes on the language after spending a few weeks building a large project in it.

• A Guide to Free CA-Signed Certificates Jan 31, 2016

  Getting a certificate that most browsers would accept used to be difficult and expensive, but that’s changing as we enter a relative golden age of web encryption. Read this for options for getting certificates issued for free.

2015

• 
  My Heroku Values Nov 5, 2015

  Some of my favorite practices and ideas from almost four years at Heroku.

• Exploring Object Hierarchies in Psql Oct 26, 2015

  Using backslash commands in psql to navigate and describe object hierarchy in Postgres and Redshift.

• 
  Running at Stripe Oct 24, 2015

  Crunching running data with prepared statements in Postgres.

• Rate Limiting, Cells, and GCRA Sep 18, 2015

  Implementing rate limiting using Generic Cell Rate Algorithm (GCRA), a sliding window algorithm without a drip process.

• Designing Alerts Aug 18, 2015

  A set of general guidelines to consider when designing a alerts for a production system.

• Postgres Job Queues & Failure By MVCC May 18, 2015

  How Postgres’ concurrency model coupled with long-lived transactions can degrade the performance of indexes on hot tables in your database.

• A Month of Kinesis in Production May 3, 2015

  A short write-up on findings, limitations, and opinion on Kinesis after a month in production.

• Kinesis Shard Splitting & Merging by Example Mar 19, 2015

  Splitting and merging in action.

• Guaranteeing Order with Kinesis Bulk Puts Mar 5, 2015

  On guaranteeing order with the bulk put API of an event stream.

• Version Variants Feb 17, 2015

  A simple mechanism for managing changes to a web API and to help cheapen the disposal of prototypes.

2014

• Small, Sharp Tools Dec 12, 2014

  A few words on the Unix philosophy of building small programs that do one thing well, and compose for comprehensive functionality.

• Dotfile Secrets and GPG Nov 10, 2014

  Learn how to start encrypting dotfile secrets with GPG, and some techniques for getting those encrypted files integrated with your toolchain.

• Package Management in Go Oct 20, 2014

  Understanding the benefits of Golang’s restrictive (but simple) import and package management system.

• Command Exit Status Sep 28, 2014

  An exercise of discovery around how to extend the shell’s API.

• The Humble Data Warehouse Sep 5, 2014

  Data warehouses aren’t just for the enterprise! Let’s examine a few of their basic characteristics, and build some examples with Go/Ruby and Postgres.

• 
  A Cheap X100S Filter Ring Hack Aug 3, 2014

  If you find that the price tag for a Fuji-official adapter ring for the X100S is a little hard to swallow, this 5-minute hack will save you 90%.

• Elegant APIs with JSON Schema May 27, 2014

  A quick tour through some of the fundamentals of JSON Schema and Hyper-schema, and how they can be applied to building APIs.

• Testing Distributed Services with JSON Schema Apr 30, 2014

  Stubbing distributed services with Sinatra-based services stubs and enabling stronger constraints with JSON Schema.

• Microservices Apr 1, 2014

  Useful distinction or new buzzword? Comments on 200-500 line services.

• The Mediator Pattern Mar 11, 2014

  Interactors by a different name.

• Healthy Anti-patterns Feb 6, 2014

  When the use of an anti-pattern is considered beneficial.

• Why I Don't Want Your SDK in Production Feb 3, 2014

  When building an app against a web API, do you pull in their SDK or just make raw HTTP calls? Here are a few reasons that I avoid SDKs when I can.

• 
  BreakTime Classic Feb 2, 2014

  In search of an alternative to BreakTime. The discovery of a very classical solution.

• 
  Lamenting the Death of the Page Jan 26, 2014

  How the page almost transitioned successfully to the digital world, but is in decline in new media. The lessons that we can learn from this age-old design element, and why we should hope for its re-emergence.

2013

• logfmt Oct 28, 2013

  A logging format used inside companies such as Heroku and Stripe which is optimal for easy development, consistency, and good legibility for humans and computers.

• Developer Accessible APIs Sep 1, 2013

  A set of patterns to make APIs more accessible to developers; lowering the barrier of entry for new users, and easing the maintenance of consuming applications.

• Scoping and OAuth 2 Jul 23, 2013

  Designing scope for Heroku OAuth, and a brief tour of other implementations on the web.

• SOA and Service Stubs Jun 3, 2013

  How we build minimal, platform deployable, Rack service stubs to take the pain out of developing applications that depend on an extensive service-oriented architecture.

• Tracing Request IDs Jun 2, 2013

  A simple pattern for tracing requests across a service-oriented architecture by injecting a UUID into the events that they produce.