Back in 2007, Gareth Rees made an amazing article (archived) on how to make the smallest possible PNG, and the byte-level reasoning behind it. I figured the full code deserved to be put on the internet, so here it is :)
The article has also been saved to article.md, just in case.