John Morrow

What does it mean to be a manager?

Sun, 02 Jul 2023 08:19:33 +0000

Over the years at Gauntlet I have come to manage a team of 20+ people across a variety of disciplines - Finance, Product, People Ops, Growth, etc. I wrote down what I thought was important for managers (and to some extent, ICs) to focus on. As I have started to advise more projects, I’ve noticed that others have struggled with clarifying management responsibilities and so I wanted to share this more broadly. For anyone reading, this will mostly apply to those working at smaller start ups.

What does it mean to be a manager?

Drive impact, scale and growth

Really simply, you need to figure out how you and your team can help the company succeed. Every company is a machine. You take in money and output software. That software then drives more money coming in, mostly from customers but occasionally from investors as well. You need to figure out how to help the company take in more money or output better software. That’s impact. As the company grows, a new challenge arises - this becomes less direct for most individuals and we need to figure out how to ensure individual efforts maximize collective impact. That’s scale. I cannot reiterate enough how strangely hard this is. You need to constantly work to drive alignment across the organization and within your team to achieve this. As Boz mentioned - comms is the job. Lastly, you need to ensure that you not only grow your ability to drive impact over time but all of your reports as well. To do this, you need to give your reports real ownership.

A bias towards real delegation and ownership

It’s really hard to scale your impact if you do not delegate to ICs very aggressively - you can only do so much yourself. It’s also really hard for people to grow if they don’t have real ownership over the initiatives they are working on. Delegation is giving people stuff to do - real delegation is giving them stuff to own. When you onboard a new employee, you face the question - is this person set up to succeed yet? You need to embrace the firehose. There should be no “ramp” when people join. A ramp is something you eat at a farm to table restaurant. Success is when a new employee actually takes ownership over their responsibilities at the company and drive impact. You cannot know if they are able to do this without giving them the opportunity to prove that they can - and with this comes the chance that they can’t. No one can achieve success without risking failure. You need to have credible failures. This means you focus on high-value, high-variance outcomes, and if you don’t hit your goals, you should die trying. You must be results-oriented, and rise to the challenge of doing so over not just one quarter, but many. It’s really hard to do this without support.

Total support

You are responsible for your employees work. If you delegate the work to them, and you should as much as possible, you should serve as the backstop to ensure that if they are not going to hit their goals, together you will. This ensures that as managers work to grow their reports as contributors, and those employees (and the company itself) are able to succeed. This starts with being very proactive.

Very proactive

You need to get ahead of things. If you or someone on your team has a goal, you need to do as much as possible to track progress and right the ship if things get off-track. Beyond this, you also need to give feedback as early and often as possible. At the end of each quarter, you should have a conversation with each of your reports to discuss how things are going. At the end of each half, this conversation will provide important context to their review which can greatly affect their compensation and career trajectory. Performance should be strictly tied to impact. If you are not proactive, you may have to surprise someone with negative feedback that they have no chance to address beforehand. This is very damaging to trust - it’s hard enough to address the feedback you get, it will feel very unfair when you have to tell reports that they are getting a lower score because of feedback you didn’t even give them yet. I will continue with a somber example as this is so important. It’s just impossible to hire the right people all of the time. Over time, we will all be faced with a circumstance where we need to let someone go. It’s not fun. However, if you set expectations really well, you will at least be on the same page with this person in that you gave them every opportunity to succeed. If not, the incredibly sad look you will inevitably receive will be replaced by a different, worse look - one of deep indignation that shows how betrayed they feel. Feedback is only one part of helping your employees succeed, you also need to manage up and advocate for them.

Managing up

Managing up is really two things -

Letting leadership know how they can better support you and your employees and increase your impact
Helping leadership understand what’s going on so they can make effective decisions.

Good employees do what you tell them to do. Great employees tell you what to do. They work to figure out how to move the company forward, get your buy-in, and tell you how to help them. Managers need to be great employees. Not just to lead by example, but because they hold the success of everyone who works for them in their hands. I mention earlier that you need to constantly drive alignment - managing up is a big part of this. To help your reports, you need to reach out to management/leadership to ensure you have everything you need to do so.

Telling leadership what’s going on is essential to all sorts of effective decision making:

Resource allocation - they need to understand both costs and progress to understand how to reallocate resources over time
Realignment and direction setting - if they don’t know what people are doing, how can they know that different teams at the company are aligned on what progress means. Impact is limited by people missing their goals, but even more so by people hitting goals that are not impactful.
Fair reviews - Management cannot reward employees for work they don’t know about. You need to advocate for them.

Strong advocacy

You need to be your report’s strongest advocate. This happens throughout the quarter, where you should make sure to communicate everyone on your team’s impact. It becomes more important at the end of each half, when you do reviews and calibration. At this time managers will need to make a case to their peers for each of their reports. It’s hard to make that case if you don’t have any ammo going in, and this should push managers to ensure their employees are growing and succeeding. If you are proactive about this responsibility, it should be easy to tell a story for a fair and good review. Strong advocacy doesn’t mean that you can just propose that all of your reports get a top review score - you need to maintain trust with other managers and you need to be honest about progress to do so.* Management knows they ask a lot from managers on their team, but they should hold themselves to that high standard as well.

A high standard

You need to hold yourself to an incredibly high standard. This means that you need to raise the bar for what is expected of employees and what is possible. This is not because you want to admonish those who do not meet this bar, but because you want to inspire people to meet it. If you work at a company that is good at this, you wont just be the best version of yourself, you’ll be a new and better person entirely. Every pitch deck template you see has a “why now?” section or something similar. Each one of these ends up pointing to a more straightforward question - “What are you going to do that everyone else thinks is impossible?”. You need to make the impossible possible or you are by definition, not innovating - without innovating you will inevitably fall to competitors. When you first start at a company as an IC, you might even be able to do well even if the company isn’t hitting its goals. As you advance, this is less and less the case. This is not “fair”, but that’s the point. You have to take responsibility for things you don’t control. The founders will - they have to. Your customers just want to save money. Your investors just want to make money. They don’t mention fairness. If you don’t hold yourself accountable, your customers and investors will.

There are some specifics here I have not totaly edited out. For instance - Your company might not have a bi-annual review cycle or do calibrations like Gauntlet did at the time of writing. However as a manager, I’ve found these concepts to be helpful. I hope you do too.

*This sounds a bit like politics. There is a fine line between politics and effective management. Effective managers ensure that their team has the scope to drive impact and that the company understands what they are working on and why. Effective politicians jockey for more scope to improve their position at the company, lean on other teams to help them hit more goals without giving credit, and frantically brag about their team and their accomplishments to create a positive perception. The difference between the politician and the effective manager is really a matter of intention - the manager puts the company first, the politician puts themselves first. It’s on management to align incentives as much as possible so that the opportunity to push yourself forward at the expense of the greater organization is as small as possible. Still, as a leader at your company, you need to set an example that shows that cynical politics are not what the company values.

Be greedy, then smart

Thu, 01 Jun 2023 08:19:33 +0000

At many companies I have worked at, people struggle to find a balance between aggressively taking responsibility (good) and stepping on others toes (not good). I summarize my thinking on this as

Be greedy, then smart

This means it is best for people to bias towards owning more things, not less things. Of course, at the same time, you need to make sure there are not ongoing overlaps in ownership that could create unclear responsibility, and that the team best suited for the job ends up with the right area of ownership. However, people need to be greedy with respect to ownership first, and then allow potential owners to decide on who is best suited to work on something long term. I wanted to highlight a couple of principles here that we should think about as we continue to drive stronger ownership and scale:

If something is important and you don’t own it, someone else at the company will have to own it
1. Even though something might not be “in your wheelhouse”, it could be even less in someone else’s wheel house
If we bias towards more ownership, gaps in ownership become collisions, which we can more easily identify and resolve
1. Two people owning something inherently has two people aware of it, a gap in ownership by default is something that must be discovered
2. For things that inherently stretch across individual responsibilities like system architecture, you want more people to try to contribute to this, not less. A market of ideas is the only way we can determine the best possible path forward
On a longer timeline, resources should not drive responsibility, responsibility should drive resources
1. For example:
  1. If we find out that a team doesn’t have the headcount to support needed development, we’ll of course need to be flexible and use other team resources to support
  2. However if we do this, we should be sure to escalate and make sure this mismatch is reflected in hiring plans
  3. We need to be flexible, but we need to make sure to close the feedback loop to align the company structure to our needs as best we can

If you are never stepping on people’s toes, you are not being aggressive enough. It’s the only way to ensure that you are taking strong ownership over what’s important. However if you bias toward ownership (be greedy), then clarify that ownership and divvy up as logically as possible (be smart), it will drive the best outcomes for your company.

DAO Incentives

Tue, 13 Sep 2022 08:19:33 +0000

Originally run as an op-ed in Blockworks . For more info on DAOs and working with them, also check out my appearance on I Pledge Allegiance, a podcast about DAO economies and governance.

Key takeaways

Transparency that comes with public code review is useful, but the history of Defi exploits shows it doesn’t solve the problem

A new payment structure will better align Gauntlet’s incentives to the DAOs they serve

It’s truly exciting that changing one line of solidity code — the language of Ethereum smart contracts — can create an entirely new financial product.

But how can we understand whether these new instruments work as intended? The risks are poorly understood, and the scale of losses is staggering — the system is broken! We can improve, if not fix, this with better incentive alignment — putting our money where our mouth is.

Professional code audits and the transparency that comes with using only open source code are not enough — we’ve seen protocols that suffer losses even though they have undergone three or more audits.

DAOs have been formed around new protocols that allow them to incentivize and coordinate contribution. Protocol DAOs need to address the risks their products create — token holders are often left holding the bag when things go awry, and one solution we’re optimistic about is based on a simple guiding principle.

If you are wrong, you don’t get paid

If users get rugged by a protocol hack, auditors should too. If the market tanks and a protocol becomes insolvent, the risk manager shouldn’t get paid either.

In this spirit, we will move all Gauntlet engagements to a new payment structure where we are held accountable for the quality of this service. How?

Service fees will be held in an escrow contract until it’s crystal clear that the work was successful.

A strong ecosystem of independent organizations contributing to protocol development and operation is essential to a successful decentralized protocol. Since the proliferation of DAOs in 2020 and 2021, we have seen a few core services emerge:

Protocol development (e.g. Comp Labs, BGD)
Risk Management (e.g. Gauntlet, BlockAnalitica)
Smart Contract Auditing

Add to that services such as tooling and analytics (e.g. Chaos, Flipside). While these service providers have taken the first step into the new decentralized economy by working with DAOs, the structure of these engagements has lacked innovation.

Compound innovation

We’d like to see more initiatives that link the fate of service providers to the success of protocols.

As a first step, we recently posted our proposal to renew our service agreement with Compound and added a clause around the concept of a “first tranche insolvency refund.”

There remain many details to figure out as we move forward, but let us be clear — this isn’t a protocol-funded backstop like the stkAAVE pool, which is intended to maintain protocol solvency.

Instead, what it means is simply that 30% of the contract will be put up against our work. It’s an idea to provide true alignment with our customers, to hold ourselves accountable for what matters to them — driving growth at safe risk levels.

Governance allows the community to weigh in on every change we make, and if the community wants to take on more risk than we propose, they can, but they would forfeit this alignment.

Traders and degens at heart Many of us came from quantitative trading. We dream of markets and live for risk. At trading firms, your compensation is almost entirely variable, and we’re applying this ethos to our work in crypto.

Merely aligning incentives via escrow barely scratches the surface of how we could leverage this incredible technology not just to decentralize finance, but innovate on it as well.

Small companies vs. Big companies

Fri, 05 Apr 2019 08:19:33 +0000

I’m on a forum run by one of our investors here at Gauntlet and someone recently asked a question I have thought a lot about:

How is being a Product Manager at a large company different from being a PM at a start-up?

I have worked at both a large companies (MSFT, Goldman Sachs) and small start-ups (when I joined Gigster I was the 9th employee and at Gauntlet was the 1st). Everyone should do both.

Big companies

Being a PM at a larger company is great in many ways. I think the main benefits are scale, support, and security.

Scale

I was the PM for a pretty obscure API in Windows and it had thousands of developers use it within the first few months of launch. Even if you are building the main API at an enterprise SaaS start-up, it might only have that sort of usage after years of development and sales. In general, working at large companies allows you to work on small things that leverage the scale of your large company to have an outsize impact. I remember talking to a PM for Excel who mentioned that one button she had added to the product created something like 5 million man-hours of user productivity because it was a button that got pressed a billion times a year.

Support

When you PM at a large company, you can dedicate almost all of your time to core PM things like data analysis, roadmap planning, writing specs, working with user research etc. You have an army of people that help you focus. When I wanted to do a user research study at Microsoft, it was one 15-min meeting showing the researcher the feature and then a month later I walked into a purpose-built room with a one-sided mirror and watched as the user research team ran through the product with a group of people they had hand-selected to match Windows usage demographics. At a start-up, you are doing all of this yourself, and you aren’t doing nearly as good of a job. Do you want to grow the impact you have at a larger company? There is probably a novel’s worth of documentation on how to do this, a company-wide mentor program to connect you with people that can help, and your manager is probably dedicating 80% of her time on management. At a start-up, the opposite is often the case - maybe no one knows what you’d have to do to get promoted to Head of Product or Senior because it’s possible no one has done that yet. You might have to push management to define these things before you will be able to accomplish them, and your manager (the person who is supposed to help) is probably spending 80% of her time on her IC responsibilities and doesn’t have the time to help you figure this out.

Security

Both job security and financial security are going to be better at a large company than a start-up. At start-ups, there is always a chance that the start-up will fail, pivot, or otherwise change in a way where your job is not longer extant or necessary. Cash compensation at large companies tends to be higher by quite a large margin than at start-ups as well. If you are starting a family, have a mortgage payment, or are otherwise unable to trade cash compensation for illiquid equity compensation (as you are asked to do at almost all start-ups), making the start-up life work can be quite a strain.

Start-ups

With that said, you might think that I am probably a MSFT lifer who has somehow grifted their way into this forum. However, I have worked at start-ups for the past 4 years and have no plans to work at a large company again in the near future. I strongly prefer working at start-ups to larger companies. I prefer working at start-ups because I think they are better for scope, opportunity, and motivation.

Scope

When you are the PM at a start-up (or even just one of a few PMs) you should have a considerable impact on everything from the overall direction of the company to the types of drinks in the fridge. If there is anything you don’t like, you can change it. If you find a process that works on your team, you can try to roll it out across the company. Also, even a relatively junior PM might be the sole owner of a product or large feature set. In general, at a smaller company you have so much more impact on the things you work on than at a large company. Now, those things impact a smaller number of people, but this can be a good way to get a lot done, bringing me to my next point.

Opportunity

Start-ups provide a tremendous opportunity for the people who work at them. When your start-up is growing really quickly, it is nigh impossible not to become a manager within a year or two. You also have the opportunity to bring something from zero to one, in Theil-speak. You get to build a new product and develop it into a successful one. This is just a better opportunity to put something on your resume that is truly impressive than you would get at a large company. Sure, at a large company you might be able to make a change that drives $2mm in incremental revenue (on a product with likely billions in total revenue). But what would make someone in the future want to hire you more? Doing that, or starting a product that had $1mm in revenue? There are better opportunities at start-ups to do something that shows to future employers how valuable you can be. Also, in the case that your company ever finds an exit, then you have the opportunity to make quite a bit more money than you would have at a large company.

Motivation

I do my best work when I am motivated, just like most people. I feel more motivated at start-ups because everything you do matters, you have broader responsibility, and nothing can stop you from getting stuff done.

Everything you do matters

As the PM at a start-up, you are building products. Your performance is likely tied to revenue, something inherently valuable. If you fail, the company might fail. Some people might find that stressful, I find it motivating.

Breadth

I mentioned a friend above at MSFT with her million man-hour button. Imagine you are that friend and your entire task for this month is to just spec a button. You schedule meetings with partner teams, get estimates from dev, etc. - but just for one button. I don’t find this particularly motivating - you are not doing anything valuable per se. Even if I was the best button-PM in the world, I still wouldn’t really know anything about building products, though I would know a ton about building buttons. The amount of specialization that you have to do is something I find inherently demotivating (to be clear - this gets much better as your career progresses). This matters a lot to me - as an entrepreneurial person who has considered founding a company, it’s hard to see being a PM at a large company as a path to doing so. At a start-up, it’s easy. If you are reading this and think that my reductive take on this button PM sounds a bit ego-driven, you might be right!

Getting things done

At a start-up you can move mountains to get stuff done. Imagine you want to release a new product that could really sell well, but your Customer Success team can’t make the changes to support this product, at least for a long time. At a very large company, that might doom your goal of releasing the product in six months let alone ever. At a start-up, processes and responsibilities are so much more flexible, you could conceivably still release quickly - smaller groups inherently can organize around value quicker. Anything you don’t like or you think could be improved, you can change.

There’s value in both

I emphasize above that everyone should work at both a small and a large company. I am extremely glad that I worked at a large company before working at a start-up. I just came in with so much knowledge of how things could be done - it was easy to make progress when faced with ambiguous paths forward since there was always a path to fall back on. Now, you won’t be able to apply your big company playbook at a start-up mindlessly - you will have a bad time - but it is a good place to start. Also, by working at both, I now have a pretty good idea of what works best for me.

One final caveat regarding start-ups - if your start-up is not going well, it can be extremely difficult to realize any of the advantages I mention here. Not surprisingly, start-ups carry quite a bit of risk. I don’t think the expected return on working at a start-up is higher than at a large company, all other things being equal. In fact I think the expected value at a large company is probably slightly higher. However for me, as someone who is risk tolerant, entrepreneurial, and inherently more motivated by 0-1 opportunities than 1-10 opportunities, start-ups are the right choice. You need to work at both to know what the right choice is for you.

Automatically printing your to-do list every day

Sun, 06 Jan 2019 08:19:33 +0000

For a long time I have used hand written to-do lists as a form of daily task tracking. As task management software has gotten better, the inconvenience of the paper list (copying it, losing it, etc.) has made even a luddite like myself ditch it. I have been using Trello for a few years now and it’s been great. However every time I have to futz with my phone to update the list, I long for the good ol’ days when I could mark my progress with the swift and satisfying swipe of a sharpie. After hearing about the Memobird printer, I figured I could maybe use it to automatically print out my To-do list everyday, so I’d have the best of both worlds.

It turns out the Memobird’s API is still in a private beta for now, but they do have an IFTTT service. This, plus a few other automation tools, is all we need to make this happen. I’ve included instructions below in case other Memobird owners want to give something like this a shot. Here’s what I did:

Set up an IFTTT applet with a webhook that prints the payload on the Memobird
Wrote an AWS Lambda endpoint to pull my to-do list from Trello, format it, and send it to this webhook
Set up a Zapier “zap” to spin up the Lambda endpoint every day at 8am

IFTTT configuration

Create a new applet and add “Webhooks” as the “this” step
There should only be one option for trigger, “Receive a web request”
You then just set the event name to whatever you like, I chose todo_print (anything in snake case is fine)
Set the “that” step to “Print a message to a memobird service”, using the 16-digit hex code ID for your device as the argument for the “Which memobird device?” parameter. You can press the button on the printer twice to print out this ID if you don’t have it handy
Set the “What message to print?” parameter to {{OccurredAt}} {{Value1}}
Look up your key and URL here, where your key is in your URL after use/

You can then test this on your printer with a simple curl command,:

curl -X POST "https://maker.ifttt.com/trigger/todo_print/with/key/{your key}?value1=<html><body><br>test+test2</body></html>"

AWS Lambda

So it’s entirely possible to set this up using just Zapier and IFTTT, if you use a multistep zap on Zapier and their Trello integration. However, you need to buy the paid tier and it starts at $250 a year, hence Lambda. I took a quick look at the limits on the AWS free tier and unless you had a very long to-do list, it’s nigh impossible to exceed them with this simple function.

Create a new AWS Lambda function here
After you click create, you’ll choose “Author from scratch”
Select a name (“getTodoListAndPost”), set the runtime to Python 2, and create a new role with the template “Simple microservice permissions”

You’ll also need to grab some info from Trello so you can point your Lambda function to the right list:

Look up the list ID on Trello (go to board, menu, more, print and export, export as json)

In the json, search for “Today” or whatever your “Doing” list is called and copy the adjacent ID

Get a key from the Trello API here, and then click through to get a token. You’ll have to grant “Server Token” permissions on your Trello account to do so)
Add your list ID, your key-token pair for trello, and your key for the maker API to the environment variables on Lambda
Add the code below (download the file here) in your lambda_handler
Test, using any input ( I used the “Hello World!” sample event). This should print your to-do list on the printer successfully at this point
Publish your Lambda function

import os
import json
from botocore.vendored import requests

def lambda_handler(event, context):

    # you should probably reduce this in your code, but this was just broken down for readability
    full_url = "https://api.trello.com/1/lists/" + os.environ.get('list_id') + "/cards?fields=id,name"
    full_url = full_url + "&key=" + os.environ.get('trello_key')
    full_url = full_url + "&token=" + os.environ.get('trello_token')
    response = requests.get(full_url)

    cards = response.json()
    names = []
    for card in cards:
        names.append(card["name"])
    output= {}
    output["value1"] = "<html><body><br><ul><li>" + "</li><li>".join(names) + "</li></ul></body></html>"

    # send to the printer's ifttt hook
    printer_result = requests.post("https://maker.ifttt.com/trigger/todo_ready/with/key/" + os.environ.get('maker_key'), data = output)

    # So, you obviously should be doing some error handling here, but for a small
    # personal project like this, I'd rather cross that bridge when I get to it
    return {
        'statusCode': 200,
        'body': str(printer_result),
    }

So now that you have your Lambda function all set up, you need to set up a way to tell it when to execute. We’ll use an API Gateway to do this.

On the page where you added the code to your Lambda function, click on “API Gateway” under “Triggers” to begin API Gateway setup

I’ve shown my settings here configured, but you might need to create the API endpoint, which should be straightforward enough.

Test your API gateway trigger with a curl command e.g.

curl -X GET "https://ijmtjfn8f0.execute-api.us-west-2.amazonaws.com/default/getTodolistAndSendToIFTTT?key=f****************************"

You can get the URL from the Lambda interface:

Adding the chron job to Zapier

Once Lambda is set up, you can just create a simple Zap in Zapier to hit it everyday. You might be wondering, couldn’t I also do this in IFTTT? The answer is, of course, but I’ve found Zapier to be a bit more reliable than IFTTT and has better error messages.

Create a new Zap whose trigger is “Schedule”, and select “Every Day” and the time you wake up
Add the action “POST” from “Webhooks by Zapier”
Add the URL you tested in the last step, with your API key from your API Gateway configuration

One last tip

I try to keep the habit of writing my daily to-do list the night before, but sometimes I forget to update it and I want to make some edits after it prints. I just have the curl command above saved on my desktop and I can reprint by quickly re-running that command.

Git for PMs

Thu, 22 Sep 2016 01:01:33 +0000

A cute little site “Oh shit, git!” came to my attention recently and I found it super useful. However, I haven’t been doing a lot of development at work (focusing on uh, my actual job) but I still found myself dusting off the old keyboard from time to time to help make a small change, normally a copy change, etc. Since I am in and out of Git less frequently, I tend to forget a lot of common commands that are helpful when working with dev to either test changes locally or make small fixes. The Github desktop UI is helpful, but it doesn’t cover everything. I figured I put together a list of a few commands I always forget.

I tried to sync a branch and I got a bunch of weird errors. I just want the latest from the repo and I don’t care about my local commits

git checkout master or select master as your current branch in the UI
git branch -D <branch name>
git reset --hard HEAD
git checkout <branch name> or select as your current branch in the UI
git pull or select “Sync” in the Desktop UI

This is usually something you see after someone else rebases / merges. Normally as a PM, you just want the latest version of a change and don’t need to merge old commits you have lying around.

Just one for now but will add more as I think of them. If you have any suggestions, there’s a link below to make a PR to this post.

A Different Kind of Interview

Mon, 16 Nov 2015 23:19:33 +0000

When people talk about work, they often use language like “that sounds like a good job”, without mentioning for whom, as if the job was something that existed on its own, without someone working it. What makes a job good is that it is a good fit, a match of employer and employee. This is something that is lost not only in language, but in the job interview as well. At the very least, an interview should:

Allow the company to ascertain if the applicant would be successful at the job in question
Allow the applicant to learn about what it would be like to work at the company

The standard format for product managers - four to six one-hour one-on-one interviews with current employees - seems to do a decent job at the former. However, from my experience being interviewed, I can say that it doesn’t do particularly well at the latter.

I recently interviewed for a position as a Product Manager at a small start-up in San Francisco called Gigster. They used a different format which felt like a much more straightforward and effective way to judge a candidate. The interview consisted of two short one-on-one interviews, and then a 3-4 hour practical.

The Interviews

The night before the on-site I had a short phone call with one of the founders who described the interview process to me for the first time. He also gave me access to the company’s Git repo, slack channels, and customer feedback data. I had until the next morning to absorb as much as I could, and also sleep a little. The next day, I had a one-on-one with each founder for 30-45 minutes each. They asked mostly design based questions, e.g.:

What is a product you like? Describe how the UI works and how you would change it if you worked there? I picked Snapchat.
What are the metrics that would define the success of the Google Search Page? How could you make it better? How would it effect those metrics?
One of our product goals is maintaining customer satisfaction (actual question was more specific, but I don’t want to talk product specifics publicly). Here’s part of the product - does it do a good job of achieving that goal? How can we make it better?

Both founders were focused on keeping things conversational, and so we ended up discussing my experience and how it prepared me to contribute without explicitly prompting it. It would be hard to say that my one-on-one interviews at other companies were interrogations, but they definitely fall on a spectrum from friendly to adversarial. These fell firmly on the friendly side, and in doing so gave me a better opportunity to ask questions and learn more about the founders and the company. This is actually similar to some of the better one-on-one interviews I’ve had - thought the length (60-90 minutes) was much less exhausting.

The Practical

For the practical, I was given a few hours to put together a pitch on two prompts:

For a product area, analyze the current solution and provide a prioritized list of product improvements.
For that same area, pick a specific piece of the UI and design an improvement.

I actually thought the 4 hours I was given was going to be enough time to do this, but this was a little naïve. Since I hadn’t really gotten a chance to use the software too much before the practical session, I had to spend a lot of time just gathering information from customer satisfaction reports and trying out various parts of the product. I ended up not quite finishing both questions, and the visuals couldn’t have been more crude. However, I got to talk through the deck I made with the founders, and we got into a long discussion about success metrics and analysis that covered a few of the things that I couldn’t get to in my deck.

Why is this better?

I can’t really speak for the founders who interviewed me, but I felt at the very least that my ability to demonstrate my effectiveness was as high or higher in this interview format. I don’t think it is necessary for a job interview to try to emulate the day-to-day of the position, but it seemed like a really effective way to achieve the two goals I mention above - especially the second. The practical problem gave me a feeling for the day-to-day of my job unlike any interview I had done before. I got to learn what kind of problems I would be solving, and then in the review of my practical, the product thinking of the company I would be working for.

Preparing for the Practical PM Interview

So, in case you were wondering, I got the job. Here are some things I did to prepare that I thought were effective. Some of these things are not specific to the practical interview, but were helpful nonetheless.

Use the product as much as possible beforehand. As well as competing products. Everything you know about the product helps make you ideate improvements faster and more confidently. Being familiar with competitors helped me understand what was unique about Gigster’s value prop and what differentiators were important to focus on in the product. Still, this is something I could have done more of. I felt familiar with Gigster and where it fit in the market, but I still wasted a lot of time in my interview poking around in the UI.
Try to guess what the practical work is and do it ahead of time. I guessed that I would have to put a deck together analyzing the product. This ended up being sort of true, but the focus was less on the state of affairs and more on how to improve it. This got me thinking about the company and the product, loading my mental cache for the next day. One part that was surprisingly helpful is that it got me used to using the tools I would be using the next day. I made a few diagrams in Powerpoint and I definitely felt more comfortable with it the next day then I did the night before.
Write down well thought-out descriptions of your contributions to previous products. In a lot of my interviews, I have been asked questions like “Tell me about a time when you had to make an engineering decision and how you made it”. I was not asked about my experience during my interview at Gigster. It was still useful to have these contributions at the tip of my tongue. At one point, I wanted to make the argument that was important for Gigster to make improvements that helped both their developers and their customers. I easily tied in my experience working on Windows APIs where I had the same dual mandate, helping reiterate my qualifications.

Finding a good fit

The practical interview felt like a real improvement over past interviews I have done. It was a straightforward way for me and my future employer to evaluate how I fit in the role. I’m sure there are a lot of ways you could prepare for a practical interview, but the three methods I mention above worked well for me.

The last time I started a job was in August 2010. I start work in two weeks. Wish me luck.

Misleading Data Use on Vox Media

Tue, 25 Aug 2015 08:18:33 +0000

Recently Vox posted an article about Carly Fiorina, in rebuttal to her recent interview with Katie Couric. In this interview, she made a few classical conservative arguments criticizing Obama’s actions on Climate Change, and was lauded by conservatives for her performance in the interview. I am someone who objects to many of the assertions she makes, as does the author of the piece, David Roberts. However, I did stumble across some puzzling data when reading his refutation of Fiorina’s talking points. In his article, I believe Roberts tries to mislead his readers, as well as uses incorrect economic data to support his points.

My Objections to the Vox article

When Roberts criticizes Fiorina’s assertion that “[California] destroys lives and livelihoods with environmental regulations”, he says “Between 1990 and 2012, the state cut per-capita carbon emissions by 25 percent even as its GDP increased by 37 percent”. Wait a second. 37 percent over 22 years? That doesn’t sound so great - less than two percent a year without even taking compounding into account (with compounding, its closer to 1 percent). Off hand, I just happen to know that 2 percent a year is on the low-end for national GDP growth, so what does this below-average growth imply? What is the point Roberts is trying to make?

Fiorina says that these regulations have hurt people in the industries that they affected. The claim that somehow overall GDP growth would mean that this wasn’t the case is not cogent. The fact that California’s economy is growing has really nothing to do with how some subset of it’s population has fared in the meantime.
Now, let’s put the validity of his implication aside here and just consider his use of this statistic on its face. Merely stating that California’s economy has grown at a rate of 1.44% really doesn’t say anything on it’s own. How does that compare to the U.S. growth rate? What about per capita growth? When someone uses a statistic in such a careless way, it’s hard to believe that they studied that facts and then came to their conclusions as a result of their findings. It seems more likely that Roberts knew what he wanted to say, and went lazily looking for some numbers which might ostensibly support it. Using the growth rate like this, without comparing it to a baseline (either historical or geographical) seems negligent at best, and at worst deceitful.
These first two issues I have are not saying Roberts is wrong, but just that his statements are misleading. The thing is, he also is wrong. If you, like me, thought it was weird that the 8th largest economy in the world was growing at what is a snails pace (1.44%) compared to other state and world economies, it’s because it’s not true. If you look at data from the Federal Reserve, it shows a total growth from 1997 to 2012 of 49.6%, which translates to a compound annual growth rate of 2.72%. This actually compares favorably to the U.S. growth rate over this time of 2.06%. So yes, economic growth in California did fare quite well as the state slashed it’s carbon emissions. However, why (or in the case of negligence, how) Roberts misstated these figures is confounding.

The real issue with Fiorina’s comments

I mentioned earlier that I am actually not sympathetic to Fiorina’s arguments against addressing climate change. Fiorina’s statement that regulations “destroy lives and livelihoods” is problematic, but not because California’s economy has grown. It is just the most hyperbolic way to describe the way that the energy sector has changed over the past 25 years. Even without government intervention, industries change and evolve over time. Sure, California’s policies may have accelerated job loss in the the coal industry, but it’s not like they reduced aggregate demand for energy. There is no direct causal link between the regulation in California and the explosions of it’s clean energy sector, but it would be naive to think that they weren’t related. The energy sector in California has changed, but to say “lives and livelihoods [were destroyed]” just isn’t an honest description of what seems to be a normal market shift, whether it was motivated by market or regulatory forces.

Friends like these

It is possible that I misinterpreted the counterpoints Roberts presents. So I put together a spreadsheet of GDP data for the US and California from 1990 to today. No matter how I slice it, I can’t figure out a way to get the 37 percent number that Roberts quotes. If I have somehow misinterpreted his statements, it is not for lack of trying to understand them.

I’m not sure what it will take to change the minds of climate-change skeptics like Fiorina. Articles like this squander the one thing that “climate hawks” like myself and Roberts have on our side - the facts. I often find it frustrating to hear conservative arguments against climate change - how could people not see how bad the effects of climate change could be? However, with allies like Roberts, it’s not hard to see why I encounter so many enemies.

I Hate That I Love the Surface Pro 3

Sat, 28 Feb 2015 22:01:33 +0000

I recently was in the market for a new machine and was considering either a Surface Pro 3 or a MacBook Pro Retina. Though the Surface Pro 3 was great, I ended up getting the Mac because support for developing Ruby on Rails (and other frameworks) is much better on Unix. However, it was a really close call.

A friend of mine was kind enough to lend me his Surface Pro 3 for a few weeks and I got a real chance to experience the device and get a feel for what it is like to own one. I had used one for a few minutes in passing before, but without a chance to live with one, I wasn’t sure what to think.

My experience with the device was great. I really only care about four things when it comes to my main machine - performance, build quality, battery life, and application/development support. For the first three, the Surface was excellent. I was especially impressed with how sturdy the Surface Pro 3 felt considering how light it is. However, I am currently building a website in Ruby on Rails and trying to get it to build on a Windows machine was quite a pain. It took me a day (well, a lazy day) to get all of the dependent packages installed and running, which included editing and recompiling sqlite3 from source. Then beyond that, I felt like I was paying a tax everyday when things didn’t work the same as on Unix machines. This of course contrasts with the limited application support and incompatibility headaches of a Linux machine, so Mac OS felt like a best of both worlds situation.

I ended up getting a MacBook Pro Retina to replace my old machine and it compares favorably to the Surface Pro 3. However, it’s heavier and runs hotter, without a noticeable performance improvement. At a similar price to the Surface Pro 3 (I had an i5 configuration in both), it’s too bad that the Windows “development tax” - the result of the long history of Windows closed ecosystem - has pushed it out of the running.

Microsoft has taken many steps to move away from their closed source past, open sourcing much of the .NET platform along with a slew of other changes. Hopefully, they will be able to make up the ground they lost to Unix over the past two decades. However, until they do, I hate that I love the Surface Pro 3, a great machine I cannot use.

Google Foobar Challenge 2: The Scheduling Problem

Tue, 18 Nov 2014 22:01:33 +0000

After completing my first challenge, I was given a slightly harder second challenge - the famous Interval Scheduling Problem.

The task was as follows:

Input is a list of start and end times, e.g. [[1,2][3,4], ... ]
The times are between 1 and 1000000, and there are never more than 100 appointments.
You must create a function where the return value must equal the maximum number of appointments that can be scheduled
I was given three days to complete the task

If at first you don’t succeed …

I decided to try to solve the puzzle without looking up the right answer. I first thought that because the number of appointments was limited by the conflicts, and a graph was a good data structure to represent conflicts, that a graph based approach might be fruitful. I quickly made simple Node and Graph data structures, and tested it out. It worked on some of the tests provided by google, but not all of them.

Try again

As I was debugging, I began to think that this had to be an over-complicated solution to the problem. I went to Wikipedia and found that the much simpler “Earliest Finishing Time” algorithm was the optimal solution:

Sort the intervals by earliest finishing time
Accept each interval of it doesn’t conflict with the previously accepted intervals

This took only a few minutes to implement and passed the tests on the Foobar website. You can even prove the correctness of this algorithm by using a charging argument.

The anti-climactic finish

After Google accepted my answer, I was notified that I was halfway done with Level 2. I typed “request” into the command line on the site …

And that was the end of my Foobar adventure. No warning. No explanation. No closure.

Testing Google link detection, please ignore: Holloway Bay