Sign in

What engineering is and is not

Suggest an edit

Two programs solve the same problem. One runs faster, and the other is easier to read and check for mistakes. Which one is better? The question has no answer until you know what the program is for. Engineering is the reasoned choice among options like these, and the word "better" always points back to a goal you have to name first.

A tradeoff is a choice where one gain costs another

A tradeoff is a design choice where gaining one property costs you another. The faster program gives up clarity, and the clearer one gives up speed. Both options are valid, and neither is broken. What separates them is which one serves the goal the work is there for. A program that prices trades a million times a second depends on speed. A program that runs once a night and must never quietly corrupt payroll depends on being easy to check.

The goal the work is there for is the parent goal, the larger thing every smaller choice underneath it is meant to serve. A smaller goal underneath it, such as speed or readability, is a child goal, and it is worth pursuing only while it serves the parent. You cannot judge a smaller choice until you have named the parent goal above it, because the same choice can be right under one parent goal and wrong under another.

Two real ways to send data between programs

Programs constantly send data to each other, and two well-known formats for it make opposite tradeoffs. Both are correct, and both are used widely every day.

JSON is text, so a person can open a message and read it directly. When a response comes back wrong, an engineer opens it, sees the bad field, and fixes it.

Protocol Buffers is a binary format built at Google. The same message in Protocol Buffers is far smaller and faster for a machine to process, but a person cannot read it without extra tools to decode it first.

Which one is right depends on the parent goal

Picture a public API, the kind outside developers around the world write code against. The parent goal there is those developers succeeding quickly. When something breaks, a developer who can open the response and read it is unstuck in minutes. The handful of bytes a binary format would save is a small child goal at that scale, invisible next to the cost of confusing thousands of people. Readability serves the parent goal directly, so JSON is right.

Now picture internal traffic between a company's own services, machines exchanging billions of messages a day with each other. Here the parent goal is keeping that system fast and affordable. At billions of messages, the size of each message and the cost of processing it stop being negligible, so the compact binary format serves the parent goal and Protocol Buffers is right.

This is the split that happened in the real world. Google built Protocol Buffers for the internal services where size and speed dominate, while most public APIs are built on JSON.

Same two data formats, opposite correct answers under different parent goalsTwo columns compare the same two data formats under two different parent goals. A public API whose parent goal is outside developers succeeding quickly is correctly served by JSON, because a human can read the response. Internal services whose parent goal is staying fast and affordable at billions of messages are correctly served by Protocol Buffers, because the messages are small and fast to process. The formats do not change, only the parent goal does. Same two formats, opposite correct answers Public API, outside developers parent goal: outside developers succeed quickly JSON a human reads the response and fixes it saved bytes: a small child goal here Internal traffic between services parent goal: stay fast and affordable at scale Protocol Buffers small and fast for machines to process readability: a small child goal here The formats do not change. The parent goal does, and it decides which one is correct.

Measuring one child goal is the wrong way to choose

Engineering is not making a thing as fast or as small as it can be. It is tempting to take one child goal, measure it, and declare a winner. "Protocol Buffers produces smaller messages, so Protocol Buffers is better." The measurement is true, but the conclusion skips the parent goal.

Put the binary format into a small public API. It saves a few bytes nobody will ever notice, and in exchange every outside developer now needs extra tooling to read a response they used to read directly. A child goal has been pushed to its best value at the direct expense of the parent goal above it.

Say the goal before you compare options

Before you weigh two options, before you measure anything, say plainly what the larger goal is that both options are meant to serve. Once it is named, most tradeoffs stop being a matter of taste and become a question of which option serves the goal you just stated.

When two people argue for different goals, such as performance against elegance, each of them is usually holding a child goal, and the way out is to name the parent both goals are meant to serve. If they cannot agree on one, they disagree about what the product is for, and no comparison of options settles that. Parent goals and child goals covers both cases in more detail.