Before adopting a metric by which you will be steering your business, you should know three things: the domain of what you want to measure, the range of the metric, and the responsiveness within the range to changes in the domain.
For anyone who napped in class, here are the math terms:
- A metric is a function that converts real-world raw data into a numeric representation.
- The domain of a function is all the values that could be inputs to it, so the domain of a metric is all the possible real situations it must represent.
- The range of a function is all the values that could be outputs from it. The range of a metric is all the numeric values it can have.
- The responsiveness is the change in the range that happens for a given change in the domain.
Your car's dashboard features many metrics:
- The fuel gauge's domain is the position of the float inside your gas tank, and its range is a needle position that goes from E to F.
- The speedometer's domain is the torque on the speedcup or the frequency with which a magnet goes by a detector, and its range is a needle position or digital readout that goes from 0 to around 140mph or 225kph.
- The oil light's domain is a signal from a pressure gauge inside the engine, and its range is on or off.
- The tachometer's domain is a frequency count on a magnet attached to the driveshaft, and its range is about 0 to 7,000rpm, with a red zone marked at the top.
The responsiveness is different for each metric:
- The fuel gauge starts to move down from F only when the float is no longer jammed against the top by the gasoline. It reads E as soon as the float is lying on the bottom (when there's not enough gasoline to hold it up). So F does not mean all the way full, and E does not mean completely empty.
- Most speedometers are set to read at around 110 percent of the calculated value -- not a big difference when you're doing 10 and it says 11, but significant in terms of avoiding a ticket if you're really doing 65 and it says 71.5.
- The oil light reduces a very large domain of possible operating pressures to a simple range: OK or not. It is usually set to allow a slight margin, so that when it comes on, you can get somewhere close by and add oil or have the engine checked, but there's not much slack in it. (It comes on briefly as you start the engine because it takes a couple of seconds for the pump to move oil up to where the sensor is.)
- The tachometer tends to be exactly accurate. People use it to check idle speed to see if they need a tune-up and to train themselves to manual-shift for fuel efficiency.
Notice the real key here: The responsiveness of the range to the domain always has a purpose. Fuel gauge responsiveness is set so that, if you use it right, you don't top up unnecessarily, and you rarely run out of gas. Speedometer responsiveness is set to keep you out of traffic court. The oil light doesn't scare you unnecessarily or require you to think about it much, but it warns you strongly when there's trouble. (If you pay attention to it every time you start your car, you always know it's working.) It responds to a huge domain with a tiny range. And because the tachometer is for slightly more sophisticated drivers who care about the car, its responsiveness is close to 1:1 -- i.e., extreme accuracy.
You see -- four purposes, and four different kinds of responsiveness.
So if your business is going to be data driven, asking "What can our raw data's domain be?" is reasonable. For example, can sales be negative? (Sure, if returns are allowed.) Can net revenue per transaction be infinite? (Yes, if money arrives in a way that you don't classify as a transaction.) What will the range of the metric be? (If it's always the fraction a subset takes up out of a larger set, for example, it has to be between 0 and 1. If you always use up some material in starting the machine, material efficiency can't go up to 100 percent.)
And, most importantly before you roll something out, what responsiveness will cause the most desired behavior in the user?