• WhileBeing
  • Posts
  • The measurement crisis & framing the problem

The measurement crisis & framing the problem

Why measurement runs the world and findings from my qualitative interviews.

Hello from London! 🇬🇧 I’m travelling until March 12, and I’ve always found that a new environment is good for coming up with new ideas.

While I made less progress on the research component this week, I was able to conduct the bulk of my qualitative interviews (4/5), which I’ll cover later on. I’m finding some new hypotheses that relate to my original ideas on development, but also ask more questions about the entry point / best way to address wellbeing.

Measurements run the world (Research)

Do you think much about the idea of measurement and how it affects your day-to-day life? In my most recent foray into Stein’s book, this has been a central topic that has really called into question some of my most basic beliefs. Small stuff, right?

Measurement has a very long history; in Episode 1 of John Vervaeke’s “Awakening from the Meaning Crisis” series, he introduces this topic through the lens of cultures and norms. In society, you interact with strangers everyday (a very foreign concept to humans, a couple thousand of years ago).

When meeting someone, he notes how we extend a hand for a handshake. In a trivial sense, this is just a greeting, but it also shows:

  • That you are unarmed

  • Your demeanour (based on the grip, length of shake, etc.)

  • Your current state (what does it say if your hands are clammy, wet, etc.)

The same idea extends to “how are you”; in most cases, you’re not actually looking for a response (35:00). In fact, in the UK, “are you alright” literally translates to “how are you” and never (to my knowledge) necessitates a full response.

Vervaeke then draws a comparison to literacy, noting how you can put words on paper, return to them later, and share them with other (literate) people to convey a concept. If you are illiterate, the universe of problems you can solve is much more limited (49:00).

Measurement creates objectivity

How does this relate to measurement? In a basic sense, measurement creates alignment between multiple parties on a singular idea. Forms of travel (planes, trains, etc.) make sense because we have a common conception of time. Universality in the metric (centimetres, metres, etc.) or imperial (inches, miles, etc.) system allows for multiple people to build the same product, whether that’s a car or a house, and not have to question what they are working with.

Measurement has also been a tool of power and control over the ages. Monarchs often used measurement as a way of unfairly getting more taxes from their citizens; levies on a house were based on the square footage, which used the King’s foot. The exact measurement of this could change based on how much tax the King wanted to collect.

In his book, Stein notes how this was the basis of the French Revolution (153). It is also a way to evaluate performance; we grade students, meat & poultry, and the efficiency of cars. It enables multiple parties to have an objective view on something, often without actually understanding what that thing is.

The politicians who make changes to the education or healthcare system have almost never worked as actual teachers or nurses, but judge the systems based on the metrics they can evaluate. Likewise, many business owners evaluate the productivity of their factories based on metrics they agree on with the foremen, not based on their understanding of engineering, working in a plant, etc.

He also notes that measurement sparks reflection… Imagine you go to work every day and feel like you’re doing a great job. The job you’re doing, and impact you’re making, often has less impact than the actual number that you’re getting paid. If you suddenly got paid 50% less for your job, you’d consider other options - likely to keep up with your current lifestyle (mortgage, family, etc.), but also because you’d ask: “is this job really worth it?”.

Stein uses a simpler example with the bathroom scale; studies have shown that having a scale in your bathroom leads to increased rates of self consciousness and has a correlation with a rise in eating disorders (163). I use the work / pay example above because I think it’s more commonplace; and I also think it has a direct tie into wellbeing.

What are the metrics that define wellbeing?

I might be able to break it down into several categories, reflecting Maslow’s Needs. Income is more straightforward, but what about mental health? Even markers on physical health will shock people, where people who look ‘healthy’ actually aren’t.

The topic of measurement - and Stein’s perspective on it - fascinates me. I mentioned before that there is no panacea to wellbeing. However, even if I try to build something that impacts multiple areas, I need to ask: are the metrics that I’m using even the most effective way to measure wellbeing?

A final note here, as we’re getting dangerously close to thinking without being in ‘Think’ 🙂 Stein mentions how measurement can be used to standardize and homogenize perception, or identify uniqueness and differences (182). He uses the example of branches outside your window; even if they are all the same dimensions, they are all unique. This can only be revealed, however, if you look beyond their physical measurements and start looking at physical characteristics (other measurements). I think believe that many would apply this concept to wellbeing; if you can measure the right things, you’ll notice how everyone’s idea of ‘wellbeing’ is actually quite diverse.

Stein also talked about the idea of a ‘just society’. As a self-proclaimed capitalist, I’ll have to admit that I gloss over some of these concepts. I was raised on equality of opportunity, not equality of outcome; just because everyone isn’t at the same place, doesn’t mean you have an unfair society.

However, Stein drew a very interesting observation about ‘just societies’ that I resonated with. He references Rawls (1971):

“… (I)n a fully just society each individual will seek his or her version of the good in ways that are unique. In so doing they therefore depend on others to do things they could not have done, as well as things they might have done but did not.

For educators in particular it is tempting to believe that each child might somehow realize all of their diverse powers — that some exemplary individuals might become complete exemplars of humanity. This is impossible… (T)he very nature of human sociability is such that we are by ourselves mere parts of what we might be” (242)

Stein, referencing John Rawls, “A Theory of Justice”, 1971.

This hit me so hard that I sought out a Tweet from an educator that was related and asked him to unpack it. Despite going to a liberal arts college, I still treated education as a game; the better I could reflect the professor’s opinion, the better grade I would get. This would enable me to get better education in the future (grad school) and get through the door for certain jobs (i.e. management consulting) where GPA was an entry point.

Work was similar… I strived in my new grad years to be the best at ‘marketing’, and while I naturally leaned towards areas I was better at, I didn’t see this as “I’ll always be part of X marketing org, where Y people are better at certain things than I am.

In running Divisional, there is a bias from founders to work with (1) person who can do everything for 20 hours per week, opposed to (2) people who complement each other at 10 hours per week each. I get this from a headspace perspective, but it does beg the question of how willing we are to have others complement our skillsets (and our ambitions) VS trying to do everything.

The ‘do everything’ debate treats a lot of decisions as zero sum; and there are a lot of similarities when discussing wellbeing.

Full cycle development & problem framing (Think)

As mentioned above, I have ventured very far into the Think category, so I’ll try to keep this section to (2) topics that are directly relevant to Build:

Do you ever leave the system?

In my recent chat with Kishore and his with work Odessa Health, we discussed the important role that community (and one’s community ties) play in development, including the idea of a closed loop.

Contrast this to our current state: You constantly enter and exit organizations, especially ones that are designed to produce an outcome. People that have mental health problems will wait 6 months to see a therapist, and then be ‘out’ of the system when they’ve met certain criteria.

As a more widespread example, we can look at organizations. Sure, some of the more cult-like tech companies have an idea of ‘alumni’ where I identify with my former colleagues from Shopify (Shopifolk) and feel an allegiance to them, but for all intensive purposes, I’m no longer part of their organization. In 95% of workplaces, I’d imagine this is the case: you’re either in or out.

Odessa Health views this differently; while they are equipping high school students to do a specific job (Community Health Worker, CHW), there are pathways for them after the program concludes as a program trainee. This may involve staying on for an internship (formal) and as a mentor (informal).

They also see their work as equipping these students for bigger goals in life. Part of their curriculum involves teaching them other skills, like personal finance, data, etc., that will help them succeed in their broader life. The viewpoint is that they are being empowered to do better overall, and given the closed loop, they will reflect their gains in their community.

I thought about more practical ways to apply this to my Build / cohort example, where graduates would stay on and continue to contribute positively to the community. While I outlined this as a goal in my last post, I think there are models that would more accurately mirror this:

  • Course graduates get employed to teach or mentor future cohorts

  • Course graduates build their own, more specialized programs, where this becomes a feeder program

  • All these aspects of involvement affect the material burden on the students

    • I.e. course costs $2,000, but you get paid $500/semester to stay on as a mentor contributing X hours

    • I.e. we subsidize a portion of the cost of your course / program (for other students), if it emerges from the course we have created

There may be parallels to this with education, i.e. StartX is an incubator / fund designed for Stanford students, alumni, and affiliates, while Berkeley has a similar model with Sky Deck. Some of these models inevitably flag the issue of aligned incentive, i.e. building the community for the sake of profit (as an investor) VS for the returned good of the community (closed loop).

In contrast, the bullets I listed above all create closed loop systems where the value continues to be captured and compounded, VS being exported out of the community. More on this in a future post!

It’s not all about solving problems

During my UK trip, I caught up with an amazing founder friend who is very steeped in the tech / VC approach to building companies. We had some spirited debate on the concept of measurement, and it sparked some new ideas that riffs on Stein’s POV: Any shifts in an education system need to parallel the shifts in broader world systems.

The prevailing narrative in tech / startups is to focus on ‘solving problems’. Don’t worry about the longer term vision; if you can solve problems, then customers will pay you and retain, which makes your business a profitable venture. This is sound logic, but becomes dangerous when we start to look at marketing and the idea of problem VS solution awareness.

What is solution awareness?

This is a simpler topic; for many problems, you know the solutions that are available to you. I.e. if you want to get in better physical shape, you are likely already aware of at-home workout programmes, personal trainers, gym memberships, etc.

The challenge for these products or services is to convince you of the value of their offering, and why they inherently fit your problem set. This is the classic messaging problem, but you’re working with both 1) A well-defined problem and 2) A well-defined space.

What is problem awareness?

This one is trickier… Most startups function in a space where there is problem awareness but there is NOT solution awareness. Their challenge then becomes proving to the buyer that their product is worth purchasing, even if they aren’t aware that such a product (read: solution to their problem) exists.

There are, however, many areas where they is NO problem awareness. This means, the prospective buyer doesn’t know they have the problem that you’re trying to solve… In this case, your challenge as a company is to both educate them on what the problem is, why they should solve it, and how your product does that for them.

There are several problems challenges with this area:

  • Your goal becomes convincing someone to notice that something is wrong, not to solve something they already have trouble with

  • You have the freedom to create problems that may not actually exist, if they improve your narrative and the ability to sell your product

  • Most importantly: If you convince someone they have a problem, and you solve that problem, then they have no reason to leave.

    • This is irrespective of whether you’ve actually solved the root problem that they experience, and irrespective of whether they are actually ‘better off’ when looking at other measurements (see, I told you it’d come back!)

Now, I’ll take a trivial example that a good CPG friend chimed in. A disposable mop head (maybe Swiffer) was created to solve the problem of people not wanting to go through the hassle of cleaning their mop head. In doing this, they also sold them on multiple other problems… The sustainability of mop heads (“it’s not really creating more waste”) and the sanitary nature of disposable heads (“there’s lots of bacteria build up in your reusable mop”).

I don’t know if this is actually the product, but you get the idea (via P&G Pro)

Is this really true? Well, it’s partially true and regardless, people can definitely be convinced. The same goes, in my line of work, for vertical marketing agencies — you can take someone’s SEO traffic to the moon, but if their product is poor, or they didn’t think properly about their customer base, then either:

  • You don’t convert any customers for them and/or

  • You drove ‘qualified traffic’ that doesn’t end up buying anything (or churns shortly after)

Some of these areas, i.e. thinking about the customer, is technically the responsibility of the SEO agency... However, I would say 9/10 of marketing agency founders would say a customer churning due to the customer’s product not being good enough (to convert SEO traffic) is not their vault. They weren’t hired to drive customers, they were hired to drive SEO traffic.

Now, compare this to education, health, and other key sectors. If your education curriculum is designed to produce better math scores, but in doing so, it creates serious mental health problems and takes away from interpersonal skills that allow a student to thrive outside of the classroom, is it really working? Have you made ‘better students’ or was that not the goal?

If you create a drug that addresses a health condition like diabetes and encourages weight loss, but doesn’t improve their lifestyle (inclusive of diet, exercise, etc.), does it really solve diabetes? Or does it just solve specific symptoms?

To be clear, I have ZERO qualms with the traditional approach to venture building. I think focusing on solving problems is the right way to approach it. However, I think it’s naive to believe that you can solve second order problems (related to your core mission) after you have scaled, as by then, it’s likely too late (and may have already caused more harm than good).

I say this because I want to pay very close attention to these discrepancies when I’m trying to address wellbeing. If I don’t, then I might end up building a point solution that may solve parts of wellbeing, but set many others up for failure if they can’t complete the program, don’t find the solution effective for their situation, etc.

Common threads in wellbeing (Build)

It’s 1am here and I regret a few things:

  • Starting this post at 1120pm

  • Starting this post after a few beers at a comedy show

  • Getting too excited about wellbeing and likely keeping myself up much later than expected, not to mention clearing ~ 3k words

    • Nah I kid, I don’t regret that one at all 😉 

I would be remiss to end this post without discussing some of the trends from my qualitative interviews. As a disclaimer, while I am planning to conduct more diverse interviews, the ones below (largely) come from:

  • Male-identifying people in their 20s

  • Working a variety of white collar jobs

  • Who come from decent (socioeconomic) family backgrounds

With that being said, I did also get people from:

  • Various ethnic backgrounds (white & POC)

  • Various income levels (ranging from $30k to $200k)

Regardless, I thought there were some awesome findings here that did relate directly to my Research / Think sections from the past couple of weeks.

What are the commonalities with goal setting & barriers?

In evaluating Maslow’s Needs, a key question to the subjects was about the goals they have and the barriers they may experience in getting there. Everyone I interviewed had both diverse goals (i.e. playing more Dungeons & Dragons with friends VS sleeping consistently) and shared goals (i.e. getting married & starting a family).

The trend came down to how they viewed the concept of ‘goals’ and progress. Everyone I interviewed said some variation of:

“I don’t want to be complacent; I’m making progress if I’m moving forward”

“A life where I go to work at a job I don’t like and come home and watch Netflix isn’t worth living”

“If I’m not improving myself, then I’m not accomplishing my goals”

(paraphrased from various subjects)

Complacency was the biggest fear regarding goal achievement. Interestingly enough, some of the biggest question marks on “will I accomplish this goal” came down to ones that were derived from society & expectations (that were internalized), such as finding a long-term partner and having children.

The other major observation was that everyone viewed their constraints as internal. Few people mentioned their upbringing, environment, relationships, or macro factors as barriers to them accomplishing their goals. “I’m not motivated enough” was a common scapegoat when they identified an area they were falling short in.

How do they make decisions & balance expectations?

Logical thinking was also a winner across all of the interviewees. Whether it was involving health, relationships, work, etc., they often took a very utilitarian view to their decisions. As an example:

“I’ll often do less exercise and eat poorly if I have a stressful week at work. But I can justify that, since I’m prioritizing my work over my other health.”

(paraphrased from one subject)

Some participants noted that, when it came to balancing expectations from others, that there was an element of ‘making up’ the difference, and seeing reciprocation, such as:

  • Doing something nice for your significant other, because you have the time and they will likely return the action

  • Choosing a compromise with a parent that partially meets your personal goals, while also meeting theirs

Are there parallels here to Kegan’s Levels? It’s hard to say; one could argue that the first bullet is more transactional (Stage 3), while the second is more considerate of the opinions of others (Stage 4). However, this disregards the context in which they were given and only looks at applying development theory to one lens of their life.

How do they view additional income & freedom?

Many of the participants agreed that there were limited opportunities to improve their income, and often their skill levels (i.e. challenges they can tackle), through promotions at their current job. This was due to:

  • Promotions being reliant on more senior folks leaving the job (and freeing up a space)

  • Promotions being reliant on the growth of the industry and their department, necessitating another manager, director, etc. (for their promotion)

Salary increase could range from 3-10% per year, but ‘phase shifts’ might be 30-40% increases (inclusive of more seniority, opportunities, etc.) when they switched jobs. I could draw an interesting parallel here to the concept of Development Intelligence from TDE; how many of these organizations are actually fostering employee growth, VS strictly focusing on company outcomes and their ability to ‘work well’?

While everyone was open to additional income, some participants made enough income where they didn’t see it as necessary. For others, they saw a tradeoff in making extra income with their mental energy & health. Put differently, they would not opt to pursue another job, venture, etc., since they are typically tired after a full work day, and see this action as a tradeoff (i.e. would spend less time with significant other, friends, etc.).

How do they make sense of change?

As a final observation for this post, I asked participants how they made sense of a major change or challenge they had in their lives. Many cited the transition in education systems - between middle to high school, high school to university, university to the workplace - and how it moved them from a contained environment to a more open one.

This was a major problem; it meant they had to develop faster to meet the expectations of their peers. This ranged from being more extroverted to getting into relationships, all of which made the individual mirror others that left them feeling unfulfilled.

From a more positive lens, this resulted in the participants getting a better understanding of themselves, how they operated, and how being true to their values felt ‘better’ over time. However, I do raise the question of sample bias - how many people do we know who haven’t adapted well to life changes?

If you read this long, I applaud you! I’ll be sharing more comprehensive research findings next week (including final survey data), some additional interviews (with a UK culture twist), and hopefully some great conversations with other development / wellbeing-minded folks.