What are organizations for?

The historian Ian Morris defines the social development of a society as “the bundle of technological, subsistence, organizational, and cultural accomplishments through which people feed, clothe, house, and reproduce themselves, explain the world around them, resolve disputes within their communities, extend their power at the expense of other communities, and defend themselves against others’ attempts to extend power.” One of the key components of social development, he continues, is organization (singular):

To be able to deploy energy for food, clothing, housing, reproduction, defense, and aggression, humans have to be able to organize it. Just as organisms break down without energy, societies break down without organization.

Morris measures organization by city size, but I mention it here because in thinking about organizations (plural) it’s worth starting with their ultimate purpose: to deploy society’s resources in useful ways that achieve the society’s goals.

It’s easy, when talking about the purpose of organizations, to slide directly into the shareholder primacy debate. (Here are a few links on that.) But this broader purpose is upstream of any particular corporate governance regime. The ultimate defense of organizations should make reference to the needs of society in general, however we decide to structure their obligations to particular groups of stakeholders.

This is why corporate mission statements actually are important. They might not always be accurate or specific, but asking for one is a way of posing the basic question of justification. What is the purpose of your organization? What socially useful goal have you set for yourself?

To get a bit more specific, as I wrote in a piece about shareholder value:

The right way to think about companies’ job in the economy [is] to create real economic value, not just paper value, and not just to transfer value from one group to another. The main way to create value is through innovation.

And from another piece:

Profits are supposed to be an incentive to create valuable products and new innovations, not a reward for lobbying regulators or being the first company to scale in a particular industry.

Luigi Zingales had a good quote in a paper about this:

Most firms are actively engaged in protecting their source of competitive advantage: through a mixture of innovation, lobbying, or both. As long as most of the effort is along the first dimension, there is little to be worried about. The fear of being overtaken pushes firms to innovate. What is more problematic is when a lot of effort is put into lobbying. In other words, the problem here is not temporary market power. The expectation of some temporary market power based on innovation is the driver of much innovation and progress.

When we talk about “creating value” in this context, it’s not just about financial value; it’s really shorthand for organizing resources in a useful way to achieve some social goal. An organization’s mission is supposed to be the ambition; profits are supposed to be the incentive; and, at least in a competitive market, innovation is the way you get it done.

Good non-technical resources for understanding machine learning

This will be a living post with links to resources I think are useful (or, in some cases, that I simply want to remember to look at) for non-ML-pros who want to understand machine learning and how it will change [work/society/etc]. Full disclosure, I worked on much of the HBR content. And while I vouch for a lot of the links below, like I said, some I haven’t yet read.

Overviews of machine learning or AI

An introduction to statistical learning (textbook)

Machine learning 101.

Andrew Ng: What Artificial Intelligence Can and Can’t Do Right Now

A visual introduction to machine learning

An NBER introduction to AI.

McKinsey’s introduction.

O’Reilly’s.

What Everyone Needs to Know About AI (book)

Andreessen Horowitz’s primer on AI and AI playbook

This MIT Technology Review article contains a fantastic plain english description of deep learning.

This slightly violates the non-technical rule, but there are many great MOOCs, several of them on Coursera. Andrew Ng’s classic ML course; his deep learning courses; and many more.

Intro to neural networks in 20 minutes 

The economics of machine learning

The single best starting point (and in book form)

Here is an NBER conference on the topic.

Big picture, trends, etc.

This CB Insights report is good on the VC-backed ML ecosystem.

Shivon Zilis’s various mappings are great comprehensive looks at the companies involved.

Benedict Evans on the next 10 years.

The New York Times on the “Great AI Awakening”

Erik Brynjolfsson and Andrew McAfee with the big picture

For managers

How to spot a machine learning opportunity even if you aren’t a data scientist

What every manager should know about machine learning

How to tell if machine learning can solve your business problem

Visualizing algorithms

StitchFix

HBR

Slightly different, but a good short tutorial on predicting who died in the Titanic

Other lists of links

from HBR ($)

The best ML resources.

Events, newsletters, etc.

Data Elixir newsletter

Tech Review’s The Algorithm newsletter

Conference: Machine Learning and the Market for Machine Intelligence

Mergers, social science, and unmeasured interactions

From Tyler Cowen’s conversation with Matt Levine:

COWEN: If we think about mergers and acquisitions, one of the standard results in the empirical finance literature is that acquiring firms do fairly poorly. That is, acquisitions don’t seem to pay off. Yet, of course, acquisitions persist.

You’ve done M&A work in your life. How do you think about this process? If it doesn’t pay off, is it about empire building? Is it about winner’s curse?

Do you somehow not trust the data? You would challenge the interpretation of the result? Or how good are acquisitions for the acquiring firm? And what goes wrong?

LEVINE: I wouldn’t challenge the data. It’s a similar story to active management in some ways. The fact that M&A is bad doesn’t mean that your merger will be bad, right?

COWEN: [laughs]

This in fact directly relates to something I’ve been discussing this week: that even in a completely randomized social science experiment, there are likely to be unmeasured variables that interact with the thing you’re trying to measure. So, while you can be confident in the average or net effect of the causal treatment, it may not apply — even directionally — to a given individual case.

So, you can take Levine to be making a cynical point about our ability to delude ourselves. (Like when he says “People want to do stuff.”) Or, you can take him to be making a point that average effects are just that. That’s how I read him when he says:

The data is not overwhelming that all mergers are bad. The data is like, on average, they’re a little bad. So you say, “Here are the reasons why we are better.” Everyone can say that, and 49 percent of them will be right.

The point is that you could run a randomized experiment with a control in which you get one group of companies to go through with a merger, and another group not to. And even if your randomization worked, and both groups were actually similar across every possible dimension of interest (itself unlikely), there still might be causally important unmeasured variables. So, what if the entire causal model was: mergers make companies worse off, except if the CEO of the acquirer was previously an M&A lawyer, in which case it makes the acquirer better off. Assume that the study does not capture acquiring CEO background at this level of detail, and that the majority of acquisitions are by companies whose CEO was not previously an M&A lawyer.

In that case, the interaction between CEO background and mergers will go unnoticed. The main effect will still be valuable — especially for policymakers and others whose business is mostly about average and net effects — but for an individual CEO considering whether to acquire a company who is familiar with the data, the question remains: what unmeasured interaction variables might there be that could apply to me?

More formally:

Confounding by unmeasured Patient Variable X Treatment Variable interactions remains a possibility.

So, what then compels an individual to accept a social science finding in the context of their own decision? Even if they’re convinced that the main causal result is true on average, what’s to keep them from coming up with some plausible unmeasured interaction that applies to them and renders the result inapplicable?

A good answer comes from Steven Pinker:

In 1954, Paul Meehl stunned his fellow psychologists by showing that simple actuarial formulas outperform expert judgment in predicting psychiatric classifications, suicide attempts, school and job performance, lies, crime, medical diagnoses, and pretty much any other outcome in which accuracy can be judged at all. His conclusion about the superiority of statistical to intuitive judgment is now recognized as one of the most robust findings in the history of psychology.

Data, of course, cannot solve problems by themselves. All the money in the world could not pay for randomized controlled trials to settle every question that occurs to us. Human beings will always be in the loop to decide which data to gather and how to analyze and interpret them. The first attempts to quantify a concept are always crude, and even the best ones allow probabilistic rather than perfect understanding. Nonetheless, social scientists have laid out criteria for evaluating and improving measurements, and the critical comparison is not whether a measure is perfect but whether it is better than the judgment of an expert, critic, interviewer, clinician, judge, or maven. That turns out to be a low bar.

The reason not to search for unmeasured interactions that might render a social scientific result inapplicable is simply that we’re not very good at it. Usually, betting the average effect will beat your intuition, because intuition is colored by motivated reasoning.

To return to M&A, the two parts of Levine’s answer are related. On the one hand, the fact that mergers are, on average, value-destroying does not necessarily mean all of them are. On the other hand, clearly one big reason lots of mergers get done is executives’ desire to do something or to build an empire. The latter is the reason it’s usually wise to ignore the former.

Another answer, though, is that this is what good judgment is all about — knowing when to bet the average and when not to. In this view, Tetlock’s forecasters know that they should usually bet the average when making predictions, but their key skill is judiciously searching for exceptions. (This sort of parallels one argument for human + algorithm teams, in which the human occasionally adds information the algorithm doesn’t have. Of course, in practice it doesn’t necessarily work so well.)

So, if a CEO proposes a merger, how do you know if they’re an unthinking anti-science empire-builder, or a Tetlockian fox? I’m not sure I have a perfect answer. But I’d say the fox begins with data, and assumes the base rate, or in this case the average effect, as the starting point for conversation. Much of the time, the fox ends there. But, across many decisions, the fox sometimes seeks to improve upon the base rate, by adding information that the algorithm (study) didn’t include, even if the causal implications of that information are uncertain or based on experience or intuition.

We can say, based on the data, that most CEOs who take on mergers are probably biased empire builders who’d have been well-advised to bet the data. But some of them are foxes, and they know something the social scientists don’t.

What makes an organization succeed?

When I covered startups and venture capital, investors would sometimes ask me if I’d see any promising new companies they should know about. The idea was that I, like them, was tasked with staying on top of the constant flow of new startups in the area, so maybe I knew something they should know. I found the question difficult, for multiple reasons. First, reporters and investors are often looking for different things in companies. Second, I was writing so many stories a day I seldom had much time to reflect on which companies I thought had promise. And third, having studied political science and worked in nonprofits prior to the job, I knew more about policy than business, and lacked a clear framework for thinking about what it was that made a startup likely to succeed.

Since joining HBR, I’ve occasionally asked colleagues to describe in as few words as possible what makes a company successful. I used my answer to that question as the lede to a piece about Snapchat:

To be successful, a company needs to provide something customers want. It must be able to do so for less than they’re willing to pay. And there must be some reason why competitors can’t just copy it when it succeeds. In management terms, it needs a value proposition, a business model, and a strategy.

Kevin Boudreau offered a similar answer in his writing on strategy and business design for entrepreneurs. In this view, companies are likely to succeed to the extent they have compelling answers to these questions:

W170830_BOUDREAU_ASKTHESE1-850x365

Since this is part of my series on thinking about organizations, it’s worth going back to my first post that lays out market-based views of organizations alongside managerial and sociological ones.

The outlines I’ve given so far are, to my mind, mostly market-based views of firm success, maybe with a bit of the managerial view mixed in.

So are there concise answers to “What makes an organization successful?” that do justice to those other views?

It’s not easy to sum up “good management” in a paragraph, but this is a nice attempt to do so, by Nick Bloom, Raffaella Sadun, and John Van Reenen:

  • Targets: Does the organization support long-term goals with tough but achievable short-term performance benchmarks?

  • Incentives: Does the organization reward high performers with promotions and bonuses while retraining or moving underperformers?
  • Monitoring: Does the organization rigorously collect and analyze performance data to identify opportunities for improvement?

So the managerial view might say organizations are likely to be successful to the extent that they do these three things well.

I have no concise explanation of how to answer the question through the sociological lens, except by repeating the common business refrain: culture eats strategy for breakfast.

The next post in this series will be about what organizations are for — what purpose they serve in society.

Notes on technology diffusion

One of the many memorable bits of Robert Gordon’s The Rise and Fall of American Growth is how rapidly technologies like radio and automobiles diffused throughout the country. But is diffusion speeding up? I won’t try to answer that question. This post is just to clip together some resources related to it.

Our World in Data has a great piece on this, with data available for download. That piece relied in part on Horace Dediu, who blogged about this here and here.

Other mentions: At HBR (with charts), The New York Times, and MIT Technology Review.

Finally, I downloaded the Our World in Data dataset, and plotted the years elapsed between the first data point for a given technology, and the year it hit 50% penetration. Lots of caveats apply. Most importantly, the first datapoint isn’t the same thing as when the technology was invented; it’s possible you’d get totally different results with that more meaningful starting point. Nonetheless:

diffusion

How are organizations organized?

Several weeks ago, I wrote about a few different ways to think about organizations: as responding to market incentives, following the direction of managers (rational or otherwise), or as dominated by a wider set of participants and their coalitions as well as by a distinct culture.

Again, the aim of these posts is to describe, as simply as possible, how to think about organizations. This post is about how organizations are structured. In the next post, I’ll write about what makes organizations successful.

Perhaps the most classic distinction in terms of how organizations are structured is between the functional structure and the divisional structureIn functional structures, functions like marketing, engineering, and finance are grouped together. In divisional structures, teams are divided up based on particular products or customer segments. Here’s how McKinsey Quarterly recounted the history of this division in 1979:

Up through the early 1950s, most companies were functionally organized. The postwar boom and subsequent economic growth led to mushrooming product lines and organizational complexity. During the late 1950s and 1960s, many companies sought to regain control and achieve “product-line rationality” by shedding their traditional functional organizations for a divisional structure based on the model initiated by General Motors and DuPont in the 1920s. For most the move proved successful; strategies became more coherent and divisional managers could be held broadly accountable for their operations.

For more on these two, and how they differ, check out Matt Yglesias’ piece at Vox on Apple and its functional structure.

Then there’s the matrix structure which blends functional and divisional structures together. In theory, it’s the best of both worlds; in practice, it’s hard to manage.

But these three are not the only structures to know about, as Nitin Nohria and Ethan Bernstein of Harvard Business School write in their note on organizational structure. (Disclosure: I work for Harvard Business Review, which is owned by HBS and sells the research note; Nohria is HBS’s dean.)

The question of organizational structure, they write, is about how to divide up work and decision-making while maintaining coordination between people and groups. There’s more to it than that, of course, and you can buy the note to learn more. They talk about functional, divisional, and matrix structures, but also include two other organizational structures.

First, there’s the employee-centric structure:

Drawing inspiration from less-architected yet highly productive forms of distributed network technology like the Internet, employee-centric (or user-centric) approaches to organizational structure let patterns of coordination emerge based on the organic networks formed by members of the organization as they do their work.

Holacracy falls in this category, and you can read Bernstein on employee-centric structures and their history at greater length in HBR. (He notes that teams in an employee-centric model can be based around functions or around products or segments.)

Finally, there’s the crowd-centric model:

If employee-centric organizations use structuring processes to define roles and authorities in a self-managed way, crowd-centric organizations build platforms upon which the collective can iteratively self-organize.

For more on this, Andrew McAfee and Erik Brynjolfsson’s latest book Machine, Platform, Crowd touches on it — which I wrote about here. And Yochai Benkler’s The Wealth of Networks remains the most definitive account of at least a subset of this phenomenon, which he calls peer production. And here’s HBR on platforms and strategy.

Of course, most organizations do not purely conform to one model or another. The trick is to combine these models in a way that reinforces the organization’s strategy. Strategy will be the subject of the next post in this series.

The pie fallacies

Too many people see the world as zero sum. They think of wealth and prosperity — “the pie” — as fixed, and so assume that if one person gets more, someone else will end up with less.

The world isn’t zero sum. (In 2004, Paul Graham called this sort of thinking the Pie Fallacy.) But way too many of the people who delight in pointing this out then go on to assume, explicitly or implicitly, that because the size of the pie isn’t fixed, the distribution doesn’t matter, or at least isn’t worth focusing on. This is just as absurd as the idea that the pie is fixed. I’ve tried to explain why before, but many pie-growers seem immune to the logic of it.

However, we have empirical evidence, thanks to Raj Chetty and colleagues. Given ballooning inequality over the past 40 years, they asked, essentially, how would the number of Americans who end up better off than their parents be different had the pie been divided as fairly as it was mid-century, vs. if the pie had grown as fast as it had at its modern peak?

Obviously, both matter a lot. But they estimate that how the pie is divided mattered more:

The researchers ran a clever simulation recreating the last several decades with the same G.D.P. growth but without the post-1970 rise in inequality. When they did, the share of 1980 babies who grew up to out-earn their parents jumped to 80 percent, from 50 percent. The rise was considerably smaller (to 62 percent) in the simulation that kept inequality constant but imagined that growth returned to its old, faster path.

The fact that the world isn’t zero sum isn’t a license to ignore the distribution.

At this point, the more ideological of the pie-growers argue that more equitable distribution will mean less growth. That is wrong, generally speaking. But it’s at least less annoying than the outright insistence that, because the pie isn’t fixed, questions of distribution just don’t matter.

The physical world and the internet

Last week I wrote:

The internet made communication and entertainment cheap, and put endless information at our fingertips. It seems likely that we’ve overdone it on cheap communication and entertainment, and underdone it on instant access to other forms of information.

One thing I didn’t mention explicitly, though, was the use of information and communication to augment or change the physical world. That was an oversight, especially because in the paper I cited in that post, the most valued internet service after search and email was maps.

My ideas for the next phase of the internet aren’t very physical, but that’s just a reflection of my own interests. Any broader attempt to think about what the internet is good for should discuss how it interacts with the physical world, so I wanted to just link to a few things that do that.

Here’s Tyler Cowen:

Many of the biggest events of 2018 will be bound together by a common theme, namely the collision of the virtual internet with the real “flesh and blood” world. This integration is likely to steer our daily lives, our economy, and maybe even politics to an unprecedented degree.

Then there’s Peter Thiel’s idea that the fact that “we’re no longer moving faster” is a sign of stalled innovation.

Finally, this Benedict Evans talk has a lot in it about the internet and the physical world.

Variety, scale, and industry concentration

Both Derek Thompson and Noah Smith have highlighted the beer industry recently, to explore the topic of industry concentration. Both pieces are worth a read, and I want to just make a couple points.

One, as Thompson notes up front, it’s notable that beer is an example of rising industry concentration; a few firms control the bulk of the industry. And yet craft breweries are also increasing their market share.

For all the attention paid to concentration ratios, oligopoly, and market power, it’s equally important to think about entry costs. And this side of the ledger seems complicated. New business starts are down relative to the economy, but growth entrepreneurship may be up. There’s some evidence that rising startup costs are holding entrepreneurs back, but also evidence that cloud computing advantages new firms.

To my mind, we still can’t quite answer the question: Is it easier or harder than it used to be to start a business? Technology probably makes it cheaper in some ways, at least to get up and running. But it might make it more expensive to become a real competitor, to the extent that scale is rewarded. And then regulation may make it more costly in certain cases, along the lines suggested by Brink Lindsey and Steven Teles in The Captured Economy.

But whatever the answer is in aggregate, the beer story raises the possibility that in some industries things may be going well for the biggest firms and well for the smallest or newest ones. That’s a possibility that ought to get more attention in the industry concentration / monopoly conversation in general. And it makes me think of Matthew Hindman’s characterization of the internet’s effect on the media industry, from his 2009 book The Myth of Digital Democracy. In it, he refers to the “missing middle”:

On the one hand, the news market in cyberspace seems even more concentrated on the top ten or twenty outlets than print media is.  On the other, the tiniest outlets have indeed earned a substantial portion of the total eyeballs…. It is the middle-class outlets that have seen relative decline in the online world.

We ought to think more about “missing middles” when thinking about industry dynamics, market power, etc. It seems, based on Thompson’s and Smith’s pieces, to be a decent explanation of the beer industry.

One last point, sort of unrelated… I was re-reading parts of Information Rules last weekend, Hal Varian and Carl Shapiro’s classic on digital economics and strategy. And it included a little two-by-two that seemed relevant to the beer example, and more broadly to questions of technology and industry concentration:

Screen Shot 2018-02-01 at 10.28.26 AM

Both Thompson and Smith, in their consideration of beer, explore the dimensions of variety and scale. As we seek to understand where the craft beer story can be repeated, this 2×2 is worth bearing in mind.

What the internet is good for

Several years ago, a Redditor posed the question: “If someone from the 1950s suddenly appeared today, what would be the most difficult thing to explain to them about life today?” The most upvoted answer? “I possess a device, in my pocket, that is capable of accessing the entirety of information known to man. I use it to look at pictures of cats and get in arguments with strangers.”

The answer itself is a paradox: Its cleverness demonstrates the internet’s potential, while acknowledging how badly we use it.

And we certainly aren’t using it well: the evidence is mounting that our internet usage is making us unhappy; even Facebook acknowledges as much. But how did we get here?

The early worries with email communication were that they’d crowd out richer interactions, like time spent in-person with friends and family. Happily, that didn’t happen, or at least not at first. The more time people spent on email, the more time they spent socializing in person and on the phone. To make time for email, they watched less TV.

The question, then, is how we went from this happy situation where the internet enabled communication and crowded out TV, to one where social media and other forms of screen-time seem to be making us miserable. I don’t have an answer, but I want to offer a few ideas.

The first is that there are, at least, diminishing returns to more communication, and quite possibly a point at which more communication is actually a negative. In this view, email was the key advance in communication technology, and everything else — instant messaging, texting, social media, etc. — has been either only a very marginal improvement or even detrimental. There is some evidence for this view. When you measure how much money people would require to give up a given technology, email is far and away the most valuable among communication tools. Social media and messaging are worth 2-3% as much as email; Facebook is less than 1%. (pg. 21)

This seems consistent with the hypothesis that we simply don’t really benefit from making it easier to communicate with each other, beyond some point. It may be that by the time we had email, we more or less had the big benefits we were going to gain from cheap communication.

Another theory would be that, whereas early internet communications were a substitute for TV, much of today’s “communication” technologies like social media are more or less a TV equivalent. In this view, we’re spending more and more time on media, much of it mindlessly entertaining ourselves watching videos and scrolling through photos.

These are just hypotheses. I don’t know that I believe either one of them. But in any case, the Redditor’s point seems right: we don’t really use the internet very wisely. The internet made communication and entertainment cheap, and put endless information at our fingertips. It seems likely that we’ve overdone it on cheap communication and entertainment, and underdone it on instant access to other forms of information. Far and away the most valuable internet service, again as measured in how much money people would require to give it up, is the search engine. It’s more than twice as valuable as email, the next most valued service.

When I think about what the next phase of the internet should look like, the Redditor’s joke serves as a decent guide. We don’t need more ways to argue with strangers nor do we need more cheap entertainment. We have a universe of information at our fingertips. We need more services that use it to make us better off.