Visualizing High Performing Enterprises

Posted by | Uncategorized | No Comments

In my previous posts, I talked through some of the interesting findings from our research at Workfront around analyzing 600M hours work getting performed across tons of different verticals. We found some interesting stats like the avg. number of people on a project is 5.4 or that 76% of work created and completed comes from a process.

We took it another step further to better understand what it looks like to be a high performing enterprise. Our mission at Workfront is to help companies operate better and to make it so that people know their work matters. To do this, we need to see what it looks like to be a high performer. We know what the numbers look like but that only took into account a couple of variables.

Our work primarily focused on how we can accurately predict whether a project would complete on time. We extracted a handful of features that we had high confidence on, created a training data set, then ran the numbers through an exploratory machine learning pipeline. This included models such as neural nets and SVMs which helped us figure out which model could consistently deliver us predictable completion dates. It’s worth noting that we had to do a lot of data normalization and transformations in order to make the numbers work out. For example, it’s hard to visualize a process that takes only 5 days to complete versus a process that takes 500 days.

What we found was like any data exploration effort: the data tells a compelling and actionable story. We found that a best-fit line that was equal and the “x” and “y” axis through our visualization represents a fairly optimal work process. When the line skewed higher on the y-axis and less on the x-axis, companies we’re dramatically underestimating the effort required to get work done in their processes. Conversely, if the line skewed lower on the “y” and higher on the “x”, it means that the organization was dramatically overestimating the complexity of their work and that they could take on more work should the learn to optimize their process.

1 – This means that users had high accuracy of their plan vs. actual minutes and our model was able to accurately predict project duration. This is the ideal state.

2 – This means that users underestimate how long projects will take. They take on too much work, rebase their delivery dates, or have poor scoping practices. This is a non-optimization process.

3 – This means that users over estimate work efforts and have not gone back to optimize their processes. The org “looks” busy but can actually take on a lot more work. This is a non-optimizated process.

Below are a couple of high performing and predictable organizations.

When we visualize organization that we can readily predict their delivery dates, we get an interesting pattern. The tighter the scatter plot, the more consistent and predictable the organization. Curiously but not necessarily surprising, we also found that these organizations completed an order of magnitude more work than organizations with a wider scatter plot.

Don’t you love it when you find little gems like that in your data? 😉

Below are a couple of low performing and unpredictable organizations.

You can easily see that the data points have a much wider distribution which means that we weren’t able to get accurate project completion date predictions. It’s also easy to see that there is a significant less amount of work items being completed.

Now, one flaw with saying that the amount of work an organization can complete due to a process is that not all work is equal in complexity and size. That said, I would still say that this is a good leading indicator that there is a decent amount of truth that shows these work items are close in size. We found that the top 10% of processes that generated 80% of the work had an average of 12 tasks per process. Qualitatively, we also know that many organizations use their processes for fairly similar use cases (Eg. marketing campaign, product prototype, etc.).

We’re doing a lot of fun data explorations such as this in order to better understand user behavior and how to contour our software to augment users to be more efficient. It’s a journey through the data that requires a lot of time and a love of the labor, but we continue to find fascinating aspects that drive our product roadmap into new areas.

Data from Analyzing 600M Hours of Work

Posted by | Uncategorized | No Comments

Part of my job blends the qualitative and quantitative analysis of customer behavior. Either route can kick start a hypothesis that sends me down a rabbit hole. I’m a data nerd and love diving deep into the numbers so this most recent endeavor has been fascinating to me.

It all started with a theory that my colleague and I had: people don’t manage projects, they manage process. It’s a simple statement but is hugely profound when it comes to organizational change. Managing projects implies managing the activities within a specific timeframe. Managing process implies managing the deliverables and outcomes that run through the process, unblocking efforts to produce outcomes.

We spent a lot of time onsite with some of our largest customers and started to hear similar ways of them bending our system. You know you’re onto a fundamental shift in behavior when your users rigorously use your product for something that you didn’t intend for them to do. These customers use highly repeatable processes to get a significant amount of their work done. They leverage process because it helps with optimization since you have clear stage gates where blockers can be tackled.

We also heard this from industry analysts too. The largest companies in the world are shifting away from a typical PMO model to a more agile (little “a”) process driven model where the key output is deliverables that they measure a change in outcome against (eg. Did this deliverable drive a difference in outcomes). These enterprises are having to get really good at this because they are getting disrupted by smaller companies who can blitz their strategy better than they can.

We found this discovery fascinating and decided to dive deep into the data. Our system has tracked over 600 million hours worth of work, so we have a good sized data set that we can rely on for significance. Here’s what we found.

75% of all the work in our system comes from repeatable processes. Specifically, for our enterprise customers, that number is higher at around 80% on average. There are certain industries (eg. creative, finance, legal) that have even higher numbers at 90%+ of their work being process driven. The average number of unique processes an enterprise has is ~100 (ones with quantifiable usage).

The average process has 12 steps. Even more than that is that over 77% of the work performed in these enterprises comes from just 10 processes or less. I’ll note that these aren’t just small companies either. These are your Fortune 500s with thousands and thousands of users in our system on a daily and weekly basis. This last fact just blew my mind. You’re telling me that 77% of the work being performed (on paper) comes from just 10 or fewer processes? Crazy.

Some other interesting heuristics we found were around working behaviors. SMBs typically have activity lasting between 7am and 4pm. Enterprises between 8am and 6pm. Government between 9am and 4pm. The highest levels of activity are Friday (closing work out) and Sunday (prepping work for the week).

These were fun tidbits of data that we discovered through the process. For those in the software world, it would make sense that project management is dying. Our world doesn’t really recognize that method of work. However, for the rest of the world, this is a huge shift that has been caused by the disruption of software. They’re taking a queue from the software world on speed and agility, trying to inject that into their own organization.

This is an active journey so I suspect that I’ll have many more data points that will come to play. I’ll make sure to update them here as we dive deeper.

Enterprises Don’t Care About Planning

Posted by | Uncategorized | No Comments

Acme Enterprise is the world leader in designing robots for manufacturing plants. They are viewed in the world as the gold standard and the ones that “have their shit together” when you work with them. They release a new robot every year or two that is generally an upgraded version of the previous one with some new bells and whistles. The teams building these robots, from concept to prototype to production, have good velocity and care deeply about their work.

Sounds like a company that is performing well, right? Here’s the other part of the story.

Acme Enterprise is under constant fire from the public market for missing their earnings per share and having a lack of control over their operation costs. They are getting disrupted by cheaper and pointed robotic solutions that move the needle farther for their customers. There’s intense market pressure to move to that direction but Acme Enterprises constantly is late in delivery their robots. That’s not to mention the fact that releasing a new robot every year or two is creating a huge disadvantage for their sales team to compete against. Sales are slowing and everyone is frustrated. No matter how hard they plan and try to execute against these plans, they always fail and have to re-baseline their delivery dates.

I can tell you this now: every damn enterprise I go and talk to that once was but isn’t now is facing the above problem. In general, everything feels like it’s going well but the reality is that they are struggling significantly. This is the “digital disruption” everyone talks about. In my opinion, digital disruption is not about whether or not you have the latest marketing personalization tech from Adobe. It’s about whether you adopt software-minded principles of development and agility to compete in an age where technology drives change at a 10x rate than we historically have seen.

One of the key points, and the general point of this post, is the sentence around Acme Enterprise doing everything they can to plan for confidence only to have the plan blow up. This follows the old adage of “no plan survives contact with the enemy”, in which the enemy is the reality we live in. When we plan, we do it often in isolation, our subjective bias, or we try to account for mishaps only to find out that we experienced none of the mishaps we planned for but all of the ones we didn’t.

One of the biggest trends we’re seeing happen in enterprises is that everyone is moving more towards an agile work methodology backed with some sort of goal structure, such as OKRs. When I say agile, I mean this in more of a little “a” where it’s not a dogmatic state. I’ve seen this in marketing departments, engineering, corporate communications, IT, and even legal! Everyone is starting to adopt a more flexible framework for the sole fact that it provides more freedom. This goes into a longer topic that I’ve written about previously around experience owners and enabling autonomy of oneself and teams to of ownership. Agile work methodologies provide just that.

The tricky thing here is that we do need some sort of structure for specific communication and delivery purposes. It’s foolish to say that we can all live in the wild west where we can do whatever we want and when (although it would be amazing!). Thus, we get to planning. We plan like crazy in order to create predictability but see a stark failure of it when we hit reality – only to be further amplified with a new agile way of working.

Historically, this is where Portfolio and Project Management uses to come heavily in play. These disciplines helped “tame the chaos” and kept teams moving along the right track with issues tracking and risk adjustments.

News Flash: Project Management is Dead.

Yep. It’s dead. Gone. Moving forward, it will no longer be something we value and here’s why:

In the new agile and modern way of working, planning only needs to be good enough.

This means that we need our plans to be flexible and completely shift our focus away from delivering a project to delivering on deliverables that change outcomes that our customers experience. That’s the key shift here. Outcomes aren’t time bound and imply an ongoing experience optimization that we continue to invest in. This is the fundamental change in where rigorous planning is going away.

This isn’t to say that planning isn’t valuable. In fact, quite the opposite. I believe that planning will shift towards a more flexible framework where we think through different scenarios and think more in probabilities than absolutes to gain confidence instead of accuracy around what we’re doing. From there, we move forward with the notion that we are, let’s say, 70% confident in the deliverables and dates with the remaining 30% getting structured, informed, and backwards propagated to our plan to achieve a “100%” plan.

I hate even saying 100% as well because, in an outcome and experience-driven world, 100% is a goal post that always shifts. I often times believe that we should instead find more effective ways of measuring a diminishing return factor, which is to say that we may continue to deliver new outcomes and experiences up until our ROI score is <1.25x or something. This way, tradeoff conversations around resourcing, focus, and alignment are forced to happen in a more objective and quantifiable way.

If we come back to the reality of what I’m seeing in the market, we have just crossed the chasm of Fortune 5000 adopting this sort of mindset. This will be a huge fundamental shift for these corporations as they’re forced to compete with more nimble startups and business as well as a generational workforce that has an extremely high propensity to change direction or rip out software on the turn of a dime. I believe that as we move more into an agile world, the above becomes the truth for everyone.