Category Archives: management

Team management in large organisations

Recently I’ve been asked to help some design teams in the bbc deliver better work faster.

This got me thinking.

I’ve spent a few years reading about management approaches and more recently have been working as part of the product management team on iPlayer radio. This involved me being responsible for day to day work in one of the teams, as well as being involved in strategy and prioritisation discussions – all great fun.

During my work with the iplayer radio team we tried a variety of approaches from agile, lean and kanban schools of thought.

As you might guess, I have realised there is often no quick fix, no magic bullet.

Hey, you know, like good management is hard!

It is likely that a particular team with a particular set of skills, will need to optimise itself in a unique way.

How do you know you’re making a good decision?

I believe that company values should help you make good decisions. Looking around, many company principles do not actually help you manage a team. Recently created companies like Facebook and Google may prove better at having company values that you can apply to management choices. “Don’t be evil” – right then that’s option b off the table, “move fast and break things.” – don’t worry too much if option d does break things – we will fix it, but it should help us get somewhere useful fast. The BBC’s “trust is at the heart of everything we do.” – helps make decisions, but also makes the leadership risk averse – will it work all the time, can the user trust it? I suspect that value was intended for content, but it is applied across the organisation – is it encouraging the right behaviour for software development? When we are also told to ‘optimise for speed’ that feels like 2 values that can conflict – this is something the BBC should discuss, since conflicting values make decision making harder, slower, and make ‘the right option’ less obvious.

Many times corporate values don’t help with decision making – “creativity is the lifeblood of our organisation.” (Bbc again, sorry) That value isn’t really going to help me make daily decisions, but it might help with longer term decisions about team membership and recruitment.

My theory remains: a few guiding principles for a project should help make quick good decisions. Determining those values may be different for each team. Likewise determining how to apply those values and how much you can apply them will vary from team to team.

My current values to help me make decisions.

Here are my current values that I try to apply on the projects I work on.

1. Optimise for speed, particularly “don’t fall in love”

“Don’t fall in love” comes from Facebook, and means get feedback from the customer within 2 weeks of your initial idea. Any longer than that and you’ll risk being in love with your idea, and therefore a little resistant to changing it based on user feedback, so move fast.

2. Optimise for learning

This is from lean management , and also peter drucker -,your team should learn as much as they can to do their jobs better.

3. Colocate the team members whenever possible.

This is my experience – multi discipline teams that are physically close are more likely to minimise communication errors and collaborate more often and more successfully.

4. Encourage open honest collaboration, no egos and a ‘one team solving problems together’ attitude.

5. Delegate responsibility. Optimise signoff processes and people.

This is really a repeat of Optimise for speed, but in large organisations the decision making is often slow, as :signoff’ involves many senior (overworked) managers, so focussing specifically on this should help the team move faster.

Reviewing progress with retrospectives

Hopefully the above principles help any manager in a large organisation. As a practical process, having used them to run a team of engineers and designers, I’ve found agile team retrospectives immensely useful (even when you’re not a particularly agile, lean or kanban team). Asking the team to focus on ‘What can we do to optimise our performance?’ has been great. Letting ideas for team optimisation come from the team is also part of applying the principle of empowerment. Applying the teams values to this conversation allows us to ensure that the teams behaviour changes in the right way. Suppose the team believes they should improve quality….If a team believed that they should introduce more process to achieve higher quality, that would probably slow down the team, so applying ‘optimise for speed’ makes the team focus on ‘is there a way to improve quality whilst optimising for speed? These values should alter what the team chooses to do. Equally, if the values are useful they can free management up through not having more decision making escalated to them.

What values do you use?

So those values are my starting point, but what values do you find useful? Tweet me @songschris or email chris@chris-thorne.co.uk.

Return on investment in ux planning

I have recently completed an attachment in the BBCs Radio and Music team as a product owner.

The BBC invests heavily in training, and I have greatly enjoyed learning and practicing kanban, scrum (agile), and release planning techniques.

One of these techniques was release planning using return on investment (ROI).

Assigning business value

In the ROI technique, the set of possible tasks are put on tickets – one ticket for each task. Each ticket is first ordered by business value. The most valuable at the top, the least valuable at the bottom. The value of each ticket of work is recorded on the ticket.

For example, here are some (made up) tickets, for a site that already exists, but is being rebuilt:

  • re-Branding of the whole site to give it a consistent feel.
  • New horizontal navigation bar to give users access to significant parts of the site.
  • Revamped search with autocomplete and search results page.
  • New homepage redesign.
  • Content management system redesign

Each task ticket is then given a number for its business value, the highest number at the top of the list of tickets, the lowest number is at the bottom.

In conversation with our stakeholders, we might decide the following business values. These values are made up during that conversation – it is not the value itself but the relative value when compared to other work, that is important.

  • New homepage redesign. (value=250)
  • Revamped search with autocomplete and search results page. (value=150)
  • New horizontal navigation bar to give users access to significant parts of the site. (value=100)
  • re-Branding of the whole site to give it a consistent feel. (value=25)
  • Content management system redesign (value=10)

ROI Release planning lets you answer:

Which bits of work will have high value to the business and be easy to complete?

We can see that the stakeholders really want a new homepage, and also really value search. They clearly see some value in a navigation bar across all pages of the site, and a limited value to rebranding the whole site. The content management system is considered a necessary cost, but if they could get away without having one, that would be fine by them.

Technical complexity using poker points

Now the tickets are re-eveluated by an experienced technical member of staff (usually a tech lead).

Each ticket is considered for its complexity – we can use planning poker points or any other approach. So, the complexity of a ticket now has a numerical value.

The numerical complexity is now recorded on each ticket.

In the case of our made up list, the list is now:

  • New homepage redesign. (value=250, complexity = 21)
  • Revamped search with autocomplete and search results page. (value=150, complexity = 21)
  • New horizontal navigation bar to give users access to significant parts of the site. (value=100, complexity=5)
  • re-Branding of the whole site to give it a consistent feel. (value=25, complexity=13)
  • Content management system redesign (value=10, complexity=13)

Points to note: Don’t worry about the numbers too much. The value of this process is that it results in an ordered list of tickets which can be discussed, not in the exact number assigned to each ticket.

Return on investment is relative

Return on investment (ROI) is now: Business value/Complexity.

The ROI can now be recorded on each ticket.

The backlog of resulting tickets can now be ordered by ROI.

In our case this gives us:

  • New horizontal navigation bar to give users access to significant parts of the site. (value=100, complexity=5, ROI=20)
  • New homepage redesign. (value=250, complexity = 21, ROI = 11.9 )
  • Revamped search with autocomplete and search results page. (value=150, complexity = 21, ROI=7.1)
  • re-Branding of the whole site to give it a consistent feel. (value=25, complexity=13, ROI=1.9)
  • Content management system redesign (value=10, complexity=13, ROI=0.76)

For large groups of tickets, its worth subdividing the group of tickets into smaller ‘feature’ groups. This will help answer “what should we do?”

The tasks in this roi list, with the highest roi number represent your best guess at what is valuable to the business and easy to do?”

 

Reusing the ROI technique in UX planning

The really exciting thing about this approach for me, is that it can be reused in a variety of situations. Particularly, it can be used in planning UX work. For example, we can replace ‘business value’ numbers with ‘user value’ (how much will the user value this feature?). Likewise our estimate of complexity can be based on just the time required to complete the ux work required, or ux+tech+test staff estimates.

So, as a creative director or head of ux, you can now look at a slate of work, and this ROI technique lets you look at the slate in a new way – which bits of work will have high value to users, and be easy to complete?

 

Release planning

Having got a list of tickets ordered by roi, you can now plan a release, by considering a group of tickets you are going to complete for that release.

Using your roi list order, you can see which tickets represent good value.
Some tickets of work will be high value, high cost (therefore limited ROI), and still be worth doing. In our made up slate, the item:

  • Content management system (CMS) redesign (value=10, complexity=13, ROI=0.76)

has very limited business value, since the stakeholders view it as having no benefit to the business (unlikely, but lets suppose). However, if the new homepage cannot be built without a new CMS, the CMS would have to built as well, even though it doesn’t represent a good ROI.

Planning a ux slate using roi

This ROI prioritisation technique can be reused with different axis (user value replaces business value) and it can be applied without having perfect estimates. If we replace business value with user value, we can use qualitative (eg a card sort with sample users) or quantitative (eg traffic data) to find user value.

The list of work ordered by ROI is showing you which tasks look like good valuable work, and which don’t.

For me, it feels like a good technique with which to have very constructive stakeholder or management conversation.

An exciting framework for product management and ux

I’ve been reading 2 interesting books recently.

“The Lean Startup” by Eric Ries is a great book about how to do product management for a new product. In his book, he talks about the development cycle: Build Measure and Learn.

BML

An illustration of the cycle of Build Measure Learn
– Build, Measure Learn. A new products goal is to repeat the build measure learn cycle as quickly as possible. Build a new feature, Measure the user response to that feature – did they want it? Did they use it in the way you expected them to?

If you measured the right things, you’ll know more about what users want. For each feature you can now improve the feature , keep it as is, or discard it. As you learn more about what users really want from your product, you are better able to predict which new features to build.

The quicker you successfully complete a build measure learn cycle, the closer you are to having a successful product.

– Your assumptions about what the audience wants are probably wrong. What a relief! Seriously, it becomes easier to reach agreement between stakeholders, if you able to admit that nobody knows what the right feature to build is. We can hopefully make an educated guess, but that is all. Until its built and you see users using it, no-one knows for sure.

I know from my own experience that a design always changes following user testing. You think you’ve delivered a great design, and there’s always something that needs to improve because users don’t want it, or don’t know how to use it.

Eric Ries recommends a book called ‘The Other Side Of Innovation‘ (written by Vijay Govindarajan and Chris Trimble) which is a great book for those of us trying to improve innovation within an existing (large) organisations.

In ‘The Other Side Of Innovation’ the authors convinced me that in order to complete the measure and learn part of the cycle you need to create ‘hypotheses of record’.

School science projects taught us about hypotheses, so this is quite an easy concept to understand.

Creating a hypotheses of record for a product

A hypothesis for product development needs to be focused on assumptions about user behavior. It also needs to be derived from the core business goals. In many businesses, the business is trying to provide a product or service that users want. By providing something that users want, the business can sell more products or services to the users. The business goal is to get more revenue and profit, which it achieves by selling more products and services to users.

In the BBCs case, in my opinion, the business goal is to increase audience reach and engagement so that when a user is asked to purchase a TV license (which funds the BBC) they feel it represents excellent value for money. There are of course additional aims, notably to educate, inform and entertain, but without achieving reach and engagement, the organisation cannot hope to achieve these subsequent aims.

An example hypothesis for the new BBC homepage might be:

Hypothesis: Given the user experience of the new BBC homepage we expect most users to use the carousel buttons at the top of the page.

What does success looklike? If, with time, more visitors (as a proportion of the total visitors) to www.bbc.co.uk are using the carousel buttons then the design works as well as we hoped. As a subsequent success we would expect to see more clicks through to items within the carousel. Each click through to an item in the carousel represents an increase in engagement. If we see a low proportion of users or a decreasing proportion of users engage with the carousel over time, then we think the carousel design needs to change.

Other parts of a products design can similarly be considered – For a particular feature, what is the user behavior we expect, if that feature works? Which of our goals is this feature going to improve? If we can’t agree with a stakeholder about which business goal this feature will improve, perhaps this feature isn’t as needed as we thought it was!

Writing down a set of hypotheses and how each hypothesis will be tested, forms a “Hypotheses of record”. Once a hypothesis is tested, and the metrics obtained, then we can check our records to see if our hypothesis is correct.

Priorisation of hypotheses

Some of our assumptions are high risk and high cost (ie we don’t know if the audience want this new feature, but in order to deliver this feature it costs a lot of money). These high risk high cost features are the hypotheses we should focus on testing first. If the hypothesis about the user is wrong, we’ve spent less money developing it if we test it sooner.

[Testing a feature can be just interviewing users with a paper illustration of the feature and asking them if this feature is something they would use? Would it make them likely to use the product more often?]

Hypotheses of record in my experience

Recently ive been trying to apply this in my work (building the new bbc.co.uk/radio).

The combination of the ideas above has been very interesting.

In my experience, the conversation between stakeholders and development team can get stressful – particularly when it comes to agreeing requirements.

Typically person A insists that they need this feature, and person B insists that they don’t need that feature, and there is very little data with which to make either case.

Either person might persuade the other that their case is valid, and therefore agree the new functionality required.

However, the conversation and persuasion can be hard, complex and slow.

A “Hypotheses of Record” allows us to complete our measure and learn parts of the Build Measure Learn cycle: We can:

  • Deduce what we need to measure for the hypothesis we are testing.
  • Agree what success or failure results look like. We’re not going to base our decisions on a snapshot of results from our testing. We are looking for a trend in results over time.
  • Learn from the results through proving or adjusting our hypotheses.

In the BBC homepage example, we might see only a few users navigating with the carousel when it launches, but if the number of users navigating the carousel is increasing significantly with time, we might say that the new carousel feature succeeds as a design. If we see the opposite trend, we might say that the carousel design is not working.

Each time we prove or disprove a hypothesis we have more accurate information, which helps us decide what features users want, and therefore what to build next.

This framework focuses stakeholders and product management on the assumptions in the current approach, and the desired user behavior if the product is a success.

These are very very powerful points to focus on – in my experience they shift the conversation from a sometimes delicate negotiation, to a more easily agreed route forward, where testing and evidence from the audience will be used to determine improvements to the product. When stakeholders and product management disagree about a feature, we can still agree the test and the user behavior we want to achieve, and therefore find a positive way forward.

Its still early days in applying this approach, but trying to “Build Measure and Learn”, and creating a “Hypothesis of Record” is proving very valuable in focusing effort and gaining agreement between product management and stakeholders.

What makes a good user story?

I have worked in many teams who define product and project requirements as user stories and acceptance criteria. What makes a good user story?

As a User I want to listen to a documentary programme from Radio 4 on my PC, so that I can increase my understanding of the subject of the documentary.

As an example, here is a bad user story:

I want to click on a ‘buy this’ link that adds the item to my shopping basket.

One of the easiest ways for us to assess a projects success during development is by using user stories and acceptance criteria. However, user stories seem to be very dependent on who has written them.

In my opinion, a good user story has the following characteristics:

  • It defines Who needs this requirement met – for example the Product Manger, a Stakeholder, or the User.

If the team don’t know who the requirement is for, and we want to find out more about this requirement, who should we talk to? Knowing who is important, to help us get further understanding.

  • It defines what is supposed to happen.

This is the surface requirement. Ideally there is some evidence, research or analysis to support this requirement. You might add this research as supporting notes for the story.

  • It defines why it is supposed to happen – what is the benefit?

Defining why a requirement is important (the ‘So that’ part of the story) increases understanding of the underlying need, and can greatly help innovation.

For example, we know that in the above ‘radio 4 programme’ story, the goal of the user is to understand the subject better. Perhaps on the web, this can be accomplished in other ways than ‘listening to the programme’ (a solution might be simply providing some related links for the programme). If we just have 1/2 the story – eg ‘As a User I want to listen to a documentary programme from Radio 4 on my PC’ , without expressing why this is important, we miss the opportunity to innovate.

To further encourage innovation, I also try to remove any reference to how from a user story. Supporting notes for the story might suggest:

  • When it is supposed to happen.
  • Where it is supposed to happen.
  • How it is supposed to happen – this is just an idea, but might be useful as inspiration for the team.

The keyword here is ‘suggest’. If we dictate the How and When and Where then we have weakened the project teams ability to innovate. If we were working on a factory production line, a lack of innovation can be useful (although Toyota seem to have done quite well through innovating on the production line), but if we’re knowledge workers working on a project, one of the outcomes should be some innovation.

Improving user stories

‘I want to click on a ‘buy this’ link that adds the item to my shopping basket.’

The above is a restrictive user story. Lets try and improve it.

Start by adding who it is for:

As a user I want to click on a ‘buy this’ link that adds the item to my shopping basket.

Now we know that if we want further information about this story, we should talk to user experience staff, or audience research staff. They may have further research that explains this story.

Now we can remove the how from the story. In this case the ‘buy this’ link is defining how the story is to be implemented. Removing this we end up with:

As a user I want to add items to my shopping basket.

This shopping basket idea might be well understood, but we would do well to define its behavior better, just to be sure we will get what we want. The shopping basket concept has several features that are worth considering separately.

The underlying function of adding items to a shopping basket is to collect all items I want to buy in one place, so that I can review all my proposed purchases and pay for them all at once.

As a user I want to collect all the items I want to buy, together in one place, so that I can review my proposed purchases and organise payment and delivery details for all of my shopping just once.

If we add a note to this user story ‘The one place that collects all items together might be a ‘shopping basket’ then we’ve explained our idea, but also left it open to design staff to think about other solutions that satisfy the same story. For example, they might think about storing previous payment and delivery details, so that repeat visits to the site are even easier.

The other aspect of the basket is being able to add, remove and change the quantities of items I have collected, before I purchase them.

As a user I want to be able to add, remove and change the quantities of items I want to purchase without journeying back to each individual products page, so that it is easy to complete my shopping.

So now we have 2 user stories, but a clearer definition of the basket functionality, and a clearer definition of the underlying needs of the user.

Conclusions

Great user experiences (I’m going to say it, sorry) for example, those delivered by Apple, are partly successful because they have worked hard to answer not just the surface requirement, but the underlying need. Great user stories are the foundations on which experiences are built. Making sure those stories are solid, researched, and prescribed appropriately for the project aims is very very important. By making sure that a user story defines who, what and why we have the basis of a good story.

Keeping how out of the story gives a project important room to innovate.

Thanks for reading – let me know your comments.

The 6ws – what are the right questions?

I’ve been thinking about the project process recently. If we ask the wrong questions, we get the wrong answers, so what are the right questions?

How often do we check – is the right problem being solved? Do we know enough about the problem to solve it well?

I’ve recently read Gamestorming – an awesome book of exercises to fire your creative juices. Similarly, I’ve found Dan Roam’s ‘The back of the napkin‘ inspiring.

Both books include an exercise ‘The 6Ws’ which I think should be applied in the initial phase of any new business project or proposal.

What are The 6Ws ?

The 6Ws are: Who, What, When, Where, Why and How. For a given goal, each of these ‘W’s is the starting word of a question. Crucially, questions created for each of these W’s cannot be answered with a simple ‘yes’ or ‘no’. Investigation of these questions leads to a better understanding of the goal, and the issues surrounding it.

Example problem: The carrot shop.

Suppose we have a new problem to solve, for example a shop sells carrots and asks us ‘How can we sell more carrots?’.

By asking questions beginning with one of the 6 Ws we can start to explore the problem. Is this the right question?

The below list took me 5 minutes to produce. By working quickly, we can focus on what the obvious questions are and hopefully therefore the important questions as well.

Carrot Shop: How can we sell more carrots?

Is this the right question? Let’s apply the 6Ws and find out:

Who visits the shop?
Who works in the shop?
Who delivers the carrots?
Who grows the carrots?
Who buys carrots?
Who could buy carrots that currently doesn’t?
What are the features of a carrot?
What types of carrot do we sell?
What types of carrot can be grown?
What do people use a carrot for?
When do carrots grow?
When do carrots get planted?
When do people buy carrots?
When do we receive the carrots?
When do we sell the carrots?
Where are the carrots grown?
Where are the carrots stored?
Why do buyers buy carrots?
Why do we want to sell more carrots?
How do carrots get grown?
How do carrots get prepared?
How do carrots get used?
How are carrots presented for sale?
How are carrots packaged?
I think the quality of business thinking about a new product or feature could be greatly improved by applying the 6ws at the earliest possible stage of a project. I suggest the 6Ws should be applied at the start of a new feature exploration, and before any user stories.

Just answering the questions raised forces the business to think critically about what the problem is they are trying to solve. Having asked these questions, does the problem still look like it is the best one to solve? Or does it suggest new, better opportunities?

Is ‘How can we sell more carrots?’ the correct question? In the Why section of the above, we ask ‘Why do we want to sell more carrots?’. By asking this basic question, we might well get the answer ‘to make more profit for the store’. So, perhaps a better question would be ‘We are a shop that sells carrots. How can we make more profit?’. Again, the 6Ws now suggest lots of ways to answer this new question.

As an example ‘What is a carrot?’ might be answered by ‘it is orange’ ‘it is a vegetable’ ‘it is a food used in cooking’ ‘it is a prop used at christmas to make a snowman’ ‘it is food for a pet’. So, new product opportunities – can we sell other vegetables? can we use or sell carrot recipes in store? Should we promote it at christmas as a thing to use in your snowman? Can we sell to pet owners?

Why I like the 6 Ws

  • Its a framework that is old

There are records of this framework being used since 1st Century BC in Greece. Which suggests to me it has some value, if the framework has survived for this length of time.

  • It feels like a solid set of lenses with which to review a stated goal
  • It is as flexible or rigid as you need it to be

If your organisation, or a part of it, needs to ask specific questions, you can use the 6Ws to define those questions.

If you’re exploring a fuzzy goal, then you can start with each of the 6Ws. Given a particular W, what are the useful and interesting questions?

  • Its useful when being strategic or tactical.

If you apply it to a tactical goal, it works, or you can apply it to define strategies.

A general set of the 6Ws for a project might be:

Who is going to do the work?
What are the roles and responsibilities of the people involved?
What does success look like?
What work is required, to achieve the project goals?
What does this project depend on?
When will the work be done?
Where will the work be done?
Why are we doing the project?
How are we going to do the project?
How can we measure success?
How does this project fit with our strategic objectives?
I’ve seen these questions on every project I’ve taken part in, but some of the projects that have had problems have not had clearly defined answers to these questions.

  • It can be easily understood and applied by teams with a variety of disciplines.
  • It can be managed.

If the process of the 6Ws results in too many questions, a manager can prioritise the questions.

The most exciting reason for applying the 6Ws:

Using the 6W framework leads to better, and faster, understanding of the issues involved to achieve a goal.

A well defined goal should be able to answer questions for each of the 6W lenses. Once a goal is well defined, a clear supporting document – for example writing a creative brief, or a strategy document, or defining business objectives, a project plan, or clear user stories and acceptance criteria becomes much easier.