Monday 19 July 2021

Setting Challenges to Drive Innovation

Establishing an innovative culture has grown to be something of a holy grail in modern technology companies, with the goal of encouraging innovation a constant presence in white paper reports on CEO priorities. Yet building an innovative environment is not as easy as it looks - in fact the challenge of "fostering an innovative culture" is such a problem that it was cited as the top barrier in this survey on innovation.

While the mantra of Facebook may have once been "Move fast and break things", few of us have the luxury of unconstrained innovation with limited responsibility (the 'r' word even caught up with Facebook eventually) and the conflicting pressures of desire for innovation and a need to deliver operational concerns can be unhappy bedfellows.

For most companies the idea of completely unconstrained innovation is a difficult one to sanction

  • Stakeholders want to see value. There may be some nervousness around allowing a team to go 'off roadmap' and so some level of direction is sensible in early experiments to provide reassurance that the team isn't just having fun
  • Time is money and as a technology leader you don't want to be seen as wasteful
  • Developers love technology and a big risk in allowing open innovation is an output that is little more than technology for the sake of it because someone wanted to try out a new toy

So how does a leader take steps to encourage innovation in their development teams without running the risk of a pointless blockchain integration? The best way I've found to introduce a more innovative culture, whilst maintaining a structure focussed around clear business value, is to frame innovation efforts through setting challenges.

A Multidimensional Problem

A few years ago I was facing a tricky problem. It was in the early stages of creating a new employee engagement product that, if done right, would replace a number of bespoke existing platforms and form the foundation of future company strategy. What was apparent from examining the existing systems we were targeting was that each had it's own highly customised organisational hierarchy that had been built into the database schema.

  • Most systems were based on a fixed database table hierarchy with a defined organisational levels specific to the management tiers of that organisation
  • One organisation had much more complex hierarchy (to the extent that on presenting it to us the company representatives gave up three layers in and wrote 'Here be dragons' underneath) so a looser team structure had been implemented, but this was so loose that identifying membership of lower level groups was extremely challenging due to the nested structure and aggregated reporting was a frightening prospect.

The CTO advised me to implement an arbitrary number of fixed hierarchical levels and try to map all customers to that. This was at least a straightforward approach but even some existing customer lists would have struggled to map into it and it didn't feel like the best starting point for an important technology platform.

I decided to present the challenge to the team as an opportunity for innovation. Initial research spikes pulled together info on all of our existing system hierarchies, and I then presented these back to the team with the challenge

Establish a system that can support all of our reference hierarchies while still allowing a performant way to identify members of each group and aggregate query reporting

Now inspiration rarely comes while staring at a problem, it usually comes later when performing routine tasks and allowing your mind to wander to make the necessary connections. So it was that an answer came to me while walking back from collecting a takeaway meal. I wasn't necessarily expecting to solve the problem myself so I excitedly mocked up the data structure in SQLServer to confirm that it worked and headed to work the next morning rather pleased with myself to be able to present a solution to the innovation challenge. As soon as I got in the team were at my desk eager to explain how they had (also) hit upon a solution. I couldn't help a wry smile as they sketched out on the white-board the exact model that I'd mocked up the night before.

The result that we'd independently landed on was a simple, powerful and flexible model that provided a solid basis for building the product user hierarchy around and has allowed the platform to support a wide range of blue-chip enterprise company structures.

Making it part of the process

By way of a more recent example - at the end of each quarter in my current organisation we devote a week to innovation. While the CEO is keen to encourage an innovative culture there was an initial concern around making sure the work was valuable - it is after all a small company and development time is precious. Setting challenges has been a great way to frame these sessions to ensure the team can input ideas, whilst at the same time driving towards a genuine business problem uncovered by our user research.

On the first occasion the challenge was to improve the initial impressions of a user hitting the landing page of our web application. We collected ideas not just from R&D but across the entire business and then combined these into a summary page with a truly novel communication recency visualisation.

In the most recent innovation sprint I set the challenge of making it easier to provide reporting to a senior manager who did not want to have to log into our system even to view a dashboard. I used the 'Crazy-8s' technique to generate ideas this time in an attempt to flush out more diverse and lateral ideas from the team. The results involved approaches which took advantage of our existing capabilities in new and interesting ways that I hadn't previously considered, such as adding reporting widgets to our own drag-and-drop email editor to support mailable performance reports.

On both occasions we generated a wealth of potential ideas, but the most important aspect was that the final solutions weren't based on just one person's idea but on the merging of different suggestions into a novel approach that could then be confidently taken into the mainstream development process.

Not all plain sailing

Setting a challenge for innovation is in some ways surprisingly easy - as long as you are able to sufficiently define a problem that is valuable to solve, then it's really a question of explaining the problem and presenting it to the people involved. There are some pitfalls to be aware of as you progress.

  • You need to be solving a real problem. Manufacturing a persona or inventing a problem statement may generate ideas but not useful outcomes. A few years ago I ran an innovation workshop against a challenge that we'd hypothesised existed for a user group, but we lacked evidence that it was a real concern. The result was a confusing workflow - the users didn't have the problem so didn't understand when we presented them with extra steps in the workflow to solve it.
  • Don't expect a perfect outcome 100% of the time. If a challenge is trivial it is not really a challenge, but a non-trivial one may not have the simple solution you were hoping for. All innovation is based on the assumption of having to learn from failure, and I advise having a back up plan to re-frame the challenge or consider a sub-optimal approach if an elegant solution isn't forthcoming.
  • Don't expect the solution you'd already thought of - when preparing challenges it's incredibly hard not to pre-design your own solutions. While you may have some ideas the whole point is to get wider input and let ideas bounce off each other. If your innovation processes are a thinly veiled attempt to push through your own solution ideas more quickly then don't expect to engage your team. In my most recent innovation exercise one potential approach that I'd thought may come up was rejected by the team as 'it wasn't really innovation', it was a straightforward development that just needed prioritising.
  • Be prepared to throw away the output - the result of setting innovation challenges is learning, not code. Even if a solution is prototyped please ensure to thrown this away, treating prototypes as early product never ends well and again the team are less likely to be engaged if your innovation efforts are an attempt to short-cut production developments without the rigour of proper process.
  • Be prepared to go through the 'groan zone'. If you are encouraging truly divergent thinking around a problem then you risk hitting the point where everyone gets slightly frustrated and feels like you're not narrowing on a solution. This is a natural consequence of really opening up your scope, if the challenge has been defined well then the process will naturally converge on a subset of ideas to progress.

An approach that grows as you do

If encouraging innovation in your organisation is on your radar but you're not sure where to start, I highly recommend establishing some innovation efforts based around setting challenges. While more open innovation may be the long term goal, working within the constraints of a known challenge addresses a lot of potential early concerns

  • Senior managers are more confident that innovation is driving towards a genuinely valuable goal.
  • The teams involved have the confidence to do some truly divergent thinking around the problem based on the outcome being a valuable one

As confidence from all parties grows in the value of such activities, you can make the challenges wider in scope and more ambitious. Keep progressing in this way and at some point the scope of the challenge set, and the aim to simply deliver value for your users, become one and the same.


  • My introduction to leading by setting challenges was through Tim Brown of IDEO's teaching such as this course and I highly recommend it for anyone stepping into an innovative leadership role - "Leading for Creativity"

Image - probably the ultimage innovation challenge, the Enigma machine. Wikimedia -

Whatsapp Button works on Mobile Device only

Start typing and press Enter to search