From idea to system transformation: Enhancing project maturity one step at a time

Source: MLSDev

While it is common practice in the private sector to have a structured process for product testing and development, such an approach is still rare in public policy and international development. By organizing their portfolio of projects or intervention models across different stages of maturity, public and social sector organizations could enhance their strategic planning and operational workflows (including for Monitoring & Evaluation), ultimately achieving higher impacts.

By Kevin Hempel | June 2023

Most entrepreneurs and leaders in public policy would agree that robust solutions to complex problems need massive amounts of testing and iterative experimentation. Personally, I have always liked that idea of structured learning to solve problems one step at a time. It makes intuitive sense. It’s like in sports: Nobody is born as a champion. People become champions through lots (!) of training and iterative improvement.

Yet, while a structured process to product and service development is a no-brainer in the business world, reflected in terminology such as “prototypes”, “proof of concept”, “minimum viable product” “minimum marketable product” or “alpha and beta testing”, a structured approach to product testing still hasn’t made big inroads in how we think about public policy or international development programs. 

Fortunately, there are some outliers. For instance, USAID’s Development Innovation Ventures (DIV) uses a tiered-funding model that distinguishes prospective grantees in terms of scale and evidence base. Similarly, the International Rescue Committee’s Airbel Impact Lab organizes its portfolio into three broad and six sub-stages, from “design research” to “scale”. I was also recently talking to a colleague in a large foundation who told me that their approach to strategic philanthropy involves grantmaking around four broad stages, from “scoping” to “transformation”, where each stage is characterized by less uncertainty regarding the benefits and risks involved.

There are more examples, but as indicated above, overall, this way of thinking and working seems to be the exception rather than the norm. Where it exists, it is often limited to an innovation unit, rather than being mainstreamed across the organization. I believe that should change…

Different stages of program maturity

When I think about stages of maturity in the context of international development programs, I tend to think about six main stages (see table below). Higher stages reflect more established programming with higher quality evidence available about the respective project or intervention model. That said, I don’t really think the number of stages matters, as each organization can find its own adequate level of granularity/aggregation.

What is a bit different in my framework compared to what I have seen from others is that I like to split the piloting phase into three distinct stages. Why? Because I believe that just having one stage for “piloting” really doesn’t give justice to the many layers of maturity and learning that need to happen during this phase, which is why I think it can help subdividing the piloting into different parts.

Regardless of the number of stages one chooses, it is essential to adopt a clear understanding of the level of programmatic maturity one expects to see at each stage and which Monitoring, Evaluation, Accountability and Learning (MEAL) tools one needs at every stage to build evidence over time.

There is no linear or automatic process in moving from one stage to the next. Sometimes, programs/approaches may have to go through the same stage several times until they are ready to “graduate” to the next level. For other programs, one may realize that the results are simply not strong enough to justify the investment of more resources and the program/approach would be dropped.

What could thinking in terms of stages of maturity look like in practice?

For example, in the area of labor market integration:

• Imagine an international NGO having a long-standing project that uses the “Graduation approach”. Based on the extensive evidence and international adoption, this project could be considered in the scaling or transformation stage (stage 5 or 6).
• At the same time, the NGO may be working on a demand-driven training program for youth that it has refined over the years and that is now being tested in several governorates across the country (stage 4).
• Finally, it may also be starting up some new initiatives to help connect women to STEM jobs and to upskill mid-career workers, but these are still new with limited evidence so far about beneficiary uptake and outcomes (stage 2).

What are the main benefits of thinking in stages of maturity

1. Clear overview: The first advantage is clarity. Rather than just having a laundry list of projects or intervention models that one supports, mapping one’s portfolio along these different stages provides more clarity about their respective maturity (see next figure below). This can be very useful for internal and external communication purposes.

2. Strategic planning: Having a good overview of the portfolio by stage of maturity and evidence is also an important input for strategy development and funding allocations. Indeed, it shows whether there is a healthy pipeline of projects/models. For instance, one may find that the portfolio is concentrated in stage 2 and 3 piloting, suggesting that more efforts are needed to move towards scaling-up. Conversely, a lack of stage 1 and 2 projects may be problematic as the pipeline of new ideas is “drying out”. 

3. Structured workflow, incl. for M&E: The third key advantage of such an approach is that it becomes very clear what can be expected and what is needed at each stage, both programmatically and in terms of learning. No need to reinvent the wheel every time. When a project or model advances to the next stage, we already have a general understanding as to the types of M&E tools we want to deploy, and we can plan accordingly (e.g., integrating certain M&E budgets, identifying researchers for rigorous evaluation, etc.).

Common challenges in the absence of a structured portfolio

When organizations don’t have a structured way to think about their portfolio, several challenges can arise, for example:

• Unrealistic expectations: When there is no structured process that projects go through to build up maturity, it is easy to forget that it takes time to get from idea to high-performing model. Donors and/or project teams may be overly ambitious and expect that money can buy results, setting high targets from the start. Instead, when there are clear stages to go through to demonstrate stage-appropriate results, expectations can be more easily managed.
• Grow too quickly: My impression is that, too often, we skip the full piloting phase (Stage 3) which seeks to solidify the project or intervention model (“make it good”). Instead, we take half-baked approaches from very basic piloting (if any) to extensive piloting across multiple sites and contexts without ever really establishing a strong delivery model. As a result, trying to set up a relatively complex operation from scratch and building knowledge about what works at the same time often becomes too overwhelming for program teams. These interventions then struggle to find enough participants, are not able to deliver high quality services, and may be unable to realize what constitutes the core of the model and what is a context-specific adaptation. As a result, they are unable to achieve the impacts otherwise possible.
• Misaligned M&E: To maximize learning and knowledge generation, various M&E tools and methodologies need to be properly sequenced. Without a clear framework as to which M&E tools to prioritize at what time, there can be over- or underinvestment in M&E. Let’s take the example of rigorous impact evaluation. There are many cases where significant resources have been spent to conduct a counterfactual impact evaluation when the project/model was simply not mature enough yet. In this case, the lack of results found by the evaluation cannot be properly interpreted, as we don’t know whether the model didn’t work or whether the results would have been better if implementation quality had been higher. Likewise, there are many interventions that are already operating at larger scale that have never been rigorously tested, thus potentially investing significant resources into “unproven” models.

Final thoughts

Structured research and development (R&D) is not only critical in the private sector, it is also paramount to solve social challenges. Organizing a project portfolio along clear stages of maturity can therefore benefit a wide range of organizations, including public sector agencies, international development institutions, foundations, and NGOs. The larger the organization and the more projects are being funded or under operation, the more such an approach is a necessity.

It does not take much to move in this direction. Organizations can refine the proposed stages of maturity illustrated in this article based on their own circumstances and preferences and start mapping their projects accordingly. This first step will already jumpstart several reflections that will shape further conversations about strategy development and learning priorities.

What are your thoughts on the topic? Have you come across good examples of a structured portfolio approach in your work? Send us an email, we look forward to hearing from you…

Kevin Hempel is the Founder and Managing Director of Prospera Consulting, a boutique consulting firm working towards stronger policies and programs to facilitate the labor market integration of disadvantaged groups. You can follow him on LinkedIn and Twitter.