“Is this the right software for the expected users?”

This question, the so-called “business validation”, needs to be addressed right from the early stages of software development. Developing a great platform full of imaginative functionalities could become a pointless exercise, unless it offers real value to the users. Many methodologies can be used to systematise business validation, but before proceeding with that, the priority should be to get the really understand users’ expectations.

User Expectations

When organisations decide to take up new software, they anticipate that it will help them operate more efficiently, enable them do something they couldn’t do before, or even both; which is exactly the case for ChildRescue.

What do ChildRescue end-user organisations want to achieve by using the platform? Greater awareness and involvement of citizens in the welfare of children, better collaboration inside and across the organisations, improved efficiency in terms of time and cost and the most important: more missing children found and returned safely to their homes.

Such expectations will play an important role in the validation of the ChildRescue platform and the best way to ensure that the developed solution will satisfy them is incorporating them from the stage of design.

User Requirements

In terms of requirements: brainstorming, questionnaires, interviews, workshops, user stories, system requirements, prioritisation and many other methods are used in the process of understanding user’s needs and turning them into a developer-friendly form.

Is this a once off or a continuous process? That depends on the development methodology. In the more traditional Waterfall* approach, everything happens linearly. Requirements are gathered and functionalities are decided right from the beginning, everyone agrees on them and that’s it. The team proceeds with the design and development and the initial requirements remain untouched until the delivery of the software. The Agile* method on the other hand adopts an iterative approach. The user interacts with the team and software throughout its development, and the initial requirements are dynamically adapted, as the users get a better understanding of what is created. Both methods have things to offer, but they also have disadvantages, as shown in figure 2.

ChildRescue adopts a hybrid methodology. Following the Waterfall model, a group of user requirements covering all functionalities of the platform, was defined right from the start of the project. These requirements have been dynamic rather than static, they are constantly revisited, adapted and finally become clearer, as in Agile projects.

Validation Framework

Back to the main subject of business validation. Metrics must be used in order to measure if the organisations are able to get closer to the expected results by using the software. These metrics, i.e. performance indicators, quantify things, and numbers are always better for comparisons and to be able to check progress. There are no one-size-fits-all performance indicators. Teams must define them based on the context of the application, and the validation frameworks that can guide them.

Performance indicators measure things, but what exactly do we need to measure?

Outputs vs outcomes vs impacts

An output is defined as the result of a process. It can be a service, functionality or a product such as the number of served customers, completed phone calls or feedback received. Outputs alone can not discern the satisfaction of the users or the value added to the organisations, but can show what was produced through the software, or in other words, they demonstrate efficiency.

On the other hand, an outcome could be the level of achievement and benefit for the users because they used the software. It indicates effectiveness and answers questions like: “Could the operators perform an activity faster than before, thanks to the platform?” or “Have the users learned something new about our cause?”.

Lastly, impact is the long-term change that happen when the users, organisations and community have adopted the software. They are the most important and at the same time the most difficult to measure. Anticipated impact is the reason software is created, and teams should really try to grasp them and even rethink their solutions to achieve them.

The ChildRescue validation model, figure 3, is based on Performance Based Management (PBM), one of the most popular validation frameworks.

It illustrates the relationships among requirements, functionalities, results and performance indicators. Performance is measured in two levels: efficiency and effectiveness.

Efficiency is linked to the outputs of ChildRescue usage and can be quantified by collecting data regarding things like “number of notifications sent for a case through ChildRescue and opened by the user” or “number of monthly ChildRescue app downloads”.

Effectiveness has more to do with outcomes and impact and in the ChildRescue case, is measured by questions like “what is the average time needed between reporting and finding a missing child” or “how many organisations have had to collaborate through ChildRescue for the success of a case”.

To bring all these indicators into existence, data is needed. The way to collect data is also part of the validation process design. Many methods could be suitable for this; however, the applicable method should be selected separately, depending on the nature of each indicator: reports, surveys, questionnaires, or even automatic methods directly from the ChildRescue system. In general, each process should be designed to be as objective as possible, free from personal bias. This is not always possible due to the need of qualitative indicators.

Conclusion

The ChildRescue business validation is designed as a continuous and iterative process, which involves both development teams and users. It covers all the necessary aspects of validation, from baseline value calculation to data collection and interpretation of results. The business validation activities will be launched together with the pilot operation of ChildRescue. It will continue even after the project’s official conclusion. This aims to facilitate the assessment of the ChildRescue Platform’s long-term impact and prove the project’s contribution towards the welfare of children.