Intro
Imagine you are a member of a theatrical team and have been preparing for months, for a new play. Hours of individual and teamwork, scene staging, preparation of costumes and numerous rehearsals. But before you perform in front of the audience, you need a last, but not least, step: the dress rehearsal*.
Rehearsing before the Public Release
We could look at software as a theatrical play. Before everything is ready, the developers must work individually but also as part of a team, and gradually develop the features of the software. Even from this stage, they must start ‘rehearsing’, i.e. testing that everything works as it should. Initially, the developers internally test the features they have developed, for serious bugs and see if all the different components fit smoothly together. Afterwards, this version of the software – even if not all foreseen features are there – is tested by other developer teams that bring a fresh perspective and have the technical background to pinpoint errors. This was the alpha testing phase*.
With all this feedback, the team continues working: Bugs are getting fixed, some of the development plans change, more functionalities are added and the software is getting closer to being feature-complete. It is a small-scale version of the final software and at this point, the development team can locate any potential problems and make changes to fit the needs of the users before everything escalates. This is the right time for a mini, unofficial ‘open dress’: invite some of the end users to play with the software and give their comments; usability, performance, wording, functionality, colours and more. This is the feedback the team behind the scenes needs before they go for the full implementation, i.e. pilot testing*.
Bearing the users’ comments in mind, the development team goes for the full implementation and the public release. However, there is one final step before the premier of the software: ’the dress rehearsal’. The system is used in real cases by the end users, who ensure that the provided functionalities operate as expected and that the system offers a reliable and bug free solution.
ChildRescue Pilot Planning
In ChildRescue we follow a similar approach, in order to ensure that everything is ready before the ChildRescue integrated platform is publicly available for the citizens and the organisations. Except for verifying (“Is our solution bug free?”) and validating (“Is our solution the right one for the users’ needs?”) the platform, the pilot operation is also a great opportunity to engage the end users with the platform and make them more familiar with the functionalities it offers. It is also a means to extract metrics and measure the success of our solution, even from the initial phases of the implementation.
Pilot testing doesn’t start just by giving access to some end users and letting them wander around the mobile and web application. On the contrary, pilot testing requires its own preparation:
1) Pilot scenarios to guide the users through the platform. These scenarios are designed to showcase how ChildRescue is integrated in the usual procedures, followed by the related departments of the corresponding organisations. The two use cases supported by ChildRescue are illustrated through three scenarios, which are based on some of the most typical cases.
ChildRescue Scenarios:
|
2) A pilot plan to define “when”, “who”, “where” and “how” the operation will take place. The pilot planning in ChildRescue follows the platform development: each pilot operation takes place in a short period after the two development milestones – Release 1.0 and Release 2.0 -, and tests the developed solution thus far. The scenarios are adapted to fit with the development process and to involve only platform features that are available in the release they test. That is why the first pilot is designed as a small-scale tabletop exercise, while the second one is a field exercise that makes use of the full feature ChildRescue web and mobile app. The pilot plan also includes the design of questionnaires for the extraction of feedback in a structured manner. Free flow of speech is good, but open and close-ended questions are even better to ensure that we get all the information we need from the users. The questions revolve around usability, workflow, operation, user interface, but they also leave space for concerns or suggestions the users may have.
3) The platform must be prepared. Web user accounts must be created and cases must be registered. The mobile application must be available for end users to install. For the first pilot operation, only demo users and demo cases were created. However, in the second pilot phase, real members of the organisations will be registered as users of the web platform. Additionally, real cases will, also be registered, so that the machine learning algorithms can start training and perform the Childrescue analytics.
4) The users must be prepared too. A training session is needed for each team of end-users, where the technical partners guide them through the features of the web and mobile app. The end-user handbook also provides detailed guidance to the users.
ChildRescue 1st Pilot Execution
The first phase of the ChildRescue pilot operation took place in Fall 2019, at the facilities of the three pilot organisations. It tested the functionalities of ChildRescue Integrated Platform Release 1.0. A small group of experienced social workers, took part in the exercise and was guided, by the technical partners, through the tabletop exercise. Afterwards, the participants, starting from the questionnaires, discussed about changes or additional functionalities they would like to see in the platform. Their valuable feedback was collected, translated in performance indicators (see the blogpost for the ChildRescue validation) and will be used to refine the platform and bring it closer to the users’ needs. Furthermore, useful insights were extracted for the pilot operation as a procedure and adjustments will be made for the second phase.
The overall perception of ChildRescue by the end users was very positive. They expect the platform to help them in their everyday work and intent to use it as a primary tool. Faster location of missing children will be enabled, as ChildRescue will facilitate the communication with even more citizens and provide insights using analytics. For the unaccompanied migrant minors use case, ChildRescue is seen as a great opportunity to promote the use of a unified system and speed up the exchange of documents for minors moving from one hosting facility to another.
What comes next?
We can say that the first open rehearsal for Childrescue was a success and created the anticipation for the public release! As we are currently speaking, we are heading towards ChildRescue Platform Release 2.0. The second pilot exercise plan is under construction and will be ready just in time, to test the second and final release. Stay tuned for the second part of Childrescue – Pilot Operation, featuring the field pilot exercise!
| *dress rehearsal: a full-scale rehearsal where the actors and/or musicians perform every detail of the performance
*open dress: an “open dress” is a dress rehearsal to which specific individuals have been invited to attend as audience members *alpha testing: testing performed by testers, at the developer’s site, to find any major bugs. The testers, usually, come from the development team *pilot testing: a selected group of end users tries the system under test, at the developer’s site before deployment, to provide feedback * beta testing: is performed by end users at the client side (last stages of development cycle). They use the system and see whether it is working as per their requirements |
Featured Photo by Brian Kraus on Unsplash