Data Impact

Since SEMA started, 69% of the offices where we deploy our tools have improved their service delivery and 46% have reduced their waiting times.

SEMA not only builds a bridge between the public officers and citizens providing feedback on the services that they have received. We do this to ensure that there is a positive change in the institutions where we set up our systems. And so we measure the impact of our work rigorously.

What does impact look like to us?​

SEMA’s Theory of Change holds that by presenting enough citizen feedback data regularly in customized and understandable formats to citizen-facing public servants, the highest ranked policy makers, and civil society partners, we can incentivize and advocate for concrete public service improvements and monitor the quality of public services over time.



Impact measurement consists of regular follow-ups to ensure that public officers:

    • Have used the SEMA citizen feedback reports in their team meetings
    • Have shared the SEMA citizen feedback reports with colleagues
    • Have hung SEMA citizen feedback reports on public notice boards for citizens to see
    • Have made concrete improvements to their office or services based on the SEMA citizen feedback reports (i.e. improved the waiting area, changed staff, provided a training)
    • Have changed their attitude or behavior towards citizens (i.e. come more often on time, smile more often, etc.)
 

These factors are recorded in the SEMA Impact Dashboard. This dashboard also tracks automatically whether citizen satisfaction and waiting times improved at an office over time, which we measure through the continuous citizen feedback systems (devices, interviews, mobile platforms). Together, these factors help us to determine whether services are improving and offices are receptive to the SEMA system. Our goal is to make and track lasting positive change at all public offices where SEMA works.

Through our time of collecting citizen feedback at the different institutions, we have received numerous  comments from citizens and the public service providers.

What citizens say.

“In the long run I think what you are doing will change something positively.”

Citizens who have visited public offices more than once have said that they have noticed a change in services since SEMA began its deployment, noting that service providers are more friendly and welcoming. As a result, citizens feel more comfortable when visiting public service providers and also like that their voices are heard, because they can rate the services they have just received at the institution rather than taking the time to talk to a public officer who may be involved with a case or not have influence over the institution: “I think this service will help people to get what they want from the offices they visit.”

What public service providers say.

IMG20210127104633
"For the first time ever, my personnel knew there was somebody watching, there was somebody to compliment what they were doing or report if something is not right. Since the feedback is not only for our police station, we developed that competitiveness [between stations]. Every morning when I am giving personnel tasks, I tell them that we are waiting for that scorecard [monthly feedback report] and that we want our marks to go higher.”
Officer in Charge

Many public service providers have received SEMA with a lot of enthusiasm. We have noticed that, in particular managers of public offices, see the SEMA reports as a tool to track their staff’s performance and improve the image of their station or office. Many reported that they did not have a (systematic) way of evaluating themselves, and that this new system allows them to learn about areas where they need to track and improve their individual as well as department performance: “It’s a really good idea. I wish we could have it everywhere.

Common improvements in service delivery through SEMA.

Almost all the offices where we have collected citizen feedback have used the SEMA reports in their meetings, and as a result some of have made specific changes:

  • Shuffling reception staff that was taking bribes according to citizens, lead to a steep decrease in reported corruption rates at one particular police station.
  • Adding navigation signs at a big municipality office led to increased satisfaction by citizens instantly, because they were able to find the department they were looking for easily.
  • One police station used SEMA’s reports to investigate recurring problems within the Child and Family Protection Unit. As a result, the station committed to start deploying a reception for the Criminal Investigation Department to reduce backlog in cases in the CFPU.
  • A health centre has greatly improved its laboratory services by changing its staff – a department which was reported by citizens to have been ‘without staff’ to help them. In fact, we have by now dozens of examples of public offices who have successfully reduced waiting times by addressing absenteeism with their staff.

In addition to this, almost all of the institutions have placed the reports on notice boards around their institutions so that citizens who visit the institution can see how other citizens have rated the services that they have received from the given public institution. This allows for public institutions to monitor the quality of services that they provide and to be more accountable to the citizens they serve.

What about the trainees who collect all the citizen feedback?

SEMA-trainee-ceremony
“Before SEMA I couldn't even go to a police station. Now I just walk right in and go straight to the Officer in Charge. When something's not right at the hospital, I say, this isn't right. I learned to give feedback, now I give feedback all the time. Feedback doesn't have to be through SEMA, it can be given everywhere."
Cephas Maili
SEMA Trainee

SEMA has so far trained 46 university students to talk to citizens about their experiences at public offices. Through the traineeship program, we have helped prepare these young professionals to learn about the issues at public institutions, become experienced researchers and step into the workforce.

The initial traineeship impact survey identified that trainees(formerly known as volunteers) generally improved their soft skills (communication, leadership, critical thinking, teamwork and work ethic) in the course of SEMA’s operations. This in turn improved their chance of employment. 70% of the trainees believed that their chances of getting employed increased a lot after volunteering with SEMA while 30% indicated that their chances increased a little.

100% of the trainees are still willing to work with SEMA if called upon and 80% of them have recommended university students to apply for the traineeship program.