Evaluation of a Pilot project on Enhancing Access to Reproductive Health via Telemedicine

Background

The United Nations Population Fund mission revolves mainly around delivering a world where every pregnancy is wanted, every childbirth is safe and every young persons’ potential is fulfilled. UNFPA believes that every individual has the right to make his/her own choices about sexual and reproductive health (SRH) including informed decisions about family planning choices. Alongside other United Nations organizations, UNFPA operates in humanitarian and emergency situations to enhance access to SRH and combat gender-based violence (GBV) and mitigate its consequences.

While reaching out to the population at large is at the core of its work, UNFPA strives to reach out to most vulnerable population including adolescents, refugees, poor, persons with disabilities, migrant women,  etc in both crisis and normal situations and ensuring leaving no one behind. Since 2011, UNFPA Lebanon CO has been considerably supporting nationwide key programmatic interventions particularly around enhancing access to SRH and preventing/responding to GBV with due attention to the Syrian refugees and Lebanese in hosting communities. Those interventions translated mainly into reinforcing national systems and structures, enhancing quality of care, improving providers and institutional capacities, coordinating joint responses, raising community awareness as well as developing/supporting advocacy campaigns to promote women empowerment and combat GBV.

In 2020, the COVID-19 pandemic has disrupted access to health care services globally including for SRH based on the available literature. It is also noticeable that the pandemic has created as well as revealed disproportional consequences in fragile settings - such as in Lebanon, where relatively weak public health infrastructure exists. In Lebanon, the combination of economic and social stresses brought on by the pandemic as well as the deteriorating economic situation, restrictions on movement, fear of contracting the virus and civil unrest have made accessing health services more difficult for the population at large including women, pregnant women, adolescent girls, as reported by partners as well as beneficiaries. The Beirut port explosion weakened further the health care and social services systems with an increased number of vulnerable men and women (both Lebanese and Syrian refugees) affected by the port explosions and in need to long term care including mental health. 

Accordingly, and in response to the multiple crisis, UNFPA prioritized its interventions in 2020 targeting mainly vulnerable and marginalized groups through collaboration with governmental entities, UN sister agencies, local and international NGOs, civil society organizations and professional entities namely  Lebanese Order of Midwives (LOM) and Lebanese society of Obstetrics and Gynecology (LSOG) to ensure that the SRH/GBV needs are met and systems and capacities are in place to prevent and respond to SRH and GBV and facilitate access to related services. In order to adapt to the changing context characterized by mobility restriction, fear of contracting the virus, unaffordability to seek health care services, prioritizing food over health care, etc, UNFPA had to consider innovative approaches to meet the urgent SRH needs of those most at risks.  Therefore and in addition to the typical ongoing partnerships for promoting access to SRH/GBV services, and in 2020 UNFPA entered into a new innovative partnership with the Social Workers Syndicate (SWS) and the Lebanese Society of Family Medicine (LSFM) for piloting a telemedicine approach  for scaling up RH services whereby SWS connect with local communities to identify vulnerable women and adolescent girls in need for RH services who are referred to LSFM network to provide them with online medical/clinical consultations and referral to care as needed as well as patient education and awareness raising. In addition, this pilot initiative consists of promoting community awareness on RH through engagement with municipalities in a series of RH topics including for COVID19 and delivered through social media platforms (namely municipalities live streamed Facebook pages).

The project was initiated in October 2020 and still ongoing, involved so far 10 social workers, 9 family medical doctors,720 targeted women beneficiaries, and more than 60 targeted communities. The evaluation will cover the period October 2020 to March 2021.

Based on the above and in order to assess mainly the effectiveness, relevance and impact of the innovative pilot initiative, UNFPA is seeking to undertake an evaluation to inform expanded, adjusted and scalable programmatic interventions.

Purpose

The main purpose of the evaluation revolves around three main axes as follows:  

1. Assess satisfaction of targeted women and girls vis-a-vis service provision through telemedicine
2. Assess effectiveness, relevance and impact of the interventions
3. Support evidence-based decision making for tailoring/scaling up/expanding interventions

Objectives

While the overall goal of the evaluation is to produce an independent and useful report, with a view to informing similar projects, the evaluation will address selected aspects of the effectiveness, efficiency, relevance, coverage, coherence and connectedness criteria.

 The specific objectives of the evaluation are detailed below:

• Assess the efficiency and effectiveness of the approaches used to achieve the project outcome.
• Assess the coherence and connectedness among all key actors namely SWS, LSFM, and the community stakeholders
• Assess perception and satisfaction of women and girls who received online consultation through telemedicine.
• Assess follow up and referral of cases  
• Identify challenges, constraints and changes encountered during the project’s implementation and assess variances/mitigation measures adopted
• Retrieve lessons learned and good practices.
• Determine elements for future possible sustainability of interventions beyond the lifespan of the project
• Propose recommendations and concrete actions for scaling up/expanding the interventions

Scope of the Evaluation

It is anticipated that the evaluation will look into the outputs, results and deliverables within various categories i.e. beneficiaries, municipalities, social workers and Family Medicine doctors involved in the project. As such, the evaluation will adopt a three staged process namely 1) Key informants interviews with representatives from targeted municipalities, LSFM, and SWS; 2) Perception/Satisfaction survey with women and girls who were part of this initiative; and 3) Consultation with Key actors about the needs/concerns/potential for expanding telemedicine among other actors and mainstreaming telemedicine into the upgraded health system

The evaluation will require 20 working days and to take place over a period of 6 weeks.

The final evaluation report will entail i) developing a matrix summarizing 1-2 findings for each of the following 6 criteria: effectiveness, efficiency, relevance, coverage, coherence and connectedness, ii) analyze variances, strengths, gaps and challenges throughout implementation, iii) assessing potential for mainstreaming telemedicine under a comprehensive telehealth  model as a hybrid approach within the health system and iv) provide a concise set of practical and feasible recommendations for scaling up and or implementing   future similar projects using Telemedicine.

Process and Methodology

Preparation and Inception phase

• Inception meeting to clarify terms of evaluation between consultant and UNFPA.
• Delivery of brief 3 – 5 pages inception report detailing the consultant’s understanding of the required tasks and the proposed approach to work, analysis approaches, and detailed timeline for product delivery.

Desk Review phase

• Desk and literature review including similar global/regional initiatives as well as documents, reports, annual review meetings reports, resource material produced in the frame of the project.

Consultations and data collection phase  (with Key informants / Satisfaction survey with beneficiaries)

• The consultant will design an evaluation methodology (including data collection tools: survey, interview guide; evaluation questions, and evaluation matrix with associated assumptions to be assessed) using quantitative /qualitative data
• The consultant will hold meetings with UNFPA, SWS and LSFM representatives, social workers and family medicine doctors engaged in the project as well as a sample of the targeted municipalities’ representatives. Meetings could be conducted in person or remotely.
• Conduct consultations with external stakeholders namely representatives from the Ministry of Public Health, WHO, UNICEF, UNHCR, the EU and the World Bank
• The consultant will assess perception of women and girls about the online medical services through this pilot
• It is expected that the consultation process will consist of 8-10 interviews/meetings, 5-6 FGDs and an satisfaction assessment among beneficiaries.

Synthesis phase

• During this phase, the consultant will proceed with the analytical work based on the desk review and consultation process
• All findings and conclusions must be backed by reference to evidence (source)

Final outcome

• The consultant will incorporate comments received from UNFPA into final report
• The consultant will highlight methodological challenges and limitations in findings and conclusions.
• The consultant will submit (a) a descriptive and analytical copy-edited and cleanly formatted English report (20-25 pages excluding annexes) with references as a Word document (template to be discussed and agreed upon), (b) an executive English and Arabic summary (2-3 pages each language) with the key findings and recommendations, and (c) a power point presentation in English/Arabic language (with 15-20 slides each language) summarizing key outcomes and findings of the evaluation exercise.
• The consultant will be expected to take part in 1 technical meeting (person or remote) depending on the situation to present the findings

Support to be provided by UNFPA

• Clarify consultancy TORs and expectations
• Avail all documentations
• Be available for meetings with the consultant when need be
• Facilitate organization of meetings and interviews with stakeholders
• Provide additional and immediate advice and information when need be
• Review draft report and provide input

Indicative time frame for the consultancy

Skills and experience required

As mentioned earlier, this consultancy will require 20 working days over 6 weeks and therefore the consultant requested to fulfill this task need to be fully dedicated to this exercise and equipped to undertake requested multiple tasks. He/she should be available to start as of April 12th, 2021 

The consultant must have relevant experience in research, evaluation, experience in development and/or humanitarian settings, knowledge of evaluation methodology, and experience and in-depth knowledge of the Lebanese context.

Qualifications

The consultant must offer the following demonstrated expertise, and qualifications:

• A university degree in public health, gender studies, sociology/anthropology, social sciences or other relevant specialty
• Excellent research and analytical skills
• At least 5 years of extensive experiences in carrying out evaluations/assessments
• Experience and knowledge in the field of SRH and/or GBV is essential
• Knowledge of the Lebanese health system is an asset
• Working with international organizations/ UN agencies/ donors is an asset.
• Experience in humanitarian /emergency contexts including familiarity with humanitarian related resources and material is an asset
• Excellent Arabic and English spoken and written skills

Individual consultant that meet the required qualifications set forth in the TORs may apply on line by sending:

1- A cover letter expressing interest

2- A curriculum vitae

3- A price schedule form (Annex VII attached).

All documents must be sent to the following email: bids_lbn@unfpa.org    by no later than April 2^nd 2021.