MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE)
MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE)
BUILDING
BACK
REQUEST FOR PROPOSALS: Impact Evaluation of Social and Economic Programs in Bosnia and Herzegovina
|
USAID Monitoring and Evaluation Support Activity (MEASURE-BiH) Contracting Entity: IMPAQ International, LLC INTRODUCTION
As part of its efforts aimed at developing a common vision of BiH’s need for more rigorous policy analysis, USAID MEASURE-BiH invites research organizations in BiH to submit proposals for rigorous impact evaluations of social and economic programs in BiH in any area of social and economic policy. MEASURE-BiH, implemented by IMPAQ International, LLC, is a USAID-funded Activity focused on technical, analytic, advisory, training, monitoring, evaluation, and related program evaluation and social science research capacity building for USAID/BiH. MEASURE-BiH, will fund the design and implementation of impact evaluation of a program funded and delivered by a government or public institution in BiH. The evaluation meets the additional selection criteria set out below.
Our goal in procuring such evaluation is to build local organizational capacity in rigorous impact evaluation and build a body of evidence from rigorously evaluated social and economic programs that can inform decision-making and increase effectiveness of development programs in BiH. This Request for Proposals (RFP) also seeks to demonstrate to a broad government, international organization, and research community audience that low-cost impact evaluations are a powerful new tool for building scientific evidence about "what works” in social and economic spending. Well-conducted impact evaluations are widely regarded as the most credible method of assessing whether a social and economic program is effective, but are often assumed to be inherently too expensive and burdensome for practical use in most areas. Recently, however, researchers have shown that high-quality impact evaluations can be conducted at low cost and minimal burden, addressing a key obstacle to their widespread use. This is achieved by measuring key outcomes with administrative data that are already collected for other purposes (e.g., financial records, student test scores, criminal arrest records, and health care expenditures), rather than engaging in original and often expensive data collection through interviews or testing. Such studies make it possible now, as never before, for policy makers to use scientific evidence about what works to increase government effectiveness.
This RFP invites BiH local research organization or consortium of organizations to conduct impact evaluations in any area of social and economic policy. The targeted maximum award amount expected to be awarded to the proposal that receives the strongest ratings from USAID MEASURE-BiH’s expert review panel, based on the criteria in this RFP, is $70,000 but we will potentially award up to $ 75,000 if the additional cost can be justified, as described below.
The following table shows the requested application materials and timeline:
Letters of interest and invited full proposals should address each of the selection criteria below, within three pages (for the letter) and eight pages (for the invited full proposal). Letters of interest should include tentative evaluation questions. Applicants may use their own format, with single or double spacing, and a font of 11 or larger. The page limit does not include attached letters or other documents specifically requested in this RFP. Please submit all items via email to contacts@measurebih.com.
SELECTION CRITERIA
For the letter of interest: While we ask applicants to address all four selection criteria below, we do not expect applicants to have finalized all aspects of the evaluation design and partnership agreements; therefore, reviewers will focus primarily on two criteria – "importance” and "experienced researcher” – in determining which applicants to invite to submit a full proposal.
For the invited full proposal: Reviewers will consider whether all five criteria listed below are satisfied.
IMPORTANCE: Is the applicant proposing to evaluate an intervention –
that is backed by highly-promising prior evidence, suggesting it could produce sizable impacts on outcomes of recognized policy importance – such as educational achievement, workforce earnings, criminal arrests, and government spending (for example, we specifically encourage proposals seeking to replicate findings from prior rigorous evaluations that are especially promising but not yet conclusive, e.g., due to only short-term follow-up, a single-site study design, and/or lack of necessary data).
- or –
for which there are other compelling reasons to evaluate its effectiveness – e.g., it is, or soon will be, widely implemented with significant taxpayer investment, and its impact on its targeted outcomes is currently unknown.
EXPERIENCED RESEARCHER: Does the applicant’s team include at least one researcher in a key substantive role who has knowledge in research methods needed for the proposed impact evaluation, research methods include impact evaluation design (both experimental and quasi-experimental design), survey methodology, quantitative data analysis, econometric and statistical methods such as summary statistics, multiple regression, matching methods, interrupted time series, or regression discontinuity design. USAID MEASURE-BiH recognizes the needs in BiH to develop local researchers’ capacities in impact evaluation and will provide technical assistance such as participating in evaluation design and econometric specification development and data analysis during the implementation of the impact evaluation.
STUDY DESIGN: Is the applicant’s proposed impact evaluation design –
feasible with the available data? Applicants, as part of their discussion of this criterion, should specify the study’s primary data sources and outcome(s) of interest, how the outcome(s) will be measured and over what length of time, how the other variables will be obtained and the feasibility of obtaining the data in time, and what analyses are planned (e.g., any subgroups to be examined, regression methods to be used).
- and –
valid? In other words, does it have a sufficiently large sample (as shown through a power analysis) and other elements needed to generate credible evidence about the intervention’s impact on one or more targeted outcomes of high policy importance? We strongly encourage designs that measure such outcomes in both the short and longer term, as appropriate for the type of intervention and study, to determine whether the effects endure long enough to constitute meaningful improvement in people’s lives.
PARTNERS: Does the applicant’s proposal have verifiable necessary support from the government/public institution that funds and delivered the program that is proposed to be evaluated. Examples of necessary parties include an agency delivering the intervention, and an agency housing the administrative data. To verify the existence of such partnership, we will look for attached letters or other communication showing, for example, that (a) a government/public institution that delivers the program that is being evaluated has agreed to provide necessary data for the impact evaluation; and (b) a data agency has agreed to provide the researcher(s) with access to the administrative data needed to measure study outcomes.
COST: Is the applicant’s proposed budget within the limits specified in this RFP, is it reasonable based on proposed scope and methodology, and is summary budget submitted?
For the letter of interest, the weighting of selection criteria is
For the invited full proposal, the weighting of selection criteria is
TASKS Task 1: Kick-off Meeting Within one week of the award, applicant’s proposed Project Director and staff as appropriate shall meet with MEASURE-BiH team and relevant USAID staff in Sarajevo to review the proposed Scope of Work for this task which will be established within the Contract between USAID MEASURE-BiH and the selected applicant (Contractor) and establish a common understanding about the evaluation objectives. Among the topics for this meeting will be: MEASURE-BiH’s goals and expectation for this impact evaluation; a review of each task encompassed under this evaluation and a discussion of any challenges that might be faced during the completion of each; review of schedule; and administrative details related to invoicing, bi-weekly progress reports, and ongoing communications. Due date after award - Week 1.
|
|
Task 2: Management and Work Plan Based on the discussions at the kick-off meetings and the tasks outlined in the Statement of Work, the Contractor shall submit a draft management and work plan for the project by the end of the third week of the contract. This work plan is expected to govern the performance of all work under this contract, and should include: a detailed outline of specific tasks, approaches, staffing and management responsibilities, and the allocation of contract resources. The Contractor shall provide a detailed allocation of contract resources and a schedule for accomplishing the substantive work of the contract. The work plan shall also include a timeline identifying start dates, completion dates, and other milestones for each task and subtask. Where there are interdependencies among tasks, the plan shall indicate the relationships between each of the tasks. When appropriate, the work plan shall also indicate any other decisions reached at the kick-off meeting. A final version of the work plan incorporating the comments from USAID MEASURE-BiH shall be due at the end of the fifth week of the contract. Due date after award:
2.1 Draft Management and Work Plan - Week 3;
2.2 Final Management and Work Plan - Week 5
Task 3: Evaluation Design/Data Collection and Analysis Plan
The Contractor shall develop a refined evaluation design and data collection and analysis plan (ED/DCAP) that shall serve as a technical blueprint for all research activities that will be undertaken to address the evaluation objectives outlined in the Statement of Work and expand upon the approaches proposed in the successful proposal. The evaluation design and data collection and analysis plan must include:
Draft evaluation design/data collection and analysis plan, incorporating all of the topics as described above, shall be submitted for review and approval by USAID MEASURE-BiH no later than ten weeks after the award date. Due date after award:
3.1 Draft Evaluation Design/Data Collection and Analysis Plan- Week 10;
3.2 Final Evaluation Design/Data Collection and Analysis Plan - Week 14.
Task 4: Data Collection & Analysis
The Contractor is responsible for developing all data collection instruments and materials. Where primary data gathering is proposed, the Contractor will be responsible for developing all data collection instruments and the Contractor is responsible for pre-testing the instruments and conducting any training necessary to carry out the data collection activities. Where administrative data gathering is proposed, the Contractor shall establish data use agreements (DUA) with each participating hosting agency. Once a DUA is in place, the Contractor shall work with the data hosting agency to access a data extract according to specifications described in the approved ED/DCAP. The Contractor shall develop a memo that summarizes the challenges and lessons learned in conducting primary data collection or accessing the administrative data sought under this evaluation. The Contractor shall make clear in their proposed methodology how the data will be used. Due date after award:
4.1 Develop data collection instruments or establish data use agreements. Week 16
4.2 Obtain Data.
4.3 Draft memo summarizing lessons learned.
Task 5: Final Report
The final report will be produced after implementation of all evaluation activities. This report will address the research questions posed for this study and include all outcomes and impact analysis done as part of this study. Due date after award:
5.1 Outline of Final Report: Week 24
5.2 Draft Final Report- Week: 30
5.3 Second Draft Final Report: Week 36
5.4 Final Report: Week 42
Task 6: Final Briefing
The Contractor shall prepare a final briefing that summarizes the overarching findings of this evaluation effort to understand more about the effects that the program had on participants. It is expected that the briefing audience will be comprised of USAID program and technical staff, MEASURE-BiH staff, program funders, and government institution and international organization staff, as well as any additional partners USAID/BiH chooses to invite. The details of the presentation will be determined in consultation with USAID MEASURE-BiH and the Contractor is responsible for the production of all briefing materials, as the Contractor and USAID MEASURE-BiH find appropriate. Drafts of materials must be submitted to USAID MEASURE-BiH for review/approval at least two weeks in advance of the final briefing. Due date after award:
6.1 Draft Presentation Materials: Week 82
6.2 Final Briefing Materials: 2 weeks after initial draft materials;
6.3 Final in person briefing to be completed by Week 92.
Task 7: Data Delivery
At the time of submission of the draft final reports the Contractor must provide a database with all information collected, having implemented quality control methods, including assurance of confidentiality, cleaning the data, clarifying errors in information collection and programming, appropriate analytic programs, and complete documentation. The Contractor must organize the data into database files, utilizing IBM-compatible microcomputer systems consistent with hardware/software adopted by MEASURE-BiH. Guidelines for the preparation of the database files will be provided by MEASURE-BiH at the kick-off meeting.
For the final deliverables for this task, the Contractor shall prepare and submit to MEASURE-BIH media that include the raw data files and data analytic files used to produce all results that appear in the final reports. These files shall be provided in micro-computer format for use with PC STATA or other suitable analytic software selected by USAID MEASURE-BiH. In addition, the Contractor shall provide full documentation for these data files including:
Due date after award: Delivery files below due no later than week 90 and include:
7.1 Raw data files; and
7.2: Analytic Data Files.
PROJECT MANAGEMENT & ADMINISTRATION
The Contractor''''s Project Director (PD) will communicate bi-weekly with MEASURE-BiH on all activities through a bi-weekly progress report. The PD will also report bi-weekly on any timeline issues and approaches to stay on schedule.
OTHER ITEMS TO INCLUDE IN THE LETTER OF INTEREST AND INVITED FULL PROPOSAL
Applicants should specify the amount of funding requested. Our targeted award amount is $70,000. Applicants requesting a larger award – up to the limit of $75,000 – should submit an attachment (no more than one page) to the letter of interest and full proposal that provides the reasons for the higher request. The summary budget must be submitted using the Attachment A.
Letter of Interest should be submitted in a sealed envelope by June 9, 5 pm local time to the address:
USAID Projekat podrske monitoringu i evaluaciji (MEASURE-BiH)
Fra. Anđela Zvizdovića 1
UNITIC Tower B, Floor 21
71000 Sarajevo
Bosnia and Herzegovina
Additionally, an identical PDF application should be submitted electronically on June 9 to the email address contacts@measurebih.com, with the following text in the Subject line: APPLICATION TO THE REQUEST FOR PROPOSALS – Impact Evaluation
Please direct all queries to:
Ye Zhang
yzhang2@impaqint.com
DISCLAIMER
Issuance of this RFP does not constitute an award commitment on the part of USAID MEASURE-BiH or USAID, nor does it commit USAID MEASURE-BiH or USAID to pay cost incurred in the submission of proposals.
Nothing in this document shall be construed as an offer by USAID MEASURE-BiH or USAID and no terms, discussions or proposals shall be binding on either party prior to execution of a definitive agreement. USAID MEASURE-BiH reserves the right to accept or reject any offer in part or full or to cancel the procurement without any obligation to any offeror.
Annex:
|
This web site is made possible by the support of the American People through the United States Agency for International Development (USAID).
The contents of the page are the sole responsibility of the editor and do not necessarily reflect the views of USAID or the United States Government.”
The contents of the page are the sole responsibility of the editor and do not necessarily reflect the views of USAID or the United States Government.”
This web site is made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of the page are the sole responsibility of the editor and do not necessarily reflect the views of USAID or the United States Government.”
Copyright© 2016 - USAID/BiH - MEASURE • All rights reserved