Step 12: Analyze Data and Adjust Plan Based on Findings

Objectives

• Ensure findings of supervision, monitoring, and evaluation inform planning and implementation approaches.

Outputs

• On-going revision of CD design.

• Regularly review meetings at the district, regional, and national levels.
• Guide distribution point staff—through training and supervision opportunities—to self-reflect on certain key data and consider what lessons they can learn for their own performance.

• Plans made at the start of a new CD model can be optimistic (and hopefully pragmatic) plans, based on how planners expect the CD approach to work. It is the results of monitoring, supervision, and regular management meetings at health facility–, district-, and regional-levels, as well as at the national level, that will inform how the CD approach develops and improves. Expect that modification and improvements will be needed, and welcome the opportunities to strengthen the system.
• Look for opportunities to streamline implementation and reduce management burden and costs. For example, one country decided to switch to a transporter who could send automated SMS delivery notifications. Countries that have been doing school distribution for several years may not need to train everyone every year, unless the processes have changed significantly.
• It is important to understand that the performance of the CD system can vary—over time and between contexts. Plans that worked well one time, and in one place, may need to be adjusted. Therefore, even after a system has been up and running for several years, on-going monitoring, regular reviewing, and adjusting will probably be needed.
• It is also important not to assume that because a system has been running for several years, it is running well enough. Many countries have had ANC and EPI distributions in place for some time; in many cases, the quality is poor and there are many opportunities for improvement.
• Key issues to examine in data are—
  • Stockout frequencies.
  • Access issues—redemption of coupons by community members or vouchers in a voucher system; proportions of expected target groups participating.
  • Proportion of those eligible for a net and visiting distribution points who actually receive a net.
  • Coherence of the audit trail.
  • Issues frequently arising in supervision and monitoring visits.
  • Equity of those receiving nets.
• Who should review and for what at what levels?
  • Distribution personnel can review their own performance by reflecting on issues that include numbers distributed versus numbers expected to be distributed, issues faced when summarizing distribution for monthly reports (which prompt them to improve accuracy and completeness of daily registers), common questions being faced from communities by recipients (which might give insights into appropriate modifications or additions to the key messages they are giving).
  • Health facility personnel may have a role in reviewing lower-level data if they are acting as supervisors to community distribution points. In these cases, specific aspects to consider are numbers of coupons distributed and unexpected fluctuations, existing stock, numbers of ITNs distributed and stocks remaining, accuracy and completeness of reports being received, and feedback and questions being received from community-level personnel. Findings will help inform guidance given during supervision visits and refresher trainings, and recommendations made up the supervision chain to districts about the model design.
  • The district level will be a key point for reviewing—for district-wide data—the same points mentioned above. District teams may hold monthly review meetings to examine the issues above and decide on actions. District teams will have some leeway to modify the approaches used in their district, as appropriate, based on the findings. Some recommendations may be best reported back up the supervision chain to regional- or national-levels to see amendments to the model nationally, if similar findings are being reporting throughout the country.
• Some data will be useful for future program planning design, but other data will be versus needs for on-going management of activities. Examine both types of data at all levels; the latter may lead to quick action at the district- or lower-levels, the former is likely to lead to slower change; the district- or lower-level personnel may not be able to make changes; nevertheless, it is important for these personnel to examine these aspects in order to feed findings and recommendations up the supervision chain.
• Donors may have specific data that requires additional efforts. Clarify this during the donor awards.
• It is important to ensure that the experiences of personnel active within the CD system inform the analysis and redesign, not just numeric data. Include representatives from the various implementation levels in the review meetings.
• At lower levels, hold review meetings more regularly (certainly monthly at the district level); coordination committee meetings can take place quarterly at the national level and can be a forum for review and redesign discussions; however, regardless of these more frequent meetings, it is advisable to have a comprehensive review and redesign discussion annually during an annual review meeting that joins personnel from a range of levels and focuses on the range of aspects involved in the distribution model.

• Health facility personnel have a vital role to play in this process, not just for health facility distribution, but also because of their likely role in supporting community-based and school-based distributions. Health facility staff are often overworked and do not have time for review and reflection. Try to build opportunities for this into monthly review meetings.
• The VectorWorks document, Guide to Health Facility-based Distribution of Insecticide-treated Nets, provides more detail on these issues..

• Because of the large number of personnel involved in community-based distribution, it may be difficult to give a reasonable proportion of these personnel a voice in review meetings. Supervision findings will be particularly key to act as a channel to feed community-level experience into review and design.
• The VectorWorks document, Guide to Community-based Distribution of Insecticide-treated Nets, provides more detail on these issues.

• Because this model often has distribution only once per year, ensure that a yearly post distribution review meeting is scheduled to assess performance and allow for a timely revision of plans for the following year.
• The VectorWorks document, Guide to School-based Distribution of Insecticide-treated Nets, provides more detail on these issues.