8. Monitor Progress

8. Monitor Progress

Introduction

On Your Mark: Start Here with Grounding Concepts and the Evidence Base

In step 8, Title V agencies examine the results of efforts to gauge whether there has been improvement or change. Inputs include NOMs, NPMs, SPMs and ESMs, performance objectives and other quantitative and qualitative information. Potential outcomes may include altering activities and shifting resource allocations to address current levels of performance and availability of resources. Feedback loops between various stages of the process allow for continuous input and re-evaluation of the outputs.

Importance of Examining Impact

Monitoring outcome impact is at the core of a Title V agency's functions; it is essential for:

  • Ongoing Effectiveness. Monitoring outcome impact is an ongoing process that enables Title V agencies to continuously assess the effects of their interventions and programs. This continuous assessment helps ensure that public health efforts remain relevant, efficient, and aligned with community needs.
  • Quality Improvement. By closely monitoring outcomes, Title V agencies can identify areas in need of improvement. This quality improvement process involves using the data collected to refine strategies, enhance program delivery, and adapt to changing circumstances, ultimately leading to more effective and efficient interventions.
  • Evidence-Based Decision-Making. Monitoring outcome impact is fundamentally rooted in data collection and analysis. This data-driven approach provides Title V agencies with the evidence needed to make informed decisions. It helps in identifying which strategies are working and which may require adjustments or reallocation of resources.
  • Resource Allocation. Outcome monitoring informs resource allocation decisions. By assessing the impact of different programs and interventions, Title V agencies can allocate resources where they are most needed and where they can have the greatest impact, optimizing resource utilization.
  • Accountability and Transparency. Title V agencies are held accountable for the results of their initiatives. Monitoring outcome impact provides the evidence necessary to demonstrate accountability and transparency to partners, policymakers, and the public, showcasing the agency's commitment to achieving its goals.
  • Timely Intervention. Monitoring allows Title V agencies to identify emerging public health issues and respond swiftly. By closely tracking outcomes, departments can detect issues, assess the effectiveness of current interventions, and make necessary adjustments in real time to protect the public.
  • Outcomes and Reduction in Adverse Health Outcomes. Monitoring outcome impact aids in identifying differences in health outcomes among different populations. Title V agencies can use this information to tailor interventions and policies to ensure that communities benefit from public health efforts.
  • Continuous Learning and Innovation. The process of monitoring outcomes fosters a culture of continuous learning and innovation. Title V agencies can use the data to generate insights and identify trends that can inform the development of new strategies and interventions to address evolving public health challenges.

Barriers to Examining Impact

There are a number of barriers that can make it difficult to monitor outcome impact. These include:

  • Limited Resources. Inadequate funding, staff, and technology can hinder the collection, analysis, and reporting of data.
  • Data Collection Challenges. Gathering relevant data can be difficult, especially when dealing with hard-to-reach populations or fragmented data sources.
  • Data Privacy and Security Concerns. Balancing the need for data collection with privacy and security concerns can be a complex issue, limiting data availability.
  • Lack of Standardization. Inconsistent data collection methods and metrics across programs and agencies make it challenging to compare and aggregate data.
  • Short-Term Focus. Prioritizing short-term outcomes due to limited funding cycles and political pressures can limit the ability to monitor long-term impact.

The MCH Evidence Center has reviewed the literature to find strategies for monitoring progress for impact on outcomes:

Learn What Works

Get Ready: Immerse Yourself in Effective Strategies and Learn More

There are a number of evidence-based strategies that can be used to monitor outcome impact. These include:

  • Data Standardization. Establish standardized data collection methods, metrics, and reporting formats across programs and agencies to facilitate meaningful comparisons and analysis.
  • Outcome-Focused Data Collection. Ensure that data collection methods are designed to capture information on differences in health outcomes, allowing for targeted interventions.
  • Community Engagement. Actively engage with communities to build trust and enhance data collection efforts. Involving the community in the process can improve data quality and relevance.
  • Long-Term Follow-Up. Prioritize long-term monitoring to assess lasting program impact, ensuring that public health interventions continue to benefit communities beyond the short term.
  • Capacity Building. Invest in the training and development of staff to enhance their analytical skills, ensuring that staff can effectively analyze complex data.
  • Interagency Collaboration. Foster collaboration among different public health agencies and organizations to break down data silos and share insights, creating a more comprehensive view of public health outcomes.
  • Transparency and Accountability. Promote transparency and accountability in reporting outcomes to stakeholders and the public, emphasizing the responsible use of resources and the achievement of public health goals.
  • Utilization of Technology. Embrace technology, such as electronic health records, data analytics, and reporting tools, to streamline data collection and analysis, making the process more efficient and accurate.
  • Policy Consistency. Advocate for consistent public health policies and long-term commitments to minimize disruptions caused by political and policy shifts.

These trainings have been collected by the MCH Navigator:

Tools to Try

Get Set: Use these Tools to Move from Knowledge to Implementation

These tools and examples have been developed by the National MCH Workforce Development Center or collected by the MCH Digital Library:

  • 30/30 Action Plan. (National MCH Workforce Development Center). A simple way to monitor progress and record future action steps in ongoing work. The idea is that the team meets for 30 minutes every 30 days to discuss progress over the past month. The progress, roadblocks and other relevant ideas are noted on the form, along with an action plan for the following month.
  • Evaluation Resources:
    • Evaluation Learning Bundle. (MCH Navigator). This training spotlight uses the CDC framework as a conceptual model to organize learning opportunities. It presents introductions to the six steps of program evaluation in short video podcasts. You can also download materials from the CDC about each step. After reviewing the introductory material, you can access additional learning opportunities to gain knowledge and skills related to each step of the framework.
    • Evaluation Toolkit. ( MCH Digital Library). This deep-dive resource contains an evaluation primer, a collection of key resources, and an interactive Choose-and-Use tool to assist users in finding instructions on how to conduct evaluations and examples of successful evaluations from the field.
  • Measurement Tables. (National MCH Workforce Development Center). Tools to document an agency's plan to monitor progress toward goals over time. These can be used to further develop measurement strategies for activities described in logic models.
  • Results-Based Accountability Tools: (MCH Evidence Center). A suite of tools designed to help agencies develop stronger Evidence-based Strategy Measures (ESMs) and implement effective measurement processes:

These resources have been collected by the MCH Digital Library:

  • Measuring What Matters in Public Health. A Health Department’s Guide to Performance Management from The National Association of County and City Health Officials
  • Using Data to Advance Health Outcomes. This white paper demonstrates how stratifying quality data is an important tool for uncovering and responding to healthcare inconsistencies.
  • Planning on Outcome Evaluation. This web portal is part of the MENTOR online learning course on Program Evaluation and focuses on preparing to evaluate program outcomes, monitoring those outcomes, setting up comparison groups, evaluating impact, and identifying instruments to measure outcomes.

Title V Tools and Examples

Go: Learn from Your Peers Across the Country

Tools

These tools have been collected by AMCHP and the MCH Evidence Center:

Stories and Strategies for Peer-to-Peer Learning

These stories and strategies have been collected by AMCHP and the MCH Evidence Center related to Step 8:

  • Delaware: Reporting back to my community partners with transparency on where we were, where we are going and also their contributions to the success in the service delivery for CYSHCN.
  • South Dakota: Our program has a series of meetings scheduled for November to look at survey results with our stakeholders.
  • Commonwealth of the Northern Mariana Islands: Partner engagement. Ensure population representation. Keep partners in the loop (informed) of every stage of the NA process.

Share iconShare Your Stories and Strategies

Please take a moment to share your stories and strategies related to Step 8 of your needs assessment that you would like highlighted on this page. By sharing your stories, you help to document the needs assessment process for use by other Title V agencies during this cycle and will help preserve this knowledge for needs assessment processes in the future.

Share your stories and strategies here