8. Monitor Progress

8. Monitor Progress

Introduction

On Your Mark: Start Here with Grounding Concepts and the Evidence Base

In step 8, Title V agencies examine the results of efforts to gauge whether there has been improvement or change. Inputs include NOMs, NPMs, SPMs and ESMs, performance objectives and other quantitative and qualitative information. Potential outcomes may include altering activities and shifting resource allocations to address current levels of performance and availability of resources. Feedback loops between various stages of the process allow for continuous input and re-evaluation of the outputs.

Importance of Examining Impact

Monitoring outcome impact is at the core of a Title V agency's functions; it is essential for:

  • Ongoing Effectiveness. Monitoring outcome impact is an ongoing process that enables Title V agencies to continuously assess the effects of their interventions and programs. This continuous assessment helps ensure that public health efforts remain relevant, efficient, and aligned with community needs.
  • Quality Improvement. By closely monitoring outcomes, Title V agencies can identify areas in need of improvement. This quality improvement process involves using the data collected to refine strategies, enhance program delivery, and adapt to changing circumstances, ultimately leading to more effective and efficient interventions.
  • Evidence-Based Decision-Making. Monitoring outcome impact is fundamentally rooted in data collection and analysis. This data-driven approach provides Title V agencies with the evidence needed to make informed decisions. It helps in identifying which strategies are working and which may require adjustments or reallocation of resources.
  • Resource Allocation. Outcome monitoring informs resource allocation decisions. By assessing the impact of different programs and interventions, Title V agencies can allocate resources where they are most needed and where they can have the greatest impact, optimizing resource utilization.
  • Accountability and Transparency. Title V agencies are held accountable for the results of their initiatives. Monitoring outcome impact provides the evidence necessary to demonstrate accountability and transparency to partners, policymakers, and the public, showcasing the agency's commitment to achieving its goals.
  • Timely Intervention. Monitoring allows Title V agencies to identify emerging public health issues and respond swiftly. By closely tracking outcomes, departments can detect issues, assess the effectiveness of current interventions, and make necessary adjustments in real time to protect the public.
  • Outcomes and Disparity Reduction. Monitoring outcome impact aids in identifying disparities in health outcomes among different populations. Title V agencies can use this information to tailor interventions and policies to ensure that communities benefit from public health efforts.
  • Continuous Learning and Innovation. The process of monitoring outcomes fosters a culture of continuous learning and innovation. Title V agencies can use the data to generate insights and identify trends that can inform the development of new strategies and interventions to address evolving public health challenges.

Barriers to Examining Impact

There are a number of barriers that can make it difficult to monitor outcome impact. These include:

  • Limited Resources. Inadequate funding, staff, and technology can hinder the collection, analysis, and reporting of data.
  • Data Collection Challenges. Gathering relevant data can be difficult, especially when dealing with hard-to-reach populations or fragmented data sources.
  • Data Privacy and Security Concerns. Balancing the need for data collection with privacy and security concerns can be a complex issue, limiting data availability.
  • Lack of Standardization. Inconsistent data collection methods and metrics across programs and agencies make it challenging to compare and aggregate data.
  • Short-Term Focus. Prioritizing short-term outcomes due to limited funding cycles and political pressures can limit the ability to monitor long-term impact.

The MCH Evidence Center has reviewed the literature to find strategies for monitoring progress for impact on outcomes:

Learn What Works

Get Ready: Immerse Yourself in Effective Strategies and Learn More

There are a number of evidence-based strategies that can be used to monitor outcome impact. These include:

  • Data Standardization. Establish standardized data collection methods, metrics, and reporting formats across programs and agencies to facilitate meaningful comparisons and analysis.
  • Outcome-Focused Data Collection. Ensure that data collection methods are designed to capture information on disparities and health outcomes, allowing for targeted interventions.
  • Community Engagement. Actively engage with communities to build trust and enhance data collection efforts. Involving the community in the process can improve data quality and relevance.
  • Long-Term Follow-Up. Prioritize long-term monitoring to assess lasting program impact, ensuring that public health interventions continue to benefit communities beyond the short term.
  • Capacity Building. Invest in the training and development of staff to enhance their analytical skills, ensuring that staff can effectively analyze complex data.
  • Interagency Collaboration. Foster collaboration among different public health agencies and organizations to break down data silos and share insights, creating a more comprehensive view of public health outcomes.
  • Transparency and Accountability. Promote transparency and accountability in reporting outcomes to stakeholders and the public, emphasizing the responsible use of resources and the achievement of public health goals.
  • Utilization of Technology. Embrace technology, such as electronic health records, data analytics, and reporting tools, to streamline data collection and analysis, making the process more efficient and accurate.
  • Policy Consistency. Advocate for consistent public health policies and long-term commitments to minimize disruptions caused by political and policy shifts.

These trainings have been collected by the MCH Navigator:

Tools to Try

Get Set: Use these Tools to Move from Knowledge to Implementation

These tools and examples have been developed by the National MCH Workforce Development Center or collected by the MCH Digital Library:

  • Evaluation Resources:
    • Evaluation Learning Bundle. This training spotlight uses the CDC framework as a conceptual model to organize learning opportunities. It presents introductions to the six steps of program evaluation in short video podcasts. You can also download materials from the CDC about each step. After reviewing the introductory material, you can access additional learning opportunities to gain knowledge and skills related to each step of the framework.
    • Evaluation Toolkit. This deep-dive resource contains an evaluation primer, a collection of key resources, and an interactive Choose-and-Use tool to assist users in finding instructions on how to conduct evaluations and examples of successful evaluations from the field.
  • Measurement Tables. A Measurement table documents your plan to monitor progress toward goals over time. If you have already completed logic models for each domain, a Measurement Table can be used to further flesh out ideas for measuring each activity described in the logic model.
  • Results-Based Accountability Tools:

These resources have been collected by the MCH Digital Library:

  • Measuring What Matters in Public Health. A Health Department’s Guide to Performance Management from The National Association of County and City Health Officials
  • Using Data to Reduce Disparities and Improve Quality. Advancing Health Outcomes. This white paper demonstrates how stratifying quality data is an important tool for uncovering and responding to healthcare disparities.
  • Types of Evaluation. CDC. This factsheet outlines the most common types of evaluation, when to use it type, what it shows, and why the collected data are useful.
  • Planning on Outcome Evaluation. This web portal is part of the MENTOR online learning course on Program Evaluation and focuses on preparing to evaluate program outcomes, monitoring those outcomes, setting up comparison groups, evaluating impact, and identifying instruments to measure outcomes.

Title V Tools and Examples

Go: Learn from Your Peers Across the Country

Tools

These tools have been collected by AMCHP and the MCH Evidence Center:

Stories and Strategies for Peer-to-Peer Learning

These stories and strategies have been collected by AMCHP and the MCH Evidence Center related to Step 8:

  • Delaware: Reporting back to my community partners with transparency on where we were, where we are going and also their contributions to the success in the service delivery for CYSHCN.
  • South Dakota: Our program has a series of meetings scheduled for November to look at survey results with our stakeholders.
  • Commonwealth of the Northern Mariana Islands: Partner engagement. Ensure population representation. Keep partners in the loop (informed) of every stage of the NA process.

Share iconShare Your Stories and Strategies

Please take a moment to share your stories and strategies related to Step 8 of your needs assessment that you would like highlighted on this page. By sharing your stories, you help to document the needs assessment process for use by other Title V agencies during this cycle and will help preserve this knowledge for needs assessment processes in the future.

Share your stories and strategies here