The Background
The Kansas Health Fund (KHF) and United Methodist Health Ministry Fund (herein the Health Fund) partnered with EduDream to conduct a developmental evaluation to capture their grantee’s strategies and approaches to developing advocacy agendas.
In 2021, KHF and the Health Fund partnered to launch the Kansas Advocates for Equity, Education, and Health (KAEEH), a three-year initiative to support four organizations in co-creating and implementing a shared advocacy agenda designed to reduce barriers to academic success and improve educational outcomes. The initiative focused on Black, Indigenous, and People of Color (BIPOC) families and children ages birth through eight, and prioritized community engagement and partnerships.
KHF partnered with EduDream to design and implement a multi-year evaluation detailing the impacts of organizational capacity building, network development, and the extent to which the collective agenda developed supports greater equity in educational outcomes for Kansas students.
The Process
Throughout the evaluation, EduDream applied Culturally Responsive and Equitable Evaluation (CREE) principles. We began with engaging grantees to co-create the evaluation plan, which included questions, data sources, and a data collection plan. With ongoing feedback from the funders and grantees, EduDream adjusted data collection tools and methods to reflect learnings and areas identified for further exploration.
The following research questions guided the evaluation:
- What is each grantee’s process for developing advocacy agendas that reflect equity in educational and health outcomes?
- What approaches did grantees take to staffing, capacity, and infrastructure (i.e., hiring, funding, or resources/support)?
- What are grantee coalition structures and functions, and how do they evolve over time?
- To what extent did grantees make progress in implementing their advocacy agenda?
- What anticipated outcomes were achieved?
Phase 1: Discovery and Planning
During the first year of the initiative, EduDream attended calls between grantees and the technical assistance provider, who supported policy and programmatic efforts. We listened, documented takeaways, and learned a tremendous amount about the grantees’ organizational structure, work, progress, and challenges. This led to EduDream developing trusting relationships with the grantees.
Phase 2: Research Design
From those calls and document reviews, EduDream outlined an evaluation plan and embedded feedback loops with grantees through facilitated conversations. The finalized two-year developmental evaluation plan reflected grantees’ feedback on the research questions, data sources, and data collection plan.
Phase 3: Research Preparation and Implementation
The evaluation plan, which would capture the grantees’ efforts in driving their advocacy agendas forward, included several data collection methods such as one-on-one bi-annual grantee interviews, monthly grantee learning logs, document reviews, and other facilitated conversations and interviews with key stakeholders, including funders and technical assistance providers.
As the evaluation was implemented and funders and grantees provided ongoing feedback, EduDream adjusted data collection tools and methods to reflect learnings and areas identified for further exploration.
Phase 4: Reporting and Communicating Findings
EduDream provided KHF and the Health Fund with an executive summary and a comprehensive evaluation report that presented our findings. We also created mini-case studies that captured grantees’ advocacy successes and challenges throughout the three-year initiative.
“I think the case studies and final report were top quality and provided insight into the organizations’ accomplishments. I appreciated the report writing skills and ability to keep the project moving forward with thoughtful questions.” – KHF Staff
To conclude the project, EduDream led a final presentation with the funders and grantees. The presentation included an overview of the key evaluation findings and grantee achievements, and allowed time for reflections and discussions among attendees.
The Impact
KHF and the Health Fund now have a holistic understanding of how their grantees’ advocacy agendas, collectively and individually, led to greater equity in educational outcomes for Kansas students. While this year marks the end of their three-year initiative, the funders gained insights into how to better support their grantees and structure similar initiatives in the future.