A Data-Driven Look at ELO-P for Program Directors

The opportunity gap in American education starts early and grows wider with each passing year. 

This inequity creates cascading challenges. For example, the chronic absence rate in California has doubled from pre-pandemic levels. Working families struggle to find safe, enriching environments for their children during the critical hours before and after school. And students who could benefit most from additional learning time and support often have the least access to it.

However, programs like the Expanded Learning Opportunities Program (ELO-P) can help bridge the divide. Research shows that afterschool programs play a critical role in supporting academic success, with 40% of participants—particularly those who attend consistently—demonstrating gains in math and reading.

Access to quality summer programming also significantly impacts the achievement gap between low-income students and their more affluent peers. Studies suggest that as much as two-thirds of the 9th-grade reading achievement gap may stem from disparities in summer learning opportunities during the elementary years.

Starting with an initial investment of $1.8 billion in 2021-22, California sought to address these challenges by investing in ELO-P. This commitment aims to provide comprehensive before-school, after-school, and summer programming for California's most underserved students.

The early results have been positive. Recent surveys show that 97% of school districts have expanded their programs, with many doubling or tripling their capacity. Perhaps most importantly, early data shows participating students attending school more regularly, feeling more connected to their education, and showing greater academic engagement.

California doubled down with an unprecedented $4 billion annual investment—more than quadrupling previous funding—the state launched an ambitious effort to provide comprehensive before-school, after-school, and summer programming for its most underserved students.

Attendly has compiled the following analysis to examine how California is building effective expanded learning programs, the evidence of their impact, and the key ingredients for success. Drawing from extensive research and recent surveys of over 115 organizations implementing these programs, it provides a roadmap for creating high-quality expanded learning opportunities that can help close the opportunity gap and support student success.

Program Quality Standards and Framework

California has established a comprehensive framework of 12 Quality Standards for expanded learning programs, developed through collaboration between the California Department of Education's After School Division and the California AfterSchool Network. These standards are divided into two key categories that address both direct student experience and organizational effectiveness.

Point-of-Service Quality Standards

1. Safe and supportive environment: Focuses on creating a secure, nurturing atmosphere that addresses students' various developmental needs.
2. Active and engaged learning: Emphasizes interactive, meaningful activities that foster collaboration and broaden students' perspectives.
3. Skill building: Aims to set high standards and intentionally develop 21st-century skills through targeted curricula and activities.
4. Youth voice and leadership: Encourages student involvement in program design and offers genuine leadership opportunities.
5. Healthy choices and behaviors: Promotes overall well-being by educating students about nutrition, physical activity, and other health-related topics.
6. Diversity, access and equity: Strives to create an inclusive environment that respects and celebrates all forms of diversity.

Programmatic Quality Standards

1. Quality staff: Emphasizes recruiting and retaining skilled personnel, with a focus on ongoing professional development.
2. Clear vision, mission and purpose: Stresses the importance of well-defined goals and measurable outcomes based on stakeholder input.
3. Collaborative partnerships: Highlights the need for building strong relationships with families, schools, and community organizations.
4. Continuous quality improvement: Utilizes diverse data sources to assess and enhance program effectiveness regularly.
5. Program management: Focuses on implementing sound administrative practices and policies that align with grant requirements.
6. Sustainability: Emphasizes building lasting community partnerships and securing various forms of support for long-term program viability.

What makes these standards particularly actionable is their detailed description of what quality looks like at multiple levels. For example, under "Youth Voice and Leadership," the standards specify that:

At the program level:

• "The program provides participants with opportunities and space to share their viewpoints, concerns, or interests in order to impact program practices or policies."

At the staff level:

• "Staff encourage and engage participants on a regular basis to share their perspectives regarding program design, what they want to learn and the quality of their experience in the program."

At the participant level:

• "Participants engage in authentic and meaningful leadership roles that are supported by staff and celebrated by the program."

As emphasized by State Superintendent Tom Torlakson, "This bold initiative provides a road map for improving expanded learning throughout California."

Read this article on extended learning program trends for additional insights into best practices and innovations driving success in after-school and summer programming. We go deeper into how these quality standards are shaping the future of expanded learning, explore emerging trends and strategies that programs are adopting to enhance their effectiveness.

Evidence of Program Impact

Recent data from spring 2024 demonstrates that California's expanded learning programs, particularly through the Expanded Learning Opportunities Program (ELO-P), are making significant strides in both reach and effectiveness.

Access and Reach

The implementation of ELO-P has dramatically expanded program access for underserved students. According to statewide survey data:

• 97% of local educational agencies (LEAs) and 88% of community-based organizations (CBOs) have increased their student service through new programs or expansion of existing ones
• Programs are effectively reaching high-need populations, with 81% of LEAs and 75% of CBOs reporting that 70% or more of their students qualify for free and reduced price meals
• 95% of LEAs have expanded their operations to include summer and intersession programs, addressing a crucial gap in year-round support

Specific examples demonstrate the scale of expansion:

• Think Together added 107 new programs (27% increase) across 61 districts
• Salinas City Elementary School District increased enrollment from 400 to 2,000 students
• YMCA of San Diego County grew from serving 8,000 to 14,000 students

"ELO-P funding has been a game changer for expanding and enhancing the support we provide to students and families, and has been a factor in recruiting families to our district," notes Jenel Edlund, Director of Special Projects at West Covina Unified School District. "The after school time has become an essential part of our plan for student success, especially around developing the whole child."

Early Indicators of Academic Impact

Programs are showing promising results in two key areas:

1. Attendance Improvements

• arc programs documented a 6.4-day increase in school attendance over one year
• Program participants showed higher attendance rates than non-participants in Oakland Unified and Vallejo City Unified
• Three out of four students reported increased motivation to attend school

2. Academic Engagement

• 82% of Oakland Unified program participants reported feeling more connected to school
• 81% of families at EDMO reported improved homework completion
• 83% of families noted their children showed increased excitement about school

"The message from families is clear," says Michael Taylor, Expanded Learning Opportunities Coordinator at San Rafael City Schools. "Without ELO-P they would not be able to get new jobs, keep their jobs or go to school. ELO-P allows students to be with caring adults who help with homework, solve problems, make friends and have access to enriching programs they might not otherwise have access to."

Quality Improvement Process

The California Department of Education emphasizes that effective program improvement involves a continuous cycle of assessment, planning, and improvement. This systematic approach helps programs achieve and maintain high quality while adapting to changing needs.

Assessment Phase

The first step involves comprehensive program assessment through multiple strategies:

1. Selecting Assessment Tools Programs can choose from several validated assessment instruments, including:

• California After School Program Quality Self-Assessment Tool
• Youth Program Quality Assessment
• Assessment of Program Practices Tool
• Out-of-School Time Observation Instrument

2. Engaging Key Stakeholders Programs should involve a diverse team including:

• School administrators and teachers
• Program staff and site coordinators
• Students and parents
• Community partners
• Advisory groups

3. Data Analysis Programs should examine their data to answer key questions:

• What are the clear data trends?
• What are the program's strengths?
• Which areas require improvement?
• What can be addressed immediately versus long-term?

Planning Phase

After assessment, programs develop targeted improvement plans:

1. Focus Areas

• For programs new to the improvement process, it's recommended to focus on just one standard
• More experienced programs may address 2-3 high-priority needs
• All improvements should align with the Quality Standards

2. Specific Objectives Plans should include:

• Measurable goals and outcomes
• Concrete activities for each objective
• Clear assignment of responsibilities
• Realistic timelines
• Required resources and support

As noted in the CDE guidance: "Each site should identify and define the goals, objectives, and outcomes to be achieved (along with concrete activities, identification of individual[s] responsible, and a reasonable timeline for meeting those goals)."

Implementation Phase

The final phase focuses on putting the plan into action:

1. Progress Monitoring Programs should track implementation through:some text
   • Formal assessments: Standardized evaluation tools to measure program quality
   • Informal discussions: Spontaneous check-ins with staff and participants
   • Direct observations: Unannounced visits to program activities
   • Staff meetings: Structured team dialogues focused on implementation challenges
   • Focus groups: Facilitated discussions with diverse stakeholder representatives
2. Staff Support Programs provide support through:some text
   • Professional development: Tailored workshops and training sessions
   • Mentoring: Pairing novice staff with experienced team members
   • Coaching: Individualized performance improvement plans
   • Access to resources: Curated libraries of best practices and materials
   • Curriculum support: Expert guidance on activity design and implementation
3. Impact Assessment Programs should regularly document:some text
   • Progress toward goals: Quantifiable metrics aligned with program objectives
   • Impact on service delivery: Changes in efficiency and effectiveness of operations
   • Benefits to students: Measurable improvements in academic and social outcomes
   • Areas needing adjustment: Identified gaps between current and desired performance

Measuring Success

Available Assessment Tools

The California AfterSchool Network, in partnership with the Department of Education, has evaluated seven major assessment tools that align with the state's Quality Standards. Each tool offers unique strengths and approaches to measuring program quality.

Key Assessment Tools Include:

1. Self-Assessment Tools:

• California After School Program Quality Self-Assessment Tool (CAN-QSA)
• New York Program Quality Self-Assessment Tool (NYSAN-QSA)
• California High School Program Quality Self-Assessment Rubric (CAN-QSAR)

2. Observation-Based Tools:

• Program Quality Assessment (PQA)
• Assessment of Program Practices Tool (APT)
• Out-of-School Time Observation Instrument (OST)
• Promising Practices Rating System (PPRS)

"No one tool will meet all of a program's information needs," notes the CAN Crosswalk report. "Programs should use multiple strategies, including self-assessment and observation of program activities."

Data Collection Gaps

Current research identifies several areas where data collection and analysis could be strengthened:

1. Current Limitations The Spring 2024 ELO-P survey highlights key gaps in statewide data collection:

• Lack of standardized participation tracking
• Limited ability to connect program participation with academic outcomes
• Inconsistent demographic data collection across programs

2. Recommended Improvements The Partnership for Children & Youth recommends:

• Including expanded learning participation data in the state's CALPADS system
• Tracking correlation between program participation and chronic absenteeism
• Monitoring academic performance of participants over time
• Standardizing demographic data collection across programssome text
  • As noted in their recommendations: "Collect and report state-level data on program participation, school-day attendance, demographics of ELO-P participants, including the percentage of unduplicated students participating, and how unduplicated families were notified about this opportunity to ensure ELO-P is impacting the students prioritized by the state."

3. Technical assistance needs to support effective data collection and use, programs need:

• Increased funding for technical assistance
• Support in blending different funding streams
• Help building programs from the ground up for new providers
• Guidance on coordinating with other educational initiatives

Path to Academic Impact

Theory of Change

The research from California's expanded learning programs suggests a clear pathway from program quality to student success. This pathway is supported by emerging evidence from the field:

1. High-Quality Programs Lead To:

• Increased School Attendancesome text
  • arc programs demonstrated a 6.4-day attendance increase
  • Three out of four students report increased motivation to attend school
• Greater School Engagementsome text
  • 82% of Oakland Unified participants feel more connected to school
  • 83% of EDMO families report their children are more excited about school
• Improved Academic Behaviorssome text
  • 81% of families report better homework completion
  • Students show increased participation in school-day activities

As noted by Armando Diaz, Vice President of Programs and Partnerships at Educare Foundation: "ELO-P funding has been instrumental in allowing us to provide a living wage for staff and deliver high-quality programs that support student success and address chronic absenteeism. By investing in our programs, supporters of ELO-P are not just investing in high-quality expanded learning programs; they're investing in the future of our youth and communities."

Next Steps for Programs

Based on Attendly’s analysis of program guidelines and the evidence of effectiveness, we recommend the following to strengthen their impact on student success, programs should focus on three key areas:

1. Quality Implementation

• Use the 12 Quality Standards as a framework for program design
• Implement the continuous improvement cycle consistently
• Engage all stakeholders in program development and assessment
• Maintain stable, well-trained staff through competitive compensation

2. Data Collection Programs need to systematically track:

• Student attendance patterns
• Program participation rates
• Academic engagement indicators
• School-day attendance correlation
• Student and family feedback

3. Evidence Building Programs should:

• Partner with research organizations
• Share program data with state systems
• Participate in evaluation studies
• Document successful practices

As Governor Gavin Newsom stated in May 2024: "California is transforming education to make schools a place where every family and student can succeed. Today we are expanding community schools across the state, offering everything from free meals twice a day, to mental health counseling, to before, after, and summer school programs to thousands more students."

The success of expanded learning programs in California demonstrates that when programs focus on quality implementation, engage in continuous improvement, and maintain strong partnerships, they can make a significant difference in students' educational experiences and outcomes.

‍