Educometrika
PT|EN

Evaluation of education policies and programs

Evaluations that check whether a policy was implemented as planned and whether it produced the expected results.

Education public policyEducation indicatorsData science in educationConsulting for education departments

01What problem this area addresses

Many education programs are kept or ended without information about their effects. Comparing participating and non-participating schools without methodological care often leads to wrong conclusions.

Policy and program evaluation defines what can be answered with the available data and uses the most suitable design to separate the program's effect from other influences.

02Questions education departments often face

  1. Q1Is the program reaching the intended schools and students?
  2. Q2Did results improve because of the program or because of other factors?
  3. Q3For which groups does the program work better or worse?
  4. Q4Is it worth scaling the program? What is the cost relative to the result?
  5. Q5How can evaluation be planned from the start of the policy?

03How Educometrika approaches the problem

  1. 01

    Evaluability

    Checking what can be evaluated, with which data and in what timeframe.

  2. 02

    Evaluation design

    Choosing between implementation, outcome or impact evaluation.

  3. 03

    Data collection and analysis

    Using administrative data and, when needed, complementary collection.

  4. 04

    Recommendations

    Conclusions with an explicit level of confidence and suggested adjustments.

04Data and methods

Data

  • Program participation records
  • Learning and progression data
  • Questionnaires and interviews
  • Cost data

Methods

  • Implementation evaluation
  • Difference-in-differences
  • Propensity score matching
  • Regression discontinuity
  • Randomized experiments, where feasible
  • Cost-effectiveness analysis

05Possible deliverables

Evaluability assessmentEvaluation planImplementation reportOutcome or impact reportExecutive summary for leadership

06When this kind of support makes sense

  • Before deciding to scale, adjust or end a program.
  • When designing a new policy, to build in evaluation from the start.
  • When accountability to funders or oversight bodies is required.
  • After the first implementation cycle.

07Related topics

08

Is your system facing a similar problem?

Describe the context and the open questions. The team will assess how it can contribute.

Talk to the team

All areas

Consulting for education departmentsTechnical support for state and municipal education departments across the whole policy cycle: from diagnosing the system to evaluating results.Education public policyDesign and implementation of education policies and programs that account for the real conditions of execution in schools.Education data analysisOrganizing and analyzing enrollment, attendance, progression and learning data to answer concrete management questions.Educational assessmentPlanning and developing learning assessments that produce useful information for teachers, schools and management.Psychometrics and IRTMeasurement methods that ensure an assessment measures what it intends to and that results can be compared.Education indicatorsBuilding indicators that show the system's situation clearly and allow targets to be tracked over time.Data science in educationData integration, statistical and predictive models and monitoring systems to support decisions in school systems.Artificial intelligence in educationUsing artificial intelligence on concrete school system problems, with risk assessment and without adding complexity for schools.