Educometrika
PT|EN

Psychometrics and Item Response Theory (IRT)

Measurement methods that ensure an assessment measures what it intends to and that results can be compared.

Educational assessmentData science in educationEducation indicatorsEducation data analysis

01What problem this area addresses

A test may contain items that are too easy, too hard or that do not discriminate well between students. Without psychometric analysis, these problems go unnoticed and affect the quality of the information.

Psychometrics, including Item Response Theory, makes it possible to check item quality, place students and items on the same scale and compare results across editions.

02Questions education departments often face

  1. Q1Are the assessment items working as expected?
  2. Q2Can this year's results be compared with previous years?
  3. Q3How can the local assessment be placed on the national scale?
  4. Q4What does each proficiency level mean in pedagogical terms?
  5. Q5Do any items favor or disadvantage a group of students?

03How Educometrika approaches the problem

  1. 01

    Instrument review

    Analysis of item and whole-test properties.

  2. 02

    Calibration

    Choosing and fitting the IRT model suited to the data and purpose.

  3. 03

    Scale and equating

    Building or linking scales and defining interpretable levels.

  4. 04

    Documentation

    Technical report with decisions, limitations and replicable procedures.

04Data and methods

Data

  • Item-level responses
  • Test booklets and design
  • Anchor items across editions
  • Student group variables

Methods

  • Classical Test Theory
  • 1-, 2- and 3-parameter IRT models
  • Equating and scale linking
  • Differential item functioning (DIF)
  • Dimensionality analysis

05Possible deliverables

Item analysis reportCalibrated parametersProficiency scale with pedagogical interpretationEquating proceduresAssessment technical report

06When this kind of support makes sense

  • When the system has its own assessment without psychometric analysis.
  • When results must be compared across years or with national assessments.
  • When building or reviewing an item bank.
  • When there are doubts about the validity of results.

07Related topics

08

Is your system facing a similar problem?

Describe the context and the open questions. The team will assess how it can contribute.

Talk to the team

All areas

Consulting for education departmentsTechnical support for state and municipal education departments across the whole policy cycle: from diagnosing the system to evaluating results.Education public policyDesign and implementation of education policies and programs that account for the real conditions of execution in schools.Education data analysisOrganizing and analyzing enrollment, attendance, progression and learning data to answer concrete management questions.Educational assessmentPlanning and developing learning assessments that produce useful information for teachers, schools and management.Education indicatorsBuilding indicators that show the system's situation clearly and allow targets to be tracked over time.Policy and program evaluationEvaluations that check whether a policy was implemented as planned and whether it produced the expected results.Data science in educationData integration, statistical and predictive models and monitoring systems to support decisions in school systems.Artificial intelligence in educationUsing artificial intelligence on concrete school system problems, with risk assessment and without adding complexity for schools.