Research Scientist at Educational Testing Service (ETS) / Jobsxl

Job Description:

Position Summary:

The Psychometrics, Statistics, and Data Sciences (PSDS) area of ETS?s Research & Development (R&D) Division invites applications for six Research Scientist positions at all levels. Specifically, we seek scientists in the areas of psychometrics, data science, artificial intelligence, statistics, computational statistics, and educational measurement to embark on exciting research to tackle the educational measurement challenges of the digital age. In this era, four realities will dominate assessment:

  • Digitally based assessments
  • Large volumes of data collected through test takers? assessment actions
  • Large volumes of other data about individuals who are being assessed
  • The growing capabilities of artificial intelligence.

In PSDS, we have launched a Measurement Frontiers Initiative to prepare ETS to thrive in the face of these realities, and we are looking for research scientists from multiple disciplines to lead this initiative alongside the national leaders in assessment, psychometrics and statistics who make up our current staff.

The research in the Measurement Frontiers Initiative will focus on the following research areas:

  • Next generation psychometrics for assessment and learning
    • Individualized learning and assessment
    • Modelling response processes and sequences of actions
    • Modelling educational big data
    • Advanced psychometric modeling for assessing complex constructs
    • Automated item generation
  • Digital assessment design, validity and fairness

    • Fairness and score comparability in AI based scoring and digital assessments
    • Fairness in assessments across cultures and languages
    • Adaptive assessment design
  • Advancing survey based group-score assessments

    • Modernizing group-score analysis methods and matrix sampling of assessment items


  • Develop novel psychometric, statistical, artificial intelligence, or machine learning methods for learning and assessment data.
  • Design and conduct scientific studies related to the focus areas of the Measurement Frontiers Initiative.
  • Consult and collaborate on problems related to the scientist?s expertise that arise in the context of other ETS research, ETS testing programs, or ETS corporate management concerns.
  • Pursue or collaborate in interdisciplinary projects, transferring techniques to new contexts or connecting scientific fields that are usually separate.
  • Serve as a subject matter expert in representing research results and the Measurement Frontiers Initiative to ETS corporate leadership, clients, and potential funding agencies.
  • Develop or assist in developing proposals for external and internal research grants and obtain financial support for new or continuing research activities. Prepare initial and final proposal and project budgets.
  • Contribute to advancing innovations in operational assessment programs.
  • Manage staff and budgets for research project and grants.
  • Mentor junior staff, including research assistants and interns, guiding their work, providing counsel, and facilitating collaboration.
  • Participate in dissemination activities by publishing papers in peer-reviewed journals and in the ETS Research Report series; issuing progress and technical reports; presenting seminars at major conferences and at ETS; or using other appropriate communication platforms (e.g., social media or books and chapters) that impact practice in the field or at ETS.
  • Develop professional relationships as a representative, consultant or advisor to external advisory and policy boards and councils, research organizations, educational institutions and educators.
  • Provide professional leadership in and foster collaboration within the research community at large.

Required Experience:


A Ph.D. or a Doctorate in psychometrics, educational measurement, statistics, mathematics, computer science, or a related field is required. Rank of the position (i.e., Junior, Full, or Senior) will be commensurate with years of progressively independent substantive research in the area of statistics, artificial intelligence, computer science, or education. For Junior level, a minimum of one year of experience is required. Research experience applying skills to measurement, education or learning is preferred. Candidates must be skilled in modern computer programming and be able to work effectively as a research team member.

Skills Needed:

  • Next generation psychometrics for assessment and learning

    • Some combination of the following: psychometrics, machine learning/AI, computational statistics, applied data science, systems engineering, data engineering, data visualization/graphic design, and software development
  • Digital assessment design, validity and fairness

    • Some combination of the following: theoretical psychometrics, educational measurement, fairness, validity, computational statistical, machine learning/AI, applied data science
  • Advancing survey based group score assessments

    • Some combination of the following: advanced psychometric modeling, computational statistics, numerical methods/programming, working with complex samples

EDUCATIONAL TESTING SERVICE is an Equal Opportunity and Affirmative Action Employer of Women and Minorities.

EDUCATIONAL TESTING SERVICE is an Equal Opportunity and Affirmative Action Employer of protected Veterans and Individuals with Disabilities.

EDUCATIONAL TESTING SERVICE is a Drug-free workplace.

From: Educational Testing Service

Source link