Monitoring and Evaluation Officer
Starting Hourly Rate Range: $35.24 - $48.50
Employee group: Casual / On Call
Employee subgroup: S1 Hrly Non-Exmpt
Location: 04-MD:School of Medicine Campus
Department name: 10003321-SOM Admin Registrar General Admin
Personnel area: School of Medicine
The mission of the Office of Assessment and Evaluation (OAE) is to enhance the educational programs in the Johns Hopkins University School of Medicine (JHUSOM) through evidence-based best practices in the assessment of learners and the evaluation of programs. OAE partners with JHUSOM faculty and staff members, administrators, and learners to design, launch, and maintain assessment and evaluation initiatives in graduate biomedical education, postdoctoral education, undergraduate medical education, graduate medical education, and continuing medical education. OAE also supports educational activities initiated by the Simulation Center, the Institute for Excellence in Education, the Office of Student Affairs, the Office of Faculty Development, and the Professional Development and Career Office.
As part of its core mission, OAE:
- Supports the assessment of learners and the evaluation of programs through the selection, design, vetting, and implementation of assessment and evaluation tools (e.g., rubric design, multiple-choice question exam development, survey design, interview guide development)
- Facilitates understanding of the assessment of learners and the evaluation of programs through the analysis, reporting, and use of related data (e.g., statistical analysis of quantitative data, thematic content analysis of qualitative data)
- Conducts faculty development activities in areas related to learner assessment and program evaluation (e.g., validity, reliability, exam blueprints, standard setting, test and item analysis)
As a secondary aim, OAE also provides consultation in research design, statistical analysis and interpretation, and manuscript preparation intended for scholarly reports in medical and biomedical education, as projects relate to assessment and evaluation initiatives at JHUSOM.
The Program Evaluation Specialist supports learner assessment and program evaluation initiatives across all JHUSOM educational programs and activities, with a heavier focus on activities related to program evaluation planning, instrument development, and analysis, reporting, and utility of quantitative and qualitative program evaluation data.
This position reports to the Senior Program Evaluation Specialist. They will work closely with other OAE team members and with faculty members, staff members, and learners throughout JHUSOM.
Specific duties & responsibilities
Specific responsibilities of the Program Evaluation Specialist include, but are not limited to, the following.
Program Evaluation Planning, Systems Design, and Instrument Development
- Participate in activities related to the development of evaluation plans and the design of comprehensive evaluation systems
- Participate in activities related to the development, implementation, and use of program evaluation instruments/data collection methods (e.g., surveys, focus groups, interviews)
- Serve as a focus group moderator or interviewer
- Complete literature searches and other information gathering activities to identify existing instruments, items, or protocols that may be adopted or adapted, as relevant
- Create instruments in Qualtrics and handle related dissemination and data collection efforts
Analysis and Reporting
- Complete appropriate descriptive and inferential statistical analyses of quantitative program evaluation and learner assessment data, and content analyses of qualitative program evaluation data
- Create technical or scholarly reports and presentations of quantitative and qualitative program evaluation and learner assessment data, including appropriate use of text, figures, and tables
- Assist in the analysis and reporting of program evaluation and learner assessment data for inclusion in documents for the Student Assessment and Program Evaluation (SAPE) Committee during curricular reviews, as needed
- Participate in analysis and technical report preparation related to cyclical reporting activities (e.g., Comprehensive Clinical Skills Exam, Clerkship Standardized Patient Exams, Doctoral Exit Survey, and Postdoctoral Entrance and Exit Surveys)
- Assist in facilitating assessment-related activities with faculty members, staff members, and learners (e.g., assessment planning and mapping, criterion-referenced scoring, standard setting, non-compensatory grading, rater training, performance-based assessment development, best practices for writing multiple-choice questions, understanding and utilizing test and item analysis data)
- Maintain professional and effective working relationships with JHUSOM educational program stakeholders and with partnering units, including the JHUSOM Office of Information Technology, Office of Medical Student Affairs, Office of Online Education, and Office of the Registrar
PhD degree in Education, Educational Psychology, Quantitative Psychology, Psychology, Social Sciences, Measurement, or a related field and at least three years of related experience; or Master’s degree in Education, Educational Psychology, Quantitative Psychology, Psychology, Social Sciences, Measurement, or a related field and at least eight years of related experience.
- Previous experience working in higher education, especially in medical or biomedical education
- Familiarity with an academic healthcare environment
- Knowledge of standards and policies related to the MSCHE, LCME, ACGME, NIH, or other accrediting bodies and funding agencies
- Knowledge of additional systems related to the collection, management, analysis, and/or reporting of program evaluation data (e.g., B-Line, Blackboard, New Innovations, Qualtrics, REDCap, Tableau)
- Intermediate skills in data visualization
- Experience in learner assessment activities in an educational setting
Special knowledge, skills, and abilities
- Experience in program evaluation activities in an educational setting
- Experience in writing technical or scholarly reports
- Experience in qualitative data analysis
- Proficiency with the use of Microsoft Word, Excel, and PowerPoint
- Intermediate to advanced skills in the use of statistical analysis software (e.g., SAS, SPSS)
- Knowledge of inferential statistical analyses, including parametric and non-parametric approaches
- Ability to work with a diverse group of learners, faculty and staff members, and healthcare professionals
- Ability to prioritize tasks, manage time, and meet deadlines effectively
- Strong organizational skills and keen attention to detail
- Strong written and verbal communication skills
- Ability to communicate complex analyses and findings to a lay audience
- Effective use of diplomacy, tact, and judgment
The successful candidate(s) for this position will be subject to a pre-employment background check.
If you are interested in applying for employment with The Johns Hopkins University and require special assistance or accommodation during any part of the pre-employment process, please contact the HR Business Services Office at email@example.com. For TTY users, call via Maryland Relay or dial 711.
The following additional provisions may apply depending on which campus you will work. Your recruiter will advise accordingly.
During the Influenza ("the flu") season, as a condition of employment, The Johns Hopkins Institutions require all employees who provide ongoing services to patients or work in patient care or clinical care areas to have an annual influenza vaccination or possess an approved medical or religious exception. Failure to meet this requirement may result in termination of employment.
The pre-employment physical for positions in clinical areas, laboratories, working with research subjects, or involving community contact requires documentation of immune status against Rubella (German measles), Rubeola (Measles), Mumps, Varicella (chickenpox), Hepatitis B and documentation of having received the Tdap (Tetanus, diphtheria, pertussis) vaccination. This may include documentation of having two (2) MMR vaccines; two (2) Varicella vaccines; or antibody status to these diseases from laboratory testing. Blood tests for immunities to these diseases are ordinarily included in the pre-employment physical exam except for those employees who provide results of blood tests or immunization documentation from their own health care providers. Any vaccinations required for these diseases will be given at no cost in our Occupational Health office.
Equal Opportunity Employer
Note: Job Postings are updated daily and remain online until filled.
EEO is the Law
Important legal information
School of Medicine - East Baltimore Campus