IT Consultant, Academic Data Engineer or Instructor

North Carolina, United States
26 Oct 2018
End of advertisement period
26 Dec 2018
Contract Type
Full Time

Job Description:

Duke University seeks an outstanding Data Engineer who will (a) facilitate new research projects through the creative integration of multiple data sources, (b) potentially work with faculty and students assisting with helping them to learn and understand the power of data engineering and (c) function as a core member of an emerging Data Analytics Practice at Duke that stimulates new uses of data analytics in instruction and research.  As an integral part of Duke’s Data Science community, the staff member will be expected to work closely with faculty, students, and other departments at Duke.  The staff member should be passionate about engineering ways to use new types of data to solve world problems, detail-oriented, and self-motivated to learn new skills and subjects as technology changes.
The staff member will have a central role in working with faculty and researchers to identify and leverage commonalities in support of new data-intensive degree programs (Duke’s Masters in Interdisciplinary Data Science (MIDS)) and other academic programs, both in the physical classroom and as online courses.  Together with personnel and support from Duke’s Office of Information Technology , the staff member will be part of a growing team focused on data analytics, and will bring to that team an emphasis on academic data analytics.

The staff member’s responsibilities would include:

  • Working with faculty to build, optimize and monitor current forms of data collection, and develop and integrate new forms of data collection
  • Working with researchers and faculty to optimize the transformation of collected data into formats appropriate for analysis
  • Working with researchers, faculty and IT developers to apply state-of-the-art approaches in cloud service, containerization and other techniques for scaling the remote staging, storage, computation and analysis of data sets, including at scale by hundreds or thousands of students in online courses
  • Providing expertise in data set hygiene to the institution, facilitating the cleanup and integration of potentially multiple incomplete, irregular, and partial data sources into interesting and useful data sources for analysis
  • Working with various offices to ensure that information is used complies with the regulatory
  • Helping develop and teach advanced courses in Data Engineering and Database Systems
  • Developing and running workshops for the campus community in different aspects of data management and wrangling
  • Advising student capstone projects in the MIDS and other programs on data engineering needs
  • Helping evaluate capstone projects with program faculty

To meet these responsibilities, the candidate should be an innovative problem solver, team-focused, highly organized, have the ability to handle-multiple and simultaneous tasks, and demonstrate an ability to stay calm and focused in emergencies.  The candidate should also enjoy working in collaborative relationships, and have an approachable and relatable demeanor that inspires trust and confidence in partners.


Master’s degree in mathematics, statistics, computer science or related field or equivalent experience. Doctorate preferred.

At least 5 years of working experience as a database engineering support personnel or a database engineering administrator within a fast-paced, complex setting.

The applicant must have expertise in the following areas:

  • distributed computing principles
  • legacy and modern database architectures
  • Hadoop-based technologies (e.g. MapReduce, Hive and Pig)
  • SQL-based technologies (e.g. PostgreSQL and MySQL)
  • NoSQL technologies (e.g. Cassandra and MongoDB)
  • Stream-processing systems (e.g. Storm or Spark-Streaming)
  • ETL tools and APIs
  • optimizing data storage and retrieval for specific use cases
  • testing and validating the accuracy of data transformations
  • cloud computing architectures, preferably with specific expertise in Microsoft Azure and AWS
  • programming in multiple programming languages, including Python and Java

Due to the applicant’s role interacting with students, faculty, and staff on projects, the applicant must also be able to demonstrate skills in:

  • Creative problem-solving that is sensitive to available time and resource constraints
  • Effective listening and communication
  • Project management

Duke University and Durham are located in the Research Triangle, a region that encompasses one of the nation's premier concentrations of academic, corporate, and public research. The Triangle region is rated among the most desirable areas in North America to live and work and has been identified by Money magazine as one of the "Best Places to Live" in the U.S.

Trustworthiness, respect, diversity, learning and teamwork are the hallmarks of Duke's guiding principles. Our accomplishments are dependent on the dedication and expertise of all who work to support Duke's mission.

Requisition Number




Duke Entity


Job Code


Job Family Level




Full Time / Part Time


Regular / Temporary




Minimum Qualifications

Duke University is an Affirmative Action/Equal Opportunity Employercommitted to providing employment opportunity without regard to anindividual's age, color, disability, gender, gender expression, genderidentity, genetic information, national origin, race, religion, sex,sexual orientation, or veteran status.

Duke aspires to create a community built on collaboration, innovation,creativity, and belonging. Our collective success depends on the robustexchange of ideas—an exchange that is best when the rich diversity ofour perspectives, backgrounds, and experiences flourishes. To achievethis exchange, it is essential that all members of the community feelsecure and welcome, that the contributions of all individuals arerespected, and that all voices are heard. All members of our communityhave a responsibility to uphold these values.

Essential Physical Job Functions:Certain jobs at Duke University and Duke University Health System mayinclude essential job functions that require specific physical and/ormental abilities. Additional information and provision for requests forreasonable accommodation will be provided by each hiring department.


Refer to Job Description

Auto req ID