About the positions

Faculty of Information Technology and Electrical Engineering

Departments and Labs:

• Engineering Cybernetics (ITK)
• Computer Science (IDI)
• Norwegian Open AI Lab (NAIL)

Explainable AI and the EXAIGON project

The recent rapid advances of Artificial Intelligence (AI) hold promise for multiple benefits to society in the near future. AI systems are becoming ubiquitous and disruptive to industries such as healthcare, transportation, manufacturing, robotics, retail, banking, and energy. According to a recent European study, AI could contribute up to EUR 13.3 trillion to the global economy by 2030; EUR 5.6 trillion from increased productivity and EUR 7.73 trillion from opportunities related to consumer experience. However, in order to make AI systems deployable in social environments, industry and business-critical applications, several challenges related to their trustworthiness must be addressed first.

Most of the recent AI breakthroughs can be attributed to the subfield of Deep Learning (DL), based on Deep Neural Networks (DNNs), which has been fueled by the availability of high computing power and large datasets. Deep learning has received tremendous attention due to its state-of-the-art, or even superhuman, performance in tasks where humans were considered far superior to machines, including computer vision, natural language processing, and so on. Since 2013, Deep Mind has combined the power of DL with Reinforcement Learning (RL) to develop algorithms capable of learning how to play Atari games from pixels, beating human champions at the game of Go, surpassing all previous approaches in chess, and learning how to accomplish complex robotic tasks. Similarly, DL technology has been used in combination with Bayesian Networks (BNs), resulting in Deep Bayesian Networks (DBNs), a framework that dramatically increases the usefulness of probabilistic machine learning. Despite their impressive performance, DL models have drawbacks, with some of the most important being lack of transparency and interpretability, lack of robustness, and inability to generalize to situations beyond their past experiences. These are difficult to tackle due to the black-box nature of DNNs, which often end up having millions of parameters, hence making the reasoning behind their predictions incomprehensible even to human experts. In addition, there is a need to better understand societal expectations and what elements are needed to ensure societal acceptance of these technologies.

Explainable AI (XAI) aims at remedying these problems by developing methods for understanding how black-box models make their predictions and what are their limitations. The call for such solutions comes from the research community, the industry and high-level policy makers, who are concerned about the impact of deploying AI systems to the real world in terms of efficiency, safety, and respect for human rights. In order for XAI to be useful in business-critical environments and applications, it should not be limited to algorithm design because the experts who understand decision-making models the best are not in the right position to judge the usefulness and structure of explanations. It is necessary to enhance XAI research by incorporating models of how people understand explanations, and when explanations are sufficient for trusting something or someone. Such models have been researched for many years by philosophers, social and cognitive psychologists, and cognitive scientists. It becomes evident that significant interdisciplinary contributions are needed for AI systems to be considered trustworthy enough for deployment in social environments and business-critical applications.

The EXAIGON (Explainable AI systems for gradual industry adoption) project (2020-2024) will deliver research and competence building on XAI, including algorithm design and human-machine co-behaviour, to meet the society’s and industry’s standards for deployment of trustworthy AI systems in social environments and business-critical applications. Extracting explanations from black-box models will enable model verification, model improvement, learning from the model, and compliance to legislation.

EXAIGON aims at creating an XAI ecosystem around the Norwegian Open AI-Lab, including researchers with diverse background and strong links to the industry. The project is supported by 7 key industry players in Norway who will provide the researchers with use cases, including data, models and expert knowledge. All involved researchers will work closely with each other, the industry partners, and researchers already working on relevant topics at NTNU, hence maximizing the project’s impact and relevance to the real world.

Duties of the positions

EXAIGON is hiring 4 PhD candidates, one in each of the following areas:

PhD1: XAI methods for supervised learning models. Supervised learning algorithms in particular the Deep Neural Networks in various forms have been behind the unprecedented success of artificial intelligence in recent times. However, their interpretability diminishes very quickly with increasing complexity of the network architecture. To address the interpretability part we envision three (but not limited to) basic approaches (a) develop ways to simplify the networks either by enriching the input space or by expressing them in some alternate interpretable form like a collection of piecewise affine representations and then develop theories to prove their stability (b) develop methods to inject physics / domain knowledge and thereby, guide the learning process of the networks (c) to develop analysis tools to get better insight into the functional form of the mapping from the input to the output space. Desirable skills: A combination of Linear Algebra, Numerical Methods, Constrain Optimization, Partial / ordinary differential equations and Machine Learning. All the programming tasks will be preferably realized in Python using Pytorch or Tensorflow libraries.

PhD2: XAI methods for deep reinforcement learning models. Instead of labelled data, deep reinforcement learning (DRL) employs the user-defined reward function to guide the training process and is mainly used in problems where optimal actions need to be computed for sequential decision making. DRL policies are in principle DNNs trained in a different manner. This mean many XAI methods pertaining to SL models are relevant here too. However, EXAIGON will give emphasis to the reward function, which rewards/penalizes desired/undesired actions, respectively. For controlled cyber-physical systems (such as autonomous ships, robotic manipulators or drones), the candidate will investigate how to detect physical laws in DNNs. We will look into how explanations can help design more efficient rewards functions. In addition, the position will investigate the rather challenging case of delayed rewards.

For this particular position, the selection committee will prioritize candidates who have a reasonable combination of the following skills: Theoretical and hands-on background within machine learning, and ideally deep reinforcement learning. Background within dynamics and control of cyber-physical systems is a strong plus, especially in case the candidate has been involved in field experiments. Regarding computer skills, Python, C++ and Robot Operating System (ROS) are the most relevant frameworks.

PhD3: XAI methods for deep Bayesian networks. Probabilistic AI investigates the intersection between probabilistic graphical models and deep learning. Such models are often called deep Bayesian networks (DBNs). Injecting probabilistic thinking into DNNs has several benefits, including robustness against overfitting and resilience against adversarial attacks. Furthermore, DBNs can quantify uncertainty in their predictions, also in ways that include model and parameter uncertainty. DBNs will to some extent cater to a causal interpretation, which provides an efficient and robust language for explaining inferences. In EXAIGON we aim to utilize these features in order to generate understandable and trustworthy explanations for model-predictions from probabilistic AI models. Furthermore, we will investigate how techniques for explanations and sensitivity analysis used for traditional Bayesian networks carry over to Probabilistic AI models. We will first consider general strategies for generating explanations from DBNs, and later apply the most promising techniques in industrial settings.

For this position, the selection committee will prioritize candidates with a strong knowledge of mathematical statistics, combined with theoretical and hands-on experience with modern machine learning. A general knowledge of the trends in Probabilistic AI is a strong positive, but not required. It is expected that the successful candidate is a skilled Python programmer, and experience with deep learning frameworks like Tensorflow and PyTorch is a plus.

PhD4: Human-machine co-behaviour. For all their merits and compelling results, the uptake of AI methods, XAI ones included, till date into actual decision-making in organisations have been slow. The theme of PhD4 is to analyse social, organisational and institutional conditions that enable and hamper uptake of AI methods in general and XAI ones in particular. The PhD4 position accordingly studies XAI through the discipline of Information System or applied AI. The focal theme of PhD4 is to analyse how different XAI methods are perceived, understood and used by different users in the partner companies of EXAIGON to identify enabling conditions and mechanisms for their uptake into actual decision-making. 

For this position, the selection committee will prioritize candidates with a strong background in Information Systems theory and concepts.  Candidates need strong skills in qualitative or interpretative research methods as empirical data will comprise observations, interviews and trace data of users. As the everyday language of the EXAIGON partner companies is Norwegian, PhD4 needs to master one of the Scandinavian languages not to miss out on crucial empirical data for the PhD viz. observations of users.

The candidates will have the opportunity to combine experimental and theoretical work in the aforementioned areas, with the aim of eventually demonstrating their successful fusion in realistic conditions. All PhD candidates will also have the option for a research stay abroad during their studies. Potential destinations are the USA and Australia, where EXAIGON has established scientific collaborators.

Required selection criteria

We seek four ambitious and highly-motivated individuals with a Master’s degree in engineering cybernetics, computer science, control engineering, applied mathematics, or a related discipline. Applicants are required to justify their candidateship by explicitly explaining their personal motivation and academic aptitude for pursuing a doctoral degree within this research field.

Applicants that expect to complete their Master degree studies by summer 2020 can apply.

Academic results, publications, relevant specialization, work or research experience, personal qualifications, and motivation will be considered when evaluating the applicants.

It is a prerequisite that the PhD scholar applies for and is granted admission to the NTNU PhD studies as soon as possible after employment. NTNUs PhD-rules require a Master degree or equivalent with at least 5 years of studies and an average ECTS grade of A or B within a scale of A-E for passing grades (A best). Applicants must be qualified for admission as PhD students at NTNU. See http://www.ime.ntnu.no/forskning/phd for information about PhD studies at NTNU.

Excellent English skills, written and spoken, are required. Applicants from non-European countries where English is not the official language must present an official language test report. The following tests can be used as such documentation: TOEFL, IELTS or Cambridge Certificate in Advanced English (CAE) or Cambridge Certificate of Proficiency in English (CPE). Minimum scores are:

• TOEFL: 600 (paper-based test), 92 (Internet-based test)
• IELTS: 6.5, with no section lower than 5.5 (only Academic IELTS test accepted)
• CAE/CPE: grade B or A.

Appointments are made in accordance with the regulations in force regarding terms of employment for PhD candidates issued by the Ministry of Education and Research, with relevant parts of the additional guidelines for appointment as a PhD candidate at NTNU.

Applicants must undertake to participate in an organized PhD programme of study during their period of employment. The person who is appointed must comply with the conditions that apply at any time to employees in the public sector. In addition, a contract will be signed regarding the period of employment.

The successful candidates will be appointed for a period of 3 years, with possible extension to a fourth year if the candidates and departments agree on teaching related duties.

The appointment is to be made in accordance with the regulations in force concerning State Employees and Civil Servants and national guidelines for appointment as PhD, post doctor and research assistant.

Personal characteristics

In the evaluation of which candidate is best qualified, emphasis will be placed on education, experience and personal suitability.

We offer

• exciting and stimulating tasks in a strong international academic environment
• an open and inclusive work environment with dedicated colleagues
• favourable terms in the Norwegian Public Service Pension Fund
• employee benefits

Salary and conditions

PhD candidates are remunerated in code 1017, and are normally remunerated at gross from NOK 479 600 per annum before tax, depending on qualifications and seniority. From the salary, 2% is deducted as a contribution to the Norwegian Public Service Pension Fund.

The Faculty of Information Technology and Electrical Engineering wants to attract outstanding and creative candidates who can contribute to our ongoing research activities. We believe that diversity is important to achieve a good, inclusive working environment. We encourage all qualified candidates to apply, regardless of the gender, disability or cultural background.

Under Section 25 of the Freedom of Information Act, information about the applicant may be made public even if the applicant has requested not to have his or her name entered on the list of applicants.

The engagement is to be made in accordance with the regulations in force concerning State Employees and Civil Servants, and the acts relating to Control of the Export of Strategic Goods, Services and Technology. Candidates who by assessment of the application and attachment are seen to conflict with the criteria in the latter law will be prohibited from recruitment to NTNU. After the appointment you must assume that there may be changes in the area of work.

About the application

Each application must contain:

• An explicit indication of which of the four positions, PhD1 – PhD4, the applicant applies for. If an applicant’s background and experience justify it, they may apply for two positions. In that case, the applicants’ research statement should reflect information pertaining both positions.
• CV including information relevant for the qualifications and contact information for at least 2 reference persons
• Certified copies of academic diplomas and transcripts
• Applicants from universities outside of Norway are requested to send a diploma supplement (or a similar document) which describes in detail the study and grading system, and the rights for further studies associated with the obtained degree
• A research statement (max. 3 pages) including:
  • A short presentation of the motivation for a PhD study
  • Why the applicant is suited for the position
  • The applicant’s view of research challenges for the PhD position, as well as his/her theoretical and methodological approach to the challenges.

Publications and any other work that the applicant wishes to be considered must also be enclosed. Joint works will be considered if a short summary outlining the applicant's contributions is attached.

General information

The city of Trondheim is a modern European city with a rich cultural scene. Trondheim is the innovation capital of Norway with a population of 200,000. The Norwegian welfare state, including healthcare, schools, kindergartens and overall equality, is probably the best of its kind in the world. Professional subsidized day-care for children is easily available. Furthermore, Trondheim offers great opportunities for education (including international schools) and possibilities to enjoy nature, culture and family life and has low crime rates and clean air quality.

NTNU is committed to following evaluation criteria for research quality according to The San Francisco Declaration on Research Assessment - DORA.

Further details about the positions can be obtained from:

PhD1: Professor Adil Rasheed, e-mail: adil.rasheed@ntnu.no

PhD2: Associate Professor Anastasios Lekkas, e-mail: anastasios.lekkas@ntnu.no

PhD3: Professor Helge Langseth, e-mail: helge.langseth@ntnu.no

PhD4: Professor Eric Monteiro, e-mail: eric.monteiro@ntnu.no

Please submit your application electronically via jobbnorge.no with your CV, diplomas and certificates. Applications submitted elsewhere will not be considered. Incomplete applications will not be considered.

Diploma Supplement is required to attach for European Master Diplomas outside Norway. Chinese applicants are required to provide confirmation of Master Diploma from China Credentials Verification (CHSI).

Mark the application with the reference number: 2020/9110.

Application deadline: April 27, 2020.

NTNU - knowledge for a better world

The Norwegian University of Science and Technology (NTNU) creates knowledge for a better world and solutions that can change everyday life.

Department of Engineering Cybernetics (ITK)

Engineering cybernetics is the study of automatic control and monitoring of dynamic systems. We develop the technologies of tomorrow through close cooperation with industry and academia, both in Norway and internationally. The Department contributes to the digitalization, automation and robotization of society. The Department of Engineering Cybernetics is one of seven departments in the Faculty of Information Technology and Electrical Engineering.

Deadline 27th April 2020
Employer NTNU - Norwegian University of Science and Technology
Municipality Trondheim
Scope Fulltime
Duration Project
Place of service Trondheim