Ground Truth Dataset for Personalized Item Ranking

We at the IVDA lab strike to create the first ground truth data set for item rankings. In a user study, we have collected data that contains pairwise comparisons between two items where decisions made by users are strongly guided by a task description presented to users.

We now want to use this raw dataset to create a ground truth dataset for human-centered item rankings that can be used to assess the results of other studies conducted at our lab but also in other research groups. Our goal is to find algorithms that can create item rankings based on the data that we have collected so far. The found algorithms should then be incorporated into a visual analytics tool that allows users to create a ground truth data set based on one of the found algorithms. The tool should also offer additional guidelines for users for the selection of the data as well as the inspection of the data itself.

In short, the project consists of the following:

• The analysis and evaluation of a raw dataset collected through a user study conducted at the IVDA lab
• The collection of requirements for ranking algorithms that can work with our data
• A search for suitable algorithms that match the defined requirements
• The design, implementation, and evaluation of a VA tool that supports users in creating a ground truth dataset with selected algorithms
• A (small) written report that elaborates on the findings of the literature review, describes the implemented tool and the chosen approaches and outlines possible areas for future work

The project will be supervised by Jenny Schmid and Prof. Jürgen Bernard. Ideally, the project should start in January or February 2023 and the duration of the project will be approximately 6 months. A group of 3-5 students will be working on the project.

The project language is English, therefore, you are required to have good English knowledge (written and oral).

The programming languages of the project are not yet defined and are up for discussion by the project team (preferably Python or Java for the back end and JavaScript for the front end). The code will be versioned through GitLab and the tool will be hosted on the IFI server. Programming skills are required for this project.

Regular meetings will be held on-site and online.

We look for students that are interested in both visual analytics as well as algorithms.

You should have the following skills for this project:

• Profound programming skills (preferably with Python, Java, and/or JavaScript), ideally with a full-stack background
• Basic knowledge of databases (e.g. how to write a database statement to retrieve data)
• Experience in working with version control systems (e.g. Git, we will be using GitLab)
• Interest in research

In addition, it would be nice if you have experience in one of the following areas (but it is not a must):

• UI / UX design
• Human-Computer Interaction
• REST or any other framework that supports the communication between the back end and front end
• CSS and HTML
• Front end frameworks such as ReactJS, Angular, Material Design, Bootstrap, etc.
• Back end frameworks such as Django, Flask, Spring, etc.

Interested students should send their complete application to Prof. Jürgen Bernard.

Applications should include:

• A short, informal letter of motivation including your expectation for the project, the skills you want to learn, and your experience with data visualizations and programming (if any)
• Your CV
• Your transcript of records from your Master's and Bachelor's Degree

If you have any questions about the project, please feel free to contact  Prof. Jürgen Bernard or Jenny Schmid.