Important Dates & Schedule

Readings and Weekly Assignments

For this course we will use Perusall and the weekly readings will be uploaded to the Perusall course specified in OLAT. The objective for using the Perusall application is to have students actively read, annotate, and discuss the readings online and before class. Perusall is an interactive application that allows you to share thoughts and questions with each other and to respond to other comments/annotations.

The reading of the papers will be graded based on: reading the paper, annotating where you have questions and comments, and the interactions with others. Therefor, make sure to read the papers within perusall and interactively annotated it.

 

In addition to reading the papers, there is also another deliverable, such as a short write up of a research question or a way to study one. The deliverables below provide more details on these.

Project

Throughout the semester, there will also be several times when we will discuss the research ideas & projects, and there will be project milestones, such as for the proposals, the write up, the peer reviews, and the presentations.

Tentative Schedule

Date Topic & Material Deliverable
20.9

Introduction

 

Intro slides (PDF, 4 MB)
27.9 (Empirical) Research in Software Engineering

3 Readings (on Perusall):

(1) What makes good research in software engineering?, Shaw, International Journal on Software Tools for Technology, 2002.

(2) A practical guide to controlled experiments of software engineering tools with human participants. Ko, LaToza, Burnett, ESE, 2013.

(3) Experimental models for validating technology, Zelkowitz et al., IEEE Computer, 1998.

 

Some questions that developers or data scientists ask about developers (as ideas for projects):

 

Optional (not required!):

Preliminary guidelines for empirical research in software engineering, Kitchenham et al., IEEE Transactions on Software Engineering, 2002.

READ and ANNOTATE all 3 papers on Perusall, the latest by midnight before class (interactively annotate each paper with comments and questions you have)

 

IDENTIFY ONE research question/problem and a way to address/study it in a course project and CREATE a max 2min presentation (max 2 slides in PDF) due on Sunday 26.9

(hand in by email, name the presentation LASTNAME_FIRSTNAME_RQ_01.pdf)

 

present to class 27.9 and take part in the discussion of papers

04.10

Developer Productivity

(DP1) The Work Life of Developers: Activities, Switches and Perceived ProductivityWhat predicts software developers’ productivity? Andre N. Meyer, Laura E. Barton, Gail C. Murphy, Thomas Zimmermann, and Thomas Fritz. IEEE TSE 2017.

(DP2) What predicts software developers' productivity? Emerson Murphy-Hill, Ciera Jaspan, Caitlin Sadowski, David Shepherd, Michael Phillips, Collin Winter, Andrea Knight, Edward Smith and Matthew Jorde. 

(DP3) How Gamification Affects Software Developers: Cautionary Evidence from a Quasi-Experiment on GitHub. Lukas Moldon, Markus Strohmaier, Johannes Wachs. ICSE 2021.

 

READ and ANNOTATE all 3 papers on Perusall, the latest by midnight before class (interactively annotate each paper with comments and questions you have)

 

Task: State in a maximum of 200 words: What are the major differences in the study methods used in the three papers, and what are the major strengths and/or weakness when comparing them?

due on Sunday 03.10

(hand in by email to Anastasia Ruvimova and me, name it LASTNAME_FIRSTNAME_TASK_02.pdf)

 

11.10
Developer Working Context & Flow

(WF1) "How Was Your Weekend?" Software Development Teams Working From Home During COVID-19. Courtney Miller, Paige Rodeghero, Margaret-Anne Storey, Denae Ford, Thomas Zimmermann. ICSE 2021.

(WF2) "Transport Me Away": Fostering Flow in Open Offices through Virtual Reality. Ruvimova, Kim, Fritz, Hancock, Shepherd. CHI 2021.

(WF3) Why do I keep interrupting myself?: Environment, Habit and Self-interruption. L. Dabbish, G. Mark, V. M. González. CHI 2011.

 

READ and ANNOTATE all 3 papers on Perusall, the latest by midnight before class (interactively annotate each paper with comments and questions you have)

 

Task: Based on the papers you read, you learned a lot about interruptions (self interruptions as well as external interruptions) and their impact on productivity and flow. Come up with ONE research question that you consider interesting in the area of interruption & flow and write down, which method you would use to examine it. For the method, choose one of the ones you saw in the papers for this week, or a combination thereof. (Maximum of 200 words, continuous text and full sentences)

due Sunday 10.10

(hand in by email to Anastasia Ruvimova and me, name it LASTNAME_FIRSTNAME_TASK_03.pdf)

18.10

Sensing Developers

(SE1) Using psycho-physiological measures to assess task difficulty in software
development. Fritz, Begel, Müller,Yigit-Elliott, Züger. ICSE'14. 

(SE2) A methodology for psycho-biological assessment of stress in software engineering. Jan-Peter Ostberg, Daniel Graziotin, Stefan Wagner, Birgit Derntl. PeerJ CS.

(SE3) Learning a Metric for Code Readability. Raymond P.L. Buse, Westley Weimer. TSE 2008. 

 
optional and possibly helpful:

READ and ANNOTATE all 3 papers on Perusall, the latest by midnight before class (interactively annotate each paper with comments and questions you have)

 

Task:  If you were asked to design a study (for the course project) that examines a measure/metric for coding difficulty: (a) which measure do you think would be interesting to study and why, and (b) how would you design the study? It does not necessarily have to be one of the ideas stated above, you could also come up with your own, but then just explain why you chose that one.  Keep in mind the time frame, i.e. you only have 3 months to design, run and analyze it. (Maximum of 200 words, continuous text and full sentences)

due on Sunday 17.10

(hand in by email to Anastasia Ruvimova and me, name it LASTNAME_FIRSTNAME_TASK_04.pdf)

 

20.10 Project Proposal due

Project Proposal

21/22.10: proposal discussions

25.10 Presenting proposals in class Proposal presentation due on 24.10
26.10 Final Proposal Due Project Proposal
01.11

Understanding and Supporting Code Comprehension
(CC1) Exploring Programmers' API Learning Processes: Collecting Web Resources as External Memory. Gao Gao, Finn Voichick, Michelle Ichinco, and Caitlin Kelleher. VL/HCC 2020.

(CC2) Supporting Code Comprehension via Annotations: Right Information at the Right Time and Place. Marjan Adeli, Nicholas Nelson, Souti Chattopadhyay, Hayden Coffey, Austin Henley, and Anita Sarma. VL/HCC 2020.

(CC3) Helping developers help themselves: automatic decomposition of code review changesets. Mike Barnett, Christian Bird, João Brunet, Shuvendu K. Lahiri. ICSE 2015.

 

READ and ANNOTATE all 3 papers on Perusall, the latest by midnight before class (interactively annotate each paper with comments and questions you have)

 

Task: 

Assume that you just developed a tool that allows developers to select web links and associate them with code elements in the IDE (e.g. a link to an API documentation on the web would be linked to a method call in the code that uses the API). Which research question(s) would be of interest for studying and how would you study it (method and study procedure)? (Maximum of 200 words, continuous text and full sentences)

due on Sunday 31.10

(hand in by email to Anastasia Ruvimova and me, name it LASTNAME_FIRSTNAME_TASK_05.pdf)

 

PER TEAM: a short update report on the project: 3 bullet points of what you did, 3 bullet points of what you'll do next week. (hand in by email)

 

08.11

Scrum + weekly meetings

short update report

15.11

Progress Presentation & Discussion

 

Study Sessions

22.11 Scrum + weekly meeting (+ controlled experiments)

Study Sessions

29.11

Scrum (+ controlled experiments)

 

Reading for writeup:

Preliminary guidelines for empirical research in software engineering, Kitchenham et al., IEEE Transactions on Software Engineering, 2002.

Study Sessions

06.12 Scrum + Weekly meeting

rough draft due 05.12 (not graded!)

12.12

8pm

Project report is due & you will receive two reports for reviewing

Project report (max 5 pages excluding references and optional Appendix)

16.12

12pm

noon

Peer reviews are due

& you will receive two peer-reviews for your project report for possible feedback before the presentation

peer-reviews of two other reports
20.12 Presenting project to class presentation