CMPUT660F25 Project

1 Project

1.1 Warning: Project definition is not complete!

1.2 When: December 8 (Final Report)

• Proposal: November 7

• Proposal Presentation: November 7

• Final report: December 8

• Presentations: December 5

1.3 Who: Your team (2-3 people)

• 4 people requires permission from instructor.

• Collaboration with other teams is to follow the consultation style of collaboration illustrated on the CS course policy website. https://www.ualberta.ca/computing-science/resources/policy-information/department-course-policies.html

1.4 What: A project relevant to Mining Software Repositories that could be conceivably be submitted to a conference.

• 3 Styles of Project

  • Challenge extension (instructor permission required)

    • the challenge was rushed but there is an interesting idea there.

    • most challenges do not lend to extension

  • Original Research (instructor permission required)

    • do a new MSR study.

    • You should consult with the instructor if it has been done before.

      • only after googling it yourself.

  • Replication with data-set time slicing

    • Hypothesis: Many MSR studies do not stable conclusions. If you evaluate the conclusions on more data from the same project, or different time-ranges of data from the same project you will get different results.

    • Importance: If shown in a majority of cases it will demonstrate that the community overclaims conclusions.

    • The easiest method is to take a MSR Paper from the previous years and to replicate it using its original data or also replicate it on the updated versions of the software that the report initially used.

    • The trick is that you’re going to evaluate the conclusions of the replicated paper across time.

      • E.g., given a paper who uses data from 2009 to 2015 are their conclusions stable or the same for 2009-2010, 2009-2012, 2009-2013, 2009-2015?

    • Limitation: don’t replicate the same paper as another team.

1.5 Why:

• To challenge you to produce a MSR relevant paper.

• To gain experience writing papers for conferences.

• To motivate you to find interesting MSR methodologies and tools

1.6 Details

1.6.1 Write Up

• The write up must conform to current ICSE Formatting guidelines.

• Deliverable is a 10 page (2 column) current ICSE conforming paper with 2 page for references and an appendix where you can describe things in more detail if necessary.

  • 24 page EMSE style conforming paper will also be accepted.

1.6.2 More collaboration

• You may share parsed data with other teams (such as transforming a file to a database).

1.6.3 Submission

Final submission is to be emailed to me by Midnight XXX. Also upload it to the eclass. A presentation is expected on either the XXX.

1.6.4 Future

• We will probably submit this to a conference or journal.

• For replication studies we might combine them into a journal paper.

1.6.5 Rubric

1.6.5.1 General Rubric

• 30 Excellent
  • Hits most of the excellent column of the rubric. Thorough, meets requirements. Potentially Publishable Paper!
• 24 Good
  • Hits most of the good and excellent column of the rubric. Thorough, meets requirements. Missing some components. Paper needs some clean up before publication.
• 18 Satisfactory
  • Hits most of the satisfactory and good column of the rubric. Missing some portions, cursory. Not thorough. Probably would not consider publication.
• 15 Unsatisfactory
  • Missing many components but there was sufficient effort to warrant some marks. Inappropriate to send for peer review.
• 0 Failure
  • Not enough effort displayed to suggest merit.

1.6.5.2 Detailed Rubric

• See the rubric for assignment 2 and replace the 4 pages with 10 + 2.
• Assignment 2 rubric

1.6.6 Presentation

Present the results of the project.

1.6.6.1 Presentation Rubric

I like to use this presentation rubric.

• 6 Excellent
• 5 Good
• 4 Satisfactory
• 3 Unsatisfactory
• 0 Failure

1.6.7 TBD

This definition is not done yet!

Author: Abram