Peer Evaluations

The 100 team points that you earn for your project will be subject to a multiplier based on peer evaluations from each of your teammates. Twice during the semester you will be asked to evaluate the contributions and work of all other members of your team. You will not rate yourself in this assessment. This process is critical for accountability. We expect you to take it seriously and to act in good faith.

Procedure

1. Visit https://endor.cs.purdue.edu/~pucsevals/307/ and log in using the credentials provided via email.
2. Determine a rating for each team member using either integers or numbers with up to two decimal places.
3. Assign an average of ten points to your team members. E.g. you should assign a total of forty points in a five-member team. Fifty points for a six-member team.
4. If you believe everyone has worked equally well, you can assign a score of 10 to everyone. All 10s are indicative of a good overall team that has worked together well and productively. It is the ideal scenario.
5. You may give higher scores to team members that have done an exceptional job while providing lower scores for team members that performed poorly or did not contribute the expected amount. Remember, however, the average rating must still be ten.
6. Briefly describe the rationale for your ratings (1-2 sentences). This is not optional. Failure to provide rationales will result in a 0.05 deduction in your own multiplier.

Another way to look at the ten point average requirement is this: there is a "pool" of X points, where
X = 10 * (number_of_team_members - 1)
You must distribute the pool of points among your team members.

Considerations

We recommend that you consider at least the following factors when assigning your ratings:

• Contribution - did this person contribute productively to team discussions and work?
• Reliability - did this person get the work done on time and as promised?
• Respect - did this person encourage others to contribute their ideas? Did they listen to others? Did they rationally consider others' positions?
• Flexibility - was this person flexible and helpful when disagreements occurred?

For a typical teammate, the multiplier should be 1.0. For a high contributing teammate this will be greater than 1.0. For a low contributing student, it will be less than 1.0.

Calculation

Each of your two multipliers will be calculated as an average of the assigned ratings. Your final multiplier will be determined by the average of the two multipliers.

For instance, if you are on a team of five people, and your teammates assign ratings of: 11.2, 10, 9.4, and 9.8 for the first evaluation, your first multiplier will be (11.2 + 10 + 9.4 + 9.8) / 40 = 1.01. Presuming they assign ratings of 10.2, 11.1, 10.8, and 10.7 for your second evaluation, your second multiplier would be (10.2 + 11.1 + 10.8 + 10.7) / 40 = 1.07. The final multiplier would then be (1.01 + 1.07) / 2 = 1.04.

If, then, your team earns 92 of the 100 points for the project, you would receive 1.04 * 92 = 95.68 points.

Limitations

No team member will be permitted to have a multiplier larger than 1.05. The multiplier does not exist as a way to compensate for poor performance (e.g. on the exam). Ratings less than 8 can only be given if your project coordinator has been contacted multiple times during the sprint(s) and is aware of an ongoing problem. If you assign a rating below 8 without establishing contact with your project coordinator regarding the poor performance during the sprint(s) in question, the rating will be capped at 8, and a mandatory meeting with the instructor will be scheduled.

If a member of your team drops the class (either officially or unofficially), the next peer evaluation will be conducted by the remaining team members rating only the remaining team members. No ratings for or from the dropping student will be allowed.

Confidentiality

The peer evaluation system is a confidential process. We want you to be able to rate teammates and provide comments without concern of this information being provided to the subject of the evaluation. Individual ratings and comments will not be released. You will be provided your average multiplier. If you feel the multiplier is inappropriate, contact Dr. Turkstra. He will let you know why your multiplier is what it is without violating the confidentiality of any of your teammates.

Example Scenario

Suppose that the following six students are on a team. Suppose the first peer evaluation looks like the following...

        Rachel  Kurt  Will  Santana  Artie  Sue  Total  Multiplier
Rachel       -  11.7  12.5     10.0   11.8 12.2   58.2   1.164
Kurt       9.8     -  11.2     10.0   11.2  8.8   51.0   1.020
Will      10.0  10.1     -     10.0   11.1  9.1   50.3   1.006
Santana   10.1   9.3  10.0        -   10.8 10.0   50.2   1.004
Artie     11.5  10.6  10.0     10.0      -  9.9   52.0   1.040
Sue        8.6   8.3   6.3     10.0    5.1    -   38.3   0.766
            

Those providing ratings are listed along the top while those being rated are listed on the left. Note again that nobody rates herself/himself. Note, too, that each column totals 10 * (number_of_team_members - 1) = 50 points. Totals are computed and divided by 50 for each person, yielding the multiplier on the far right. You can also see that Santana gave each teammate a 10.

Each teammate completes a second peer evaluation at the end of the semester. Supposing it leads to the multipliers shown in the "2nd Mult" column below, we can then compute the final multipliers. Presuming the team earns 92 points on the project, we then multiply that by the final multiplier to determine each team member's project score.

      1st Mult  2nd Mult  Fnl Mult  Project
Rachel   1.164     1.152    1.0500   96.600
Kurt     1.020     1.038    1.0290   94.668
Will     1.006     0.953    0.9795   90.114
Santana  1.004     0.997    1.0005   92.046
Artie    1.040     1.104    1.0500   96.600
Sue      0.766     0.756    0.7610   70.012

We can see that Rachel and Artie, who worked hard on the project, receive high multipliers. Note, too, that both end up capped at 1.05 per above. Kurt and Santana, who worked well, receive scores slightly higher than 92. Will appears to have slacked off a bit in the second half of the semester and receives a score slightly lower than 92. Sue, who did very little and was disruptive to the team, receives a score substantially lower than 92.