The grading process
The gradeomatic divides the grading process into three parts
- Create the exam, prepare the questions, add raw material for feedback, and upload students
- Evaluate student work and adjust the grade distribution
- Review student performance and release feedback to students
Of course, grading is a complex and organic process. So, you can generally jump around between the stages. There's no need to be tortured by a typo which you didn't notice until 5 exams in. It's easy to add additional items for assessment and feedback mid-grading when your students discover a novel way to misunderstand a concept.
While reviewing student work in the grading stage, you enter scores for each question and its subsidiary tasks. This automatically builds individualized feedback for each student. You may further customize each student's feedback by editing the compiled text on the fly.
Once you're finished grading, you use helpful statistics and visualizations of student performance make any necessary adjustments to the grade distribution
With two clicks you can email each student a unique link to their feedback. This link allows each student to securely view their grade and individual feedback along with detailed charts indicating how well they performed vis-a-vis their peers.
Once students have had enough time to review their feedback, you may remove access with another click. You may also print out the feedback and hand it back to students.
One last click exports your students' scores and grades to a spreadsheet. A quick copy and paste into your grade book, and you're ready for a celebratory beverage.
Create and edit exams
What exams are
An exam is the basic unit of organization. We called this an 'exam' but it could be a quiz, an assignment, a paper, or virtually any other activity for which you want to assess multiple students on the same criteria at approximately the same time.
An exam is identified by its name, and the year and term in which it is given. You associate a roster of students with each exam.
Every exam contains at least one question. Questions usually (implicitly or explicitly) contain several tasks which a student must complete in order to receive full credit for the question. These tasks are elements.
The following table summarizes the fields which comprise an exam. If a field is Required, you must enter a value in order to create the exam. If a field is Optional, you may choose to leave it blank. The Visible to Students column indicates whether the content of the field will be shown to your students.
|Field||Required||Visible to students|
Enter a short descriptive name in the
Exam Name box. This will be the name you see in various lists of your exams. It will also be displayed as part of the feedback your students see.
For our examples, we'll use a midterm for philosophy 101 about Descartes' famous skeptical argument from the beginning of the Meditations.
Thus let's name the exam: Phil101 midterm Descartes cogito
Use the dropdown menu to select the term in which you are giving the exam. The term is required.
Use the dropdown menu to select the year in which you are giving the exam. The year is required.
Once the name is filled in and the term and year are selected, click
to save the exam and move on to the next step.
Create a new exam from an existing exam
If you give the same or similar exams in multiple semesters, you can save virtually all of the setup time by cloning a past exam.
The cloning process duplicates all the questions, elements, and feedback from the parent exam. It does not duplicate the parent exam's roster, student scores, or any individualized feedback.
The newly created exam will be named 'Clone of ' followed by the original exam's name. Click
Edit to use the exam edit page to rename and update the term and year.
Create and edit questions
What questions are
Every exam has at least one question. Again, the system is designed to be extremely flexible. 'Questions' are really just whatever items a student's grade depends upon.
For example, if you were using the gradeomatic to grade long form essays, you could have 'questions' like Organization or Grammar in addition to the prompt and set the point values accordingly. See alternative uses of questions for an example.
The following table summarizes the fields that comprise a question. If a field is Required, you must enter a value in order to create the question. If a field is Optional, you may choose to leave it blank. The Visible to Students column indicates whether the content of the field will be shown to your students.
|Field||Required||Visible to students|
Creating and editing questions
In setting up the exam, you create a question by giving it a brief
Question Name (to use as a reminder while grading) and, optionally, the full
Question Text that was presented to the student.
Students were asked to answer the question:Explain the skeptical method Descartes uses in the Meditations
Question Name: Explain skeptical method
Question Text: Explain the skeptical method Descartes uses in the Meditations.
Maximum points possible
Enter the maximum number of points a student can achieve on a question in the box marked
If you will not assign grades with the gradeomatic (and don't need question data for assessment), set the max score to 0.
If you want all questions to count equally toward the overall exam grade, assign each question the same number of points. If all questions count equally, we recommend assigning each question 100 points.
If you want some questions to count more toward the final grade than others, adjust the maximum scores accordingly.
For example, if Question 1 and Question 2 are each worth 25% of the grade, and Question 3 is worth 50%, you could set the maximum scores for Question 1 and Question 2 to 100, and the maximum score for Question 3 to 200 points (alternatively: Q1 = 25 points, Q2 = 25 points, and Q3 = 50 points).
To change the order of questions click and hold the Move button for the question whose position you want to change.
While holding the button down, drag the question to its new position.
Once in position, release the button. The other questions will update their numbers accordingly.
Saving question edits
No changes to the questions are saved until you click
To remove a question, click its Delete button.
When you click the button, a confirmation dialog will pop up.
Once you confirm by clicking
Delete in the pop up, the question will disappear from
the screen. However, the question will not actually be deleted from the database until you click
Please be very careful with deleting questions. If a question is deleted, all student scores for that question are permanently removed. Any elements associated with the question are also deleted, along with any student scores for those elements.
There is no way to undo the deletion or to recover the lost data
Miscellaneous: alternative uses of questions
We mentioned earlier that while an exam must have at least one question, there is no need for the questions to be 'questions' in the traditional sense.
For example, suppose you have an exam with four questions each worth 20% and want the student's grammar on all questions to count for 20%. Simply add an extra question called 'Grammar', and set the number of points that portion is worth.
Miscellaneous: Feedback only
While each question must have a maximum score, the maximum score can be 0. This might be helpful if you want to use the gradeomatic to create feedback for students but don't want to assign a grade.
For example, suppose you are going to grade a rough draft of an essay credit/no credit, and then give actual grades for the final draft. You would create the questions and elements as usual.
For the rough draft you set the points for each question to 0. After grading, you would generate the feedback as usual. Students would see all the feedback but without a grade.
Then, when the final draft is turned in, simply clone the 'exam' you used for the rough draft and change the points for each question to their actual values.
Create and edit elements
What elements are and what they do
Elements have multiple roles and are very flexible. On their most basic use, elements have three jobs:
- Elements comprise questions
- Elements form the basis of feedback
- Elements let you compare student performance across superficially very different exams
The following table summarizes the fields that comprise an element. If a field is Required, you must enter a value in order to create the element. If a field is Optional, you may choose to leave it blank. The Visible to Students column indicates whether the content of the field will be shown to your students.
|Field||Required||Visible to students|
|Element response: Missing||Optional||Yes|
|Element response: Poor||Optional||Yes|
|Element response: Fair||Optional||Yes|
|Element response: Excellent||Optional||Yes|
Elements comprise questions. On more complex questions, where you are (explicitly or implicitly) asking the student to do multiple things or where explaining a concept requires several different components, each task or component should be represented by an element.
Multi-part questions or short essays spanning several paragraphs might have several elements. While a short-form question might only have one. A question can have zero elements if you only want to use the gradeomatic for reporting scores.
If you are familiar with using rubrics to grade an exam, you are already familiar with how to divide a question into its elements.
For those unaccustomed to rubrics, consider this, likely familiar, scenario:
Scene: A good student comes to your office, distraught after doing poorly on one question on the exam.
Student What did I miss? What should I have done?
You Well, let's see. You needed to do x, y, and z. You did a good job on x. But you totally forgot to bring up y. And there were several mistakes in your explanation of z.
Whatever you filled in for x, y, and z in that conversation are probably the elements of the question
Creating elements and comments
To create an element, provide a short name for the element which which will be meaningful to you when you see it during grading and when displayed in charts.
An explanation of methodology isn't going to make much sense without knowing what it's for. So the first task is to explain Descartes' goal.
ElementName: Explain Descartes goal
You then also provide some text that will form the basis for all feedback that you give on the element.
Element Response, enter a basic description of what the student should do to
answer the element.
Element Response: In order to say why Descartes has adopted the skeptical method of the Meditations, you need to tell the reader what Descartes is hoping to achieve. Namely, he is trying to discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. That is, the idea is to find some beliefs which he can't be wrong about. Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
If you want to give every student the same feedback regardless of how they do, you may stop here.
Pressing the Customize Responses button allows you to further customize the text to reflect performance.
Here you modify that basic comment to tailor it based on the student's performance. By default, gradeomatic allows for four responses varieties: "missing", "poor", "fair" and "excellent". These responses will be what the student sees once you have graded the exam.
Let's use the following as the stock feedback for this element.
Example: Stock response
In order to say why Descartes has adopted the skeptical method of the Meditations, you need to tell the reader what Descartes is hoping to achieve. Namely, he is trying to discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. That is, the idea is to find some beliefs which he can't be wrong about. Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
This can be a bit tricky. So here's a detailed example of how we might alter the stock feedback for different levels of performance. In each variation, the new or altered text is underlined.
In order to say why Descartes has adopted the skeptical method of the Meditations, you need to tell the reader what Descartes is hoping to achieve. is trying to discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. That is, the idea is to find some beliefs which he can't be wrong about. Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
In order to say why Descartes has adopted the skeptical method of the Meditations, you need to tell the reader what Descartes is hoping to achieve. discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. That is, the idea is to find some beliefs which he can't be wrong about. Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
in order to say why Descartes has adopted the skeptical method of the Meditations, what Descartes is hoping to achieve. is trying to discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. That is, the idea is to find some beliefs which he can't be wrong about. Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
in order to say why Descartes has adopted the skeptical method of the Meditations, the reader needed to be told what Descartes is hoping to achieve. is trying to discover what kinds of beliefs can be the completely certain foundations upon which the rest of our knowledge can be built. is to find some beliefs which Descartes can't be wrong about.Then he can work backwards to explaining why and when, for example, scientific beliefs count as certain knowledge.
To add additional elements, click Add Element
Example: Stock response
You need to explain the role doubt plays in Descartes method. He is using a principle like "If I can find grounds for doubting that a kind of belief is true, then no beliefs of that sort count as knowledge". So, for example, if we're talking about beliefs based on seeing things in the distance, I might believe that I see a plane. But then someone points out that birds are often confused with faraway planes. Now I can't say that I know that object in the distance is a plane until I can be sure that it is not a bird.
None of your edits are saved until you click
Create and edit student rosters
Rosters: Associating students with an exam
Each exam has a student roster with all the students who will take the exam. On the "edit roster" page you can import students from a .csv file and edit student information.
The only required information is the first and last name of the student. You may also include the student's ID number (if, for example, you wish to grade the exams blindly) and their email address if you want to have a link for accessing feedback emailed directly to the students.
We strongly recommend only including the fields that you will need. If you won't be using student id numbers, do not include them the uploaded file. If you won't be emailing students their feedback, do not include student email addresses
To import students, click
Edit Student Roster on the exam setup page. You will also be
automatically taken to the import students page after you save elements for the last question.
Preparing the roster for importing
CSV stands for "Comma Separated Values". It is a very simple text format for storing data. It is basically what would be left over if you removed all the formulas, all the formatting, and all the other things which make an Excel spreadsheet useful.
To make a .csv file, it's easiest to start with an Excel spreadsheet. Once you've added all your data (if you don't want to lose the formatting, first save it as you would any other spreadsheet).
Preparing the spreadsheet
Make sure that all of the information is correct. Remove all data except for the student's last name, student's first name, (optional) student id, and (optional) email address.
Make sure the columns are in the proper order, from left to right:
- Last name
- First name
- Student id (optional)
- Email address (optional)
The import process will be easiest if the first row of the sheet has exactly the following column names:
- last name
- first name
- student id
Saving as .csv
To create a .csv file from Excel in Windows or Mac, go to "File", then "Save As", and select "CSV (Comma delimited) (*.csv)".
A message may pop up, to warn you that saving as a .csv file will lose all the file's formatting. If this is a problem, make sure you've saved the file as a normal spreadsheet (e.g., .xls or .xlsx) before proceeding.
Once you're ready, click Confirm.
Importing students from file
Once your csv file is prepared, click Import Roster
A dialog will pop up allowing you to select the file to upload. Make sure you choose the file which ends in '.csv'
Check over the imported records to make sure that everything is in the correct location
You can sort the list by clicking on the column title you want to sort by. For example, to
sort by first name, click the
First Name at the top of the table.
If you need to edit or correct student information, click on the item to be edited and type your changes.
If you need to add a student by hand, click Add Student
Fill in the student information.
If you need to delete a student, click the 'X' in their row.
A warning message will ask you to confirm that you really want to delete the student.
Deleting a student will destroy all of their scores, feedback and grades.
There is no way to restore the student's data once they are deleted.
The student has been set for deletion. The actual deletion will take place once the roster is saved
Save & Finish to save any changes to the roster. Note that any changes you make
will not be saved until you press
Save & Finish
Grade student work
Choose the exam to grade
When you are done setting up the exam and ready to grade, click
Grade in the
navigation bar at the top of the page.
That will take you to the grade selection page
Click Grade to start grading the exam
Student selection area
The student selection area allows you to choose which exam to grade.
If you prefer to grade exams without knowing the students' names, click the pencil
icon. The student names will be replaced by 'Name Hidden'. It may help to click
ID to sort the list of
students by student id.
You may also search for a student by clicking on the
ID display boxes at the top and start typing. An autocomplete box will appear and allow you to select the appropriate student
The dashboard displays statistics to keep you motivated and grading quickly.
The button marked
Start manually starts and stops the timer. When you take a
break, it's a good idea to click pause. Otherwise the timer will keep running for the last exam you graded. This will throw off the average time and reduce the accuracy of the estimated remaining time.
The fields displayed in the dashboard are
- Time This Exam
- How long you have spent grading the current exam.
- Average Time
- The average amount of time spent grading for the exams you have graded so far.
- Total Time
- The amount of time spent grading all exams so far.
- Time Remaining
- The estimated amount of time until you are done grading. It is calculated by multiplying the average exam time with the number of ungraded students (with a few basic corrections for outliers).
When you click on a student's row, the gradeomatic loads the student's record and highlights their row in blue to show that it is currently being graded. This will reveal the questions for the exam and a slider for each element.
Enter element scores and customize feedback
After reading the student's response, move the slider to a value that corresponds to the student's performance (values in between the circles are fine). This will record a numeric value that you can use for assessment. The slider value won't affect the question grade, but it will affect the feedback given to the student.
If you wish to tailor the student's feedback individually, modify the text box next to that element. The text you enter will be shown to that student alone.
As you move a slider, you will notice that the text in the box to the right of the slider will update to contain the text you entered for that level of competence. If you did not customize the text for each degree of performance when you created the element, the text will not change as you move the slider. However, even if you have no feedback set, the score indicated by the slider will still be recorded so you can use it in improving your course.
To continue to the next question, click the appropriate tab at the top of the input area.
If the student didn't need to answer a question, simply leave the score area blank.
You may notice that once a grade is entered for at least one question, the student's exam is considered graded for timing and release purposes. For example, the values used in calculating the time remaining will update. This was necessary to accommodate exams which give students multiple options of which questions to answer.
Enter question score
There are two ways to enter a score for a question.
You may type a score in the
Score box. Make sure it is a number and less than the maximum
Alternatively, you may select a letter grade from the dropdown menu. The gradeomatic will enter a percentage of the maximum possible score in the box for you.
You might be looking for a Save button or wondering when the scores are saved for an exam.
Unlike the setup pages where no changes were saved until you moved to the next step, during grading, every time you move a slider, click a button, or type in a box and then click somewhere else on the page, the gradeomatic sends all the data for the current exam to the server.
Since the data gets sent to the server behind the scenes, you may occasionally see a message informing you that there was a problem. This is usually the result of a transient interruption in your internet connection. Simply repeat your last action and it should save properly.
Adjust grade distribution
If you want to use the gradeomatic to assign grades, click the Assign button on the grade exam select page.
Score adjustment area
The gradeomatic calculates the maximum possible score on the exam from the maximum scores of each question. It then provides suggestions for the minimum total score for each letter grade.
To adjust the grade distribution, alter the values in the boxes. The accompanying charts will update to show the new distributions.
If you don't want to use a letter, empty the corresponding box and it will not be a possible grade.
Once you've made the necessary adjustments, click Save Assignments.
The Assign Grades page includes two charts.
The upper chart on the right-hand side of the page is a histogram of the number of students currently receiving each grade.
When assigning grades, it is often helpful to look for natural gaps between student scores. The bar chart on the lower right-hand side of the page shows the total exam score for each student along with the grade that they would receive under the present distribution.
To help you impartially assign grades, each number on the x axis identifies a student ordered by total score from low to high. The number has no relation to the student's name, id, order graded, or any other identifying information
Release and hide feedback
Clicking "Release Exam" officially releases the exam. This involves emailing unique links to all students who have been graded and who have valid email addresses. This link will all them to view their grade and feedback.
The lock is the reverse. Clicking Lock cuts off all access to all students for that exam. Once an exam is locked, it must be released again. If a locked exam is re-released, students will receive a new email with a new link.
Once an exam has been locked, any links that have been sent to students will not work. This can be confusing, so it is best to use the commands for resending links to individual students if the need arises (e.g., if a student loses their email).
Once the exam is released, the release button changes color to green. This indicates that students have access to their feedback. The lock button changes to blue. This indicates that it may be clicked to remove student access to feedback
Export scores and grades
Through the grading process you've assigned grades to each student and collected a bunch of important data for improving your teaching. Press the Export button to export the data so that you can use it in a spreadsheet
The analytics page is currently under development and thus rather incomplete. Feel free to take a look and use the tools as they become available.
Eventually, the analytics page will contain three kinds of tools.
- Tools for visualizing and analyzing student performance on the present exam at different levels of detail
- Tools for quality control in grading. These will help you identify exams on which you might have erred in grading so that you can return exams with confidence that the grades were fair.
- Tools for comparing student performance across different terms and exams. These will help you construct better exam questions and improve your teaching.
We will be grateful for suggestions about other useful tools and visualizations
Currently available tools
Total score box plots: The first chart is a box plot detailing student score variation for each question.
The vertical lines (the 'whiskers') show the lowest and highest grades. The edge of bottom of the box is the lowest quartile. The edge of the top of the box the third quartile.
The circles mark the mean and median.
Individual student controls
The students button takes you to a page with individual student controls. On this page, you may review an individual student's feedback or send them a new email with the link to their feedback.
Clicking the Email button will send a new notification email with a link to their feedback to the selected student. This is often used when a student loses the initial email. The feedback link will be the same link they were sent in the original email.
This can also be helpful if you discover a grading error and want to update a few students' scores and feedback after the exam has been released. They will be sent the same link as in the original email (whereas, if you had locked and then re-released the whole exam, every student would need to receive a new link).
Review individual feedback
By clicking the Review button, you can preview the feedback that the student will receive.
This is often helpful in office hours when a student wants to talk about their exam, but does not have a printed copy of their feedback.