Crowdmark

About Crowdmark

Crowdmark is an online platform that helps instructors manage and grade both digital and paper assignments and tests quickly and efficiently. Crowdmark works especially well with large classes or classes with distributed grading (marking assistants or multiple instructors).

Assessing and Grading

The benefits of using Crowdmark include:

Grades are automatically totaled and combined with analytics
Comment library supports Markdown and LaTeX
Marking quality and consistency is easier to monitor.
Markers within the grading team can each be assigned specific questions and this grading can all occur simultaneously without the need to exchange paper based assessments.
Multiple choice questions are auto graded.
Up to 200 bubble-sheet questions

Key characteristics of Crowdmark

Assessments and exams that are created in Crowdmark can be delivered in person or remotely online . In person exams are scanned and uploaded into the platform. Remote and online assessments are uploaded by the students. Both are then graded in a similar way.

To watch a demo on Crowdmark, click HERE.

Exam Scanning

Using Crowdmark for Multiple Choice (bubble sheet) scanning

Crowdmark can automatically grade up to 200 multiple choice questions in either a QR coded or Non-QR coded Administered (in-person) assessment. If you enable multiple choice on your assessment in Crowdmark, Crowdmark will add up to two 100-question bubble sheets with responses A–E to the end of each student’s booklet. Multiple answers to a question is supported.

Benefits of using Crowdmark’s multiple choice bubble sheet workflow include:

Fully auto-graded
Partial credit for multiple correct responses
One-click editing of answer keys for automatic re-scoring across the assessment before, during or after grading
One-click changes to student responses or errors
Compatible with standard printing and scanning
Compatible with standard 8.5×11 or A4 paper, and printable in large-format for accessibility
Option for AI-assistance matching students to booklets
Full-spectrum, exportable analytics
One-click sync to LMS grade book, or download results to CSV for further analysis

Walkthrough of using Crowdmark for Multiple Choice (bubble sheet) scanning

Create your assessment template. Crowdmark requires at least one page (a cover page) in the template. You can include the multiple choice question prompts in your template or leave them out and print them outside of Crowdmark. Crowdmark doesn’t currently offer a way to “link” the question prompts to the bubble sheet but we recommend including the questions in the template so students can reference them after grades are returned.
Create an Administered assessment. During the setup process, you will have the option to enable multiple choice, by choosing the number of bubble sheets you’d like to include. You can use bubble sheets with both QR and Non-QR coded Crowdmark booklets. When the time comes to define the locations of your questions you do not need to label the multiple choice questions in your template.
Set default scoring options. From the assessment’s Dashboard, navigate to the Questions page in the sidebar. Click the Multiple choice settings button. Here, you have the option to:
- Edit the default points value for a correct response. By default students receive a score of 1 for a correct response.
- Edit the default points value for an incorrect response. By default, students receive a score of 0 for an incorrect response.
- Select how a question with multiple answers is interpreted. There are 3 options:
  - Either/or – students can select any one correct response to receive the full points value for the question, with no incorrect responses selected.
  - Partial – students receive partial credit for any correct response selected, with no incorrect responses. Points are distributed equally across the number of correct responses.
  - Exact – students must match all correct responses to receive credit, with no incorrect responses.
Note that any incorrect response selected will cancel out a correct response and award 0 points (or negative points, if that setting is used). These settings apply to all multiple choice questions in this assessment. Question-specific changes in the answer key (step 4) will override these settings.
Complete the answer key and customize question points. This is what allows Crowdmark’s AI to grade bubble sheets automatically. On the Questions page, click on “Answer key” beside the MC questions. Click the bubbles to select the correct answer(s) to each question. You may select multiple answers to a question. If the page shows more questions than you have, leave the extras blank; Crowdmark will only grade questions you have selected answers for. You may use the input boxes beside each question to change the number of points students will receive for a correct answer.
Print, administer, scan, and upload the completed booklets. Students can use pen or pencil to fill in the bubbles. Use the edge detection setting on your scanner to ensure that no black or grey background is included around the edges of the scanned images.
Crowdmark’s AI will automatically grade the bubble sheets that have been uploaded. Students receive the default scores (set in step 3) for each correct and incorrect response. The MC question/page score is the total for all multiple choice questions on that page. Changes to the answer key at any time before, during or after grading will trigger automatic re-scoring across the assessment.
Resolve any unknown responses. If Crowdmark is unsure about a response on a page or a student has not answered a question, the page will be marked with “unknown response” (and ungraded). This may happen if a student has mistakenly filled in a bubble but has attempted to cross it out or fill in a different bubble. On the Dashboard, under the Grading heading, you will see the MC question blocks. If there are unknown responses, click MC. You’ll see the first unresolved page in the grading interface. The unknown response will be highlighted in yellow with a “?”. Clone the evaluation and click the bubble that the student selected to resolve the response.

Troubleshooting bubble sheet issues

If you find there are errors with the bubble sheet scans or auto-grading, here are some things you can verify:

Errors can be caused by bad can quality or a visual artefact disrupting the detection of the multiple choice region (a line, shape, or writing/drawing).
It’s important to make sure the black triangles surrounding the bubble area are clear and present.
If a line is appearing, your scanner may need to be cleaned.
If the area where the staples were cut off is scanned in black it may cause a problem with detection. To solve this issue, ensure that your scanner makes any extra space around the scan white.

If the multiple choice area cannot be detected at all, Crowdmark treats the page as any other page in the template and you may grade it manually. You can enter a score in the sidebar and add annotations and comments to the page.

Getting Started for Instructor

How to Pair your Brightspace Course with Crowdmark

Go to your course page on Brightspace.
Go to Content. Follow the following steps based on the Content version you are using:

New Content experience:

Click into a Module (or create a new one)
Click Existing Activity
Select External Learning Tools
Select Crowdmark LTI 1.3
A link will be added to your Content page.

Legacy (original) Content experience:

Click into a Module (or create a new one)
Click on Add Exiting
Click on External Tool Activity
Select Crowdmark LTI 1.3
A link will be added to your Content page.

Click on the link that was added to Content. This will redirect you to Crowdmark where you should be brought right into your new course page. From here you can enrol students and create assessments. You Crowdmark experience should be mostly the same as before!

Notes:

If you don’t want students to access the link that is on the Content page, you can 'hide it from users'.

If this is your first time using Crowdmark, you will need to create an account with Crowdmark.

Enrolling Students into your Crowdmark course

In your Crowdmark course, click on Enroll students from Brightspace - LTI3.
Crowdmark will sync with the Brightspace course and access some of your students' details. You will be asked to confirm the students Name, Email, student ID#. The student role does not need to be mapped so select Nothing (ignore) for that data. Once you have confirmed the different categories click Continue.
Review your classlist and click Complete Import.
Students are now enrolled on Crowdmark