EDT 8140 - LMS and Evaluation
In this page I will go through the steps needed to perform the following tasks and provide an evaluation regarding the ease and quality of the result for two learning management systems, D2L and Blackboard Learn.
1. Creating an Electronic Gradebook
2. Creating a Dropbox and Grading Rubric
3. Creating Quiz Questions in a Question Library
4. Creating Quiz with Randomizing Question Set
5. Conduct Quality Check with Demo Student
1. Creating an Electronic Gradebook
2. Creating a Dropbox and Grading Rubric
3. Creating Quiz Questions in a Question Library
4. Creating Quiz with Randomizing Question Set
5. Conduct Quality Check with Demo Student
D2L Evaluation
1. Creating an Electronic Gradebook - D2L
The gradebook set up in D2L is greatly facilitated by the “Setup Wizard” which can be found under the “GRADES” tab of the Home page. This opens a page with the current grade book settings together with a link at the bottom of the page which will step through the various options for the grading design. Click on “Start” at the bottom of the page to begin the setup process (see arrow in figure). From the next several windows the grading system is defined through prompted questions and selections.
The gradebook set up in D2L is greatly facilitated by the “Setup Wizard” which can be found under the “GRADES” tab of the Home page. This opens a page with the current grade book settings together with a link at the bottom of the page which will step through the various options for the grading design. Click on “Start” at the bottom of the page to begin the setup process (see arrow in figure). From the next several windows the grading system is defined through prompted questions and selections.
2. Creating a Dropbox and Grading Rubric - D2L
Grading rubrics are defined from the “COURSE ADMIN” tab from the Home page. Select “Rubrics” from the list of options. To define a new rubric, select the “New Rubric” button from this page (see arrow in figure). This selection will take you through a set up page where you can define the name of the rubric, provide a description, and set up the basic properties of the rubric such as the number of grading levels and whether it will be a holistic or analytic system. The other tab in this setup routine is where the levels and criteria for the rubric are designated and scores defined for each level. Once the rubric is completed, be sure to return to the main properties tab and set the Rubric Status to “Published”.
The Dropbox option is defined using the “Dropbox” selection from the “COURSE ADMIN” tab on the Home page. Select “New Dropbox” to move to a set up page. Here you must define a name for the Dropbox and define other options such as performing an originality check (to detect plagiarism) and whether the Dropbox will be for individual or group submissions. Then select the Category and Grade Item (see previous page) to assign to the Dropbox and append the appropriate rubric.
Grading rubrics are defined from the “COURSE ADMIN” tab from the Home page. Select “Rubrics” from the list of options. To define a new rubric, select the “New Rubric” button from this page (see arrow in figure). This selection will take you through a set up page where you can define the name of the rubric, provide a description, and set up the basic properties of the rubric such as the number of grading levels and whether it will be a holistic or analytic system. The other tab in this setup routine is where the levels and criteria for the rubric are designated and scores defined for each level. Once the rubric is completed, be sure to return to the main properties tab and set the Rubric Status to “Published”.
The Dropbox option is defined using the “Dropbox” selection from the “COURSE ADMIN” tab on the Home page. Select “New Dropbox” to move to a set up page. Here you must define a name for the Dropbox and define other options such as performing an originality check (to detect plagiarism) and whether the Dropbox will be for individual or group submissions. Then select the Category and Grade Item (see previous page) to assign to the Dropbox and append the appropriate rubric.
3. Create Quiz Questions in Question Library - D2L
Access to the Question Library is via the COURSE ADMIN tab on the Home page. Select “Quizzes” and then click on the “Question Library”. This will open the list of questions in the library and provide access to links for writing new questions or importing questions from a csv file. Selecting “New” brings down a menu to choose the type of question (e.g. true-false, multiple choice, short answer). There are 12 different questions types available. In addition, a question “Section” can be defined. This will be useful to gather all questions relevant to a particular topic that can then be randomly chosen for exams.
Choosing “Import” opens a page where the file to be imported is chosen and the location for the files is defined. There also is a sample csv file that can be downloaded from this page if the Import Source is selected to indicate “From a Desire2Learn Text Format File”.
The following video demonstrates how to set up quiz questions and import questions into the library.
Access to the Question Library is via the COURSE ADMIN tab on the Home page. Select “Quizzes” and then click on the “Question Library”. This will open the list of questions in the library and provide access to links for writing new questions or importing questions from a csv file. Selecting “New” brings down a menu to choose the type of question (e.g. true-false, multiple choice, short answer). There are 12 different questions types available. In addition, a question “Section” can be defined. This will be useful to gather all questions relevant to a particular topic that can then be randomly chosen for exams.
Choosing “Import” opens a page where the file to be imported is chosen and the location for the files is defined. There also is a sample csv file that can be downloaded from this page if the Import Source is selected to indicate “From a Desire2Learn Text Format File”.
The following video demonstrates how to set up quiz questions and import questions into the library.
4. Create Quiz with Randomizing Question Set - D2L
This task seems somewhat non-intuitive in D2L; at least for me. Before setting up the randomized quiz, it will be easiest if you have loaded questions into the Question Library. With this done, select the ASSESSMENT tab from the main menu and choose “Quizzes and Exams” from the drop down menu. Then select the “New Quiz” button. A new page will appear in which you first must give a name to the quiz. Save this selection and when you are returned to the section page, select the name of your new quiz section and then select “Add/Edit Questions” button. Yet another page appears. Here you select the “New” button and click on “Random Selection” from the drop down menu (see arrow in figure) and give this randomized section of questions a name. Save this and now you can select “Import” and choose the Import Source as “From an Existing Collection” and Source Collection as “Question Library”. Specify the Source Selection if needed. Select the questions from the list provided and click on the “Save” button. Finally, fill in the box indicating the number of questions each randomized test will contain and be sure to click on the “Save” button before clicking on “Done Editing Questions” to leave this dialogue box. As above, don’t forget to activate the quiz under the “Restrictions” tab.
This task seems somewhat non-intuitive in D2L; at least for me. Before setting up the randomized quiz, it will be easiest if you have loaded questions into the Question Library. With this done, select the ASSESSMENT tab from the main menu and choose “Quizzes and Exams” from the drop down menu. Then select the “New Quiz” button. A new page will appear in which you first must give a name to the quiz. Save this selection and when you are returned to the section page, select the name of your new quiz section and then select “Add/Edit Questions” button. Yet another page appears. Here you select the “New” button and click on “Random Selection” from the drop down menu (see arrow in figure) and give this randomized section of questions a name. Save this and now you can select “Import” and choose the Import Source as “From an Existing Collection” and Source Collection as “Question Library”. Specify the Source Selection if needed. Select the questions from the list provided and click on the “Save” button. Finally, fill in the box indicating the number of questions each randomized test will contain and be sure to click on the “Save” button before clicking on “Done Editing Questions” to leave this dialogue box. As above, don’t forget to activate the quiz under the “Restrictions” tab.
5. Conduct Quality Check as Demo Student - D2L
It is important to check that the quizzes have been loaded properly and appear correct. To do this quality check as a demo student, first select the CLASSLIST tab on the right. Find the Demo Student in the class list (typically the last one) and click on the arrow to the right of the name. From the pop-up list, choose “Impersonate” (see arrow in figure). A confirmation window will appear to verify this selection. Now the screens will look just as a student would see them. Select GRADES from the Home page and then choose a quiz to examine. You can answer the questions as well. The impersonation mode is terminated by selecting the student name in the upper right corner of the Pilot page and clicking on the “x” next to the demo student’s name.
It is important to check that the quizzes have been loaded properly and appear correct. To do this quality check as a demo student, first select the CLASSLIST tab on the right. Find the Demo Student in the class list (typically the last one) and click on the arrow to the right of the name. From the pop-up list, choose “Impersonate” (see arrow in figure). A confirmation window will appear to verify this selection. Now the screens will look just as a student would see them. Select GRADES from the Home page and then choose a quiz to examine. You can answer the questions as well. The impersonation mode is terminated by selecting the student name in the upper right corner of the Pilot page and clicking on the “x” next to the demo student’s name.
Blackboard Learn Evaluation
1. Creating an Electronic Gradebook - Blackboard Learn
As tests and assignments are created in Blackboard, they automatically appear in the Grade Center. To access this information, select “Full Grade Center” under the Grade Center menu of the Control Panel (see arrow in figure). Student grades will appear here when assignments and tests are completed. The instructor has considerable flexibility in how the grades are displayed in this window and how they appear on the student’s grade page. In addition reports can be generated for students and administrators from options on this page.
As tests and assignments are created in Blackboard, they automatically appear in the Grade Center. To access this information, select “Full Grade Center” under the Grade Center menu of the Control Panel (see arrow in figure). Student grades will appear here when assignments and tests are completed. The instructor has considerable flexibility in how the grades are displayed in this window and how they appear on the student’s grade page. In addition reports can be generated for students and administrators from options on this page.
2. Creating a Dropbox and Grading Rubric - Blackboard Learn
A grading rubric is created from the “Rubric” option under the “Course Tools” menu of the Control Panel. Select “Create Rubric” from the page that appears and provide a name for the rubric. Then scroll down to fill in the details of the Levels of Achievement, Criteria, and scoring for each. Finally click on the “Submit” button to save the rubric (see arrow in figure).
A grading rubric is created from the “Rubric” option under the “Course Tools” menu of the Control Panel. Select “Create Rubric” from the page that appears and provide a name for the rubric. Then scroll down to fill in the details of the Levels of Achievement, Criteria, and scoring for each. Finally click on the “Submit” button to save the rubric (see arrow in figure).
Assignment dropboxes are created in the Content page of the main Course menu. Select “Assignments” from the Assessments menu (see arrow in figure) to open the page for creating the assignment. Provide a name, instructions and other information as needed. Only the name and points are required entries. Here you can add the grading rubric that you created above. Click on “Submit” and the assignment will now appear on the course Home page.
3. Create Quiz Questions in Question Library - Blackboard Learn
In Blackboard Learn, the Question Library is called a Pool. To create a question pool, select “Tests, Surveys, and Pools” under the “Course Tools” menu of the Control Panel. Then Select “Pools” (see arrow in figure). From the page that appears, you can first create a pool and give it a name by selecting the Build Pool option. Then the pool must be loaded with questions. This is done by selecting the “Create Question” button. A variety of question types can be created. Select one and then fill in the appropriate components and options. Alternatively, you can select “Upload Questions” and direct the browser to a specially formatted txt file containing the questions.
In Blackboard Learn, the Question Library is called a Pool. To create a question pool, select “Tests, Surveys, and Pools” under the “Course Tools” menu of the Control Panel. Then Select “Pools” (see arrow in figure). From the page that appears, you can first create a pool and give it a name by selecting the Build Pool option. Then the pool must be loaded with questions. This is done by selecting the “Create Question” button. A variety of question types can be created. Select one and then fill in the appropriate components and options. Alternatively, you can select “Upload Questions” and direct the browser to a specially formatted txt file containing the questions.
4. Create Quiz with Randomizing Question Set - Blackboard Learn
Once a question pool is created, you can create tests. To make a test with randomized question selections, Select “Tests” from the “Tests Surveys, and Pools” page described above. Then select “Build Test” and give the test a name, description and other options if needed. Clicking on “Select” then brings up a page where questions will be added to the test. Select the “Reuse Question” menu and click on “Create Random Block” (see arrow in figure). This opens a new window where you will select the question pool to provide the questions and the question types to be included. Click on “Submit” to include the question pool in the test and return to the main window. Here you indicate how many of the pool questions will appear on the test and how many points are offered for each question. Be sure to click on “OK” at the bottom of the page to confirm your selections.
Once a question pool is created, you can create tests. To make a test with randomized question selections, Select “Tests” from the “Tests Surveys, and Pools” page described above. Then select “Build Test” and give the test a name, description and other options if needed. Clicking on “Select” then brings up a page where questions will be added to the test. Select the “Reuse Question” menu and click on “Create Random Block” (see arrow in figure). This opens a new window where you will select the question pool to provide the questions and the question types to be included. Click on “Submit” to include the question pool in the test and return to the main window. Here you indicate how many of the pool questions will appear on the test and how many points are offered for each question. Be sure to click on “OK” at the bottom of the page to confirm your selections.
5. Conduct Quality Check as Demo Student - Blackboard Learn
While there is a means in some versions to add a test student (see arrow in figure) or preview the course as a student, in the version we have available these option are not present. For some functions, turning off the Edit mode can be used to examine the student view but this is not adequate for examining tests.
While there is a means in some versions to add a test student (see arrow in figure) or preview the course as a student, in the version we have available these option are not present. For some functions, turning off the Edit mode can be used to examine the student view but this is not adequate for examining tests.
LMS Weekly Recommendation
1. The gradebook widard in D2L was extremely easy to use and useful. The step-by-step instructions were clear and informative. Alternatively, there is not much to do at all in Blackboard Learn. The gradebook is automatically populated as assignments and tests are created. There are easily available options for viewing the class grades and for creating reports in Blackboard Learn. Overall, I give the extra point to D2L on this one for the great setup wizard.
Score: Blackboard – 0 : D2L – 1
2. The dropbox and assignment grading rubric for both systems were pretty similar. D2L also has the plagiarism option built right in (although this is a third-party program). All in all they were similar in most respects and so get equal points.
Score : Blackboard – 1 : D2L – 1
3. Creating quiz questions will be an onerous task in either system if the questions are entered one by one through the web interface. Setting up a file to upload may be easier although the format and syntax of the file requirements don’t make this option very easy either. In the end, D2L had more options for the uploaded questions so that additional editing would not be necessary. In addition D2L provides the option for including an image in the question – not an option in Blackboard Learn. So again D2L gets the extra point.
Score : Blackboard – 0 : D2L – 1
4. Both systems use a similar design for constructing a randomized test. Questions are selected from already existing questions in the question library (or pool). The screens that appear in the D2L LMS are very similar so that it is difficult sometimes to know where you are in the process of defining the test, selecting the questions, and choosing options. The Blackboard Learn method has fewer options and the screens that are encountered along the way are different enough that you don’t lose your way as readily. It’s a close call here, but I think I give the extra point to Blackboard Learn, just for it’s simplicity.
Score: Blackboard – 1 : D2L – 0
5. Here’s where the version of Blackboard Learn that we have to evaluate fails. There is no effective way of examining the assignments and tests created in Blackboard Learn as a student. Apparently institutional versions of the program can make demo students which the instructor then can impersonate so this may not be a problem for university courses. Nevertheless, individual questions and assignments can be previewed, although the grading and dropbox functions cannot be used. D2L has a very good means to evaluate the course as a student. Tests can be taken and grades generated. For the versions we have to evaluate, D2L takes the extra point.
Score: Blackboard – 0 : D2L – 1
Final score:
Blackboard – 2 : D2L – 4
Although not a perfect score, the D2L system furthers it's lead this week.
1. The gradebook widard in D2L was extremely easy to use and useful. The step-by-step instructions were clear and informative. Alternatively, there is not much to do at all in Blackboard Learn. The gradebook is automatically populated as assignments and tests are created. There are easily available options for viewing the class grades and for creating reports in Blackboard Learn. Overall, I give the extra point to D2L on this one for the great setup wizard.
Score: Blackboard – 0 : D2L – 1
2. The dropbox and assignment grading rubric for both systems were pretty similar. D2L also has the plagiarism option built right in (although this is a third-party program). All in all they were similar in most respects and so get equal points.
Score : Blackboard – 1 : D2L – 1
3. Creating quiz questions will be an onerous task in either system if the questions are entered one by one through the web interface. Setting up a file to upload may be easier although the format and syntax of the file requirements don’t make this option very easy either. In the end, D2L had more options for the uploaded questions so that additional editing would not be necessary. In addition D2L provides the option for including an image in the question – not an option in Blackboard Learn. So again D2L gets the extra point.
Score : Blackboard – 0 : D2L – 1
4. Both systems use a similar design for constructing a randomized test. Questions are selected from already existing questions in the question library (or pool). The screens that appear in the D2L LMS are very similar so that it is difficult sometimes to know where you are in the process of defining the test, selecting the questions, and choosing options. The Blackboard Learn method has fewer options and the screens that are encountered along the way are different enough that you don’t lose your way as readily. It’s a close call here, but I think I give the extra point to Blackboard Learn, just for it’s simplicity.
Score: Blackboard – 1 : D2L – 0
5. Here’s where the version of Blackboard Learn that we have to evaluate fails. There is no effective way of examining the assignments and tests created in Blackboard Learn as a student. Apparently institutional versions of the program can make demo students which the instructor then can impersonate so this may not be a problem for university courses. Nevertheless, individual questions and assignments can be previewed, although the grading and dropbox functions cannot be used. D2L has a very good means to evaluate the course as a student. Tests can be taken and grades generated. For the versions we have to evaluate, D2L takes the extra point.
Score: Blackboard – 0 : D2L – 1
Final score:
Blackboard – 2 : D2L – 4
Although not a perfect score, the D2L system furthers it's lead this week.