Making / Learning / Work was an adult learning innovation project hosted at the MIT Media Lab from September 2014 to May 2016. This site is an archive of the project documentation.
After redefining the goals of Libranet we needed to start looking for content!
To begin, we wanted to get a better sense of what kinds of basic education online learning content was out there. Because one of our other prototypes, Read Out Loud, is focused on developing basic literacy skills, we decided to narrow our search to basic numeracy content that was available freely online.
After a quick initial search for online and offline adult basic education courses designed to teach basic numeracy, we noticed that many courses had the explicit goal of preparing learners to take the GED (or other high school equivalency exam) and/or a college placement exam (e.g. COMPASS or Accuplacer). With this in mind, we defined our target learners as:
- Individuals who are not college-ready, needing remedial math to prepare for the GED or a college placement exam.
Next, we defined our content selection criteria:
- Length: Can the content be easily chunked into a 6-8 week study path?
- Cost: Is the content free and openly available to anyone with a computer and internet access?
- Format: Is the content preformatted into a coherent course of study?
- Quality of instruction: Is the content’s instruction high-quality and designed for an adult learning audience? Is the instruction interface easy to navigate?
- Quality of feedback: Does the content provide mechanisms for delivering meaningful and helpful feedback to users?
- Quality of assessments: Are the assessments easy to use? Do the assessments adapt to learner needs and enhance the learning experience?
With our target learners and content selection criteria defined, we set out to compile a list of potential content to choose from. However, this search process proved to be much more difficult than we initially anticipated. There really isn’t much online learning content out there that falls into the basic education category. The majority of course offerings start at the university level, and those that do focus on basic education topics are often explicitly designed for a high school audience, a feature that can be discouraging for adult learners. And even some of the basic math courses that we found to be intentionally built for an adult audience were unnecessarily “kiddie-fied” in their design.
Yet despite some of these initial findings, we were able to identify three potential choices for content: EdReady, Khan Academy Math, and Udacity’s Intro Algebra Review. To choose between these three, we used a decision matrix and the six selection criteria defined above. We assigned a weighting to each criterion, with the constraint that the total weighting score be equal to 10. Then, we rated each potential choice as to how well it achieved the criteria on a scale from 1-3 (1= slight extent, 2= some extent, 3= great extent). Finally, we applied the weighting to each individual score and summed them up to get overall scores for each of the choices.
Here you can see our final matrix:
With a score of 25, Khan Academy Math emerged as the winner. And if we look closely at the decision matrix, we can see that Khan Academy’s high quality of feedback and assessment were the two criterion that really set this solution out from the others. Khan Academy uses an adaptive web-based exercise system that generates problems for the user based on performance. These exercises also feature a fairly robust system of hints that allow the user to get help when they are struggling with a problem. Neither Udacity’s Intro Algebra course nor EdReady featured practice problems and assessments that we felt were this helpful to the learner. In fact, EdReady’s initial diagnostic test takes at least an hour to complete and makes it so that the learner never receives any feedback on how she did. With this as the first step in using the EdReady platform, we were worried that many learners might get overly discouraged, resulting in EdReady’s score of 1 in Assessment. So while EdReady had a great score in Instruction because of its contextual videos used for lesson introductions, its overall score was the lowest because of feedback and assessments. Udacity’s Intro Alegebra Review performed moderately well in most categories, with its most impressive feature being its format as a structured weekly course. But again, with fairly standard assessment and feedback, it did not surpass Khan Academy in overall score.
Given these results, we are planning to move forward with using Khan Academy Math for our initial pilot round. In the coming weeks, we will be playing around more with this platform and exploring how we might tailor the math content to align with the GED curriculum. In the future, we plan on revisiting the selection criteria defined here and tweaking them, given all that we will have learned from the pilot. It is very likely that our current criteria are skewed toward what we think will work best, and the hope is that as we try this out, we will be able to make better decisions about what types of content work best for small study groups of adult learners.