Net Assessment

Online test creation software
(IMS-compliant, XML/Java-based)

Client: The Office of Technology in Education at Carnegie Mellon University

Challenge: To design a system to enable professors in the math and sciences to create, administer, and grade tests, quizzes, and homework more efficiently and effectively.

Process: We performed 4 iterations of research, rapid prototyping, and user testing to hone the application flow and learn about the domain and user base. We applied a variety of usability methods such as contextual inquiry, card sorting, cognitive walkthrough, and “think aloud” user testing. From this research, we developed 3 disparate personas – each being a prototypical composite from various users' habits and personalities. We then created mockups to best support these different goals and workflows and re-tested these with more users. Our four prototypes were developed using paper, Flash, and 2 in Java, increasing fidelity in each round.

Solution: Our solution followed the open source IMS (Instructional Management Systems) specification for xml markup for standardized educational objects. One of the many issues we elegantly designed was the ability to manage enormous amounts of detailed data objects easily and intuitively. Some of the features included: 'Wizard' dialogs for the novice, 'Accelerator' controls for the experts, and 'Gentle Pedagogy' for all to help create more balanced tests through statistical analysis of the test's attributes. My roles included gathering requirements, writing functional specifications, interaction design, visual design, and building Flash prototypes.

Impact: Our prototype was used as the basis for a continuing project that is being developed at Carnegie Mellon University.

The repository browser window:

Users can browse, search, filter results, and add question objects from local and/or shared databases to their test.

Progress / statistics window:

Shows the progress of your test compared to an initial set of desired criteria (such as time length) broken down by pedagogical concept.

The question selection window:

Marks which question is selected and provides editing tools.