Portable Competencies
Mark J. Norton
June 4, 2001
There is a strong desire on the part of trainers and educators to measure competencies in individuals and design courseware and testing to improve them. There is a need to measure skills and to able to certify mastery of concepts and skills. While there are well understood methods of creating a system which implements this need, can it be done in a portable manner?
Use Case
For the sake of exploration, let’s consider a simple use case. Being able to add numbers together is a skill which utilizes certain math facts, and algorithms to extend them to larger cases. The facts are the combinations of the single digits from 0 to 9. Algorithmic extensions introduce the notion of a carry.
We can be build courseware to teach this skill (Math Blaster is a good example). We can also create an online test to determine if a user has a mastery of addition. Now suppose this course and this test are present in a centralized learning system.
A user can take this course, then the provided test. If she passes, we can say that she has mastered the skill of addition. If we are designing an integrated system (for a K12 environment, say), we might indicate mastery by awarding a certificate. Alternatively, we might make an entry in a grade book, etc.
Consider a second course called “Arithmetic” . This course is composed of four modules: Addition, Subtraction, Multiplication, and Division. In this case, the skill to be mastered is “Arithmetic”, which includes four sub-skills. Is the Addition skill in this course equivelant to the standalone course above?
Problem Statement
The crux of the problem is this: how is addition as a competency represented in the system? Can we identify competencies in a manner which is portable from system to system? Competency need to be associated with courseware, tests, and learning profiles.
If a test results in a certificate, these can be collected into the user’s learner profile, perhaps as a text string. We could generalize this a bit, and assign some sort of ID to a competency. Profiles would then refer to the ID number. Unfortunately, such ID’s will be different on every system. Whenever a course, test, or user profile is moved, the IDs will be invalid.
Ultimately, skills, competencies, objectives, achievements, etc. are all bound in natural language. There is no master structure of human knowledge which would allow universal identifiers for mastered concepts or skills. Although attempts at taxonomies have been made, they change over time. If taxonomy identifiers are used with competencies, a reference needs to be made to the taxonomy used. This is partially specified in the IMS Meta-Data specification.
Representing Compentencies
A competency can be defined to include the following pieces of information:
Identifier (GUID)
Dependencies (list)
IMS meta-data (structure)
Name
Description
Taxonomy reference
Each competency is uniquely identified using a GUID. A GUID is a string which is garaunteed to be globally unique. Beyond this identifier, IMS meta-data can be included to define a name, description, taxonomic entry, etc.
Dependencies allow one competency to define a relationship with another one. Such relationships can be either required or optional. They can be used to create prerequisite requirements, or define larger competencies based on the mastery of smaller ones.
These elements allow hiearchies of competencies, skills, and concepts to be created. They allow dependencies to be idenfied and managed. As long as a test points to the same compentency that a course teaches, the mastery of skills and concepts by a user can be measured and recorded.
Portability
Measuring competency is done via assessment. Assessment usually consists of a series of questions or exercises designed to probe the users understanding of the concepts or elements which make up the competency that it is testing. If an acceptable level of correct answers are made, the user is said to have mastered that competency. As such, a test needs to identify the competency that it tests for.
If the user fails the test, it would be nice to make a set of recommendations to remediate the student’s understanding. In order to do this, courseware must also be labeled with the competency that it is designed to teach. This technique is called perscriptive testing. Testing can also be adminstered after the completion of one or more units of learning. Somehow, the test associated with this set of learning activities must be made so that the student knows which test to take given the work just completed. If these learning activities and test are similarly identified with a common competency identifier, the appropriate test can be adminstered. This is sometimes called a mastery test.
If we then move a course from one system to another, we are no longer assured that there is a corresponding test for it. We could require that corresponding tests always be moved along with the course, but that’s a simplistic soution that is likely to break down. Consider that the target system may already have a good test for that competency and there is no need for another.
Instead of relying on tightly coupling learning material and tests, a means of defining equivelency of competencies is needed. Two kinds of equivelences are recommended: systematic and administered. Systematic equivalents are based by comparing two competency entries. If the identifiers are equivalent, the competencies are considered to be the same. Adminstered equivalents, on the other hand, are done manually by an administrator. When a course or test is added to the system, the administer is notified that this competency does not correspond to an existing one and they should define it as new, or set it equal to some other competency. Equivalency relationships can be made more complex if they are represented as production rules stored with the competencies.
User Profiles
Accomplishments, certificates, notes of completion, etc. are strongly associated with competencies, objectives and skills. These are normally part of a Learner Information Package (see IMS LIP specification). Entries are made into the learner’s profile when conditions are met (passing a test, etc.).