ESCOLA DE ENGENHARIA DA UNIVERSIDADE DO MINHO

CENTRO DE INVESTIGAÇÃO ALGORITMI

Technical Report EI-12002 (Dec 2002)

Question Based Learning in Computer

PAULO GARRIDO
Department of Industrial Electronics and also
Algoritmi Research Centre
School of Engineering
University of Minho
4700 Braga, Portugal
[email protected]

Abstract
This paper proposes concepts and a methodology for the design of question-based computer
courses. It is inspired in Socratic maieutics on one hand and in automatic generated or adaptive
tests on the other. For brevity one will call them m-courses and the computer systems able to run
them m-systems.
The core of the proposal lies on putting together several ideas. Presenting simultaneously a multiple
sentence question and a learning process for the knowledge the question refers to. Structuring the
knowledge domain of the course in molecules that satisfy a learnability or constructibility relation.
Associating to each molecule a set of questions and a learning process. Presenting the course by
presenting the questions in a sequence that respects the learnability relation. Allow the advance of
the learner in the sequence according to her answers.
It is believed that m-courses will be attractive, will enforce activity of the learner and will direct
efficiently his learning efforts. This belief can only be proved or disproved in practice.
Besides giving a general overview of the proposed systems, the paper describes the knowledge
structure with emphasis on the learnability relation. The two main components of an m-system, the
m-description and the m-engine are specified to a level that a mapping analysis to available
software may be made onwards.
Design considerations for m-courses and lines of development and research are also considered.

Keywords
Question-based learning, question-based courses, e-courses design, e-learning, automatic generation
of tests, learning interfaces.
1 Introduction
Posing questions is crucial for learning and teaching. In Western culture, acknowledging
the importance of asking questions for a learner’s advance goes back to Socrates and his
pedagogical method known as maieutics [7]. Socrates’ goals were different from the goals
of Science and Engineering but the research method is the same: ask a question, get an
answer, ask another question…
If learning is seen as process of discovery, then maieutics ideas apply. Good questions
direct a learner’s attention and energy in profitable lines of enquiry, reflection or
experimentation. In addition, they reveal to the learner his/her conceptual weaknesses for
the learning of the matter at hand, and so promote self-reflection on his/her actual
knowledge.
Questions are also used as standard and ubiquitous tool in evaluating a learner’s
knowledge. Tests and examinations usually are constituted by a set of questions – and,
more often than not, only by a set of questions.
In classical teaching, one poses questions orally to a class of students or writes them
down in exercises, tests and examinations. This use of questions stands short from the
whole potential of the method, unless the class is small or the teacher has the time,
willingness and opportunity to discuss individually or in short groups the answers given.
The use of computers as an aid to learning, learning evaluation and teaching rises the
opportunity for automating question posing and answer evaluation. Depending on the
degree of sophistication and characteristics such automation may attain, one may create
several tools for facilitating learning and teaching tasks, from automating multiple-choice
tests to guiding learning. Guiding learning is the objective to consider in this paper.
Of course, it is not expectable in the near future that a computer program may exhibit
the capabilities of an experienced teacher working with an individual learner or a small
group of learners. However, given that programs with the capabilities described in this
paper are available, their use by many students may speed up their learning progress
considerably without teacher intervention. This benefits the time available to exercise more
dialogue with students.

1.1 M-systems and M-courses

In this paper, one proposes conceptual foundations and methodological issues for the
design of computer programs and environments aimed at facilitating learning, based on
goal oriented question posing and answer evaluation. For brevity, one will call such
programs or environments m-systems or m-courses.
A concrete m-system or m-course is a connection of an m-system engine with an m-
learning description for the knowledge domain of the course. The m-system engine is the
abstract computer-learning environment able to behave towards a user as a goal oriented
question poser. It makes this on the basis of the m-learning description (also, m-
description) to be provided by a course designer. The m-learning description is patterned
after a knowledge description structure.
In this paper one considers m-systems where the knowledge description structure sees
the knowledge domain as nodes or molecules connected by arrows as to form a directed
acyclic graph. Each node or knowledge molecule is described by a label or name, a set of
pre-requisite nodes to be learnt (or passed) before starting the node, a learning process for
the molecule at the node including questions, learning resources, messages and guiding
directives. Questions are to be written in natural language in a multiple sentence format and

1
Paulo Garrido Question Based Learning in Computer

answering allowed by choosing one value of an alphabet1. Here one will suppose a binary
alphabet with the meanings: “the sentence is true”; “the sentence is false”.
Adding to the m-learning description resources as texts, Web access to digital libraries,
presentations, simulations, digital imaging, access to Web labs, will augment significantly
the ease-of-use and power of the m-course.
A user follows an m-course by registering as a learner to the m-system and then going
into a sequence of learning knowledge molecules. At each step of the sequence the user is
presented a question capable of testing his or her knowledge of the referred molecule.
What is to be learnt for answering the question is indicated. When the user submits a
passing answer she or he is led into the next step of the sequence.
An m-system or m-course may support non-assisted learning or it may be integrated in
a larger learning context. It can be a productivity tool for the team constituted by the
teacher and students, helping for improved learning activity and results. Although the
development of the concept of an m-course made here targets a personal tool, its
integration in collective environments is highly desirable.

1.2 Key notions and their interest

For guiding learning, m-systems make converge and explore one pedagogical notion and a
model of knowledge structure. The explored pedagogical notion is question posing as an
enhancer for learning. Asking “why this?” or “how this?” is a more efficient method to
turn a passive spectator in an active learner than telling him “learn this!” At the level of
information communication a message that requires response, implies that the receiver
becomes active or gives up of the transactional process. At the level of knowledge building
i.e. learning activity is recognized as a necessary condition for learning. As in Natural
Learning [8]. Besides, if immediate information is given on what the learner can do to find
a satisfactory answer in a reasonable time, one can expect directed action of the learner
toward the goal of learning. This is an important point to stress. A question in an m-system
should imply for its answer a reasonable, and so satisfactory, learning process2.

Discrete, enumerated, structured domain to learn

Structuring a domain means to define elements of the domain and relations over it. The
model of knowledge structure assumed here for m-systems sees a knowledge domain to be
learnt as discretisable in knowledge nodes or molecules. The term ‘node’ comes from the
proposed structure being representable by a graph3. The term ‘molecule’ comes from
seeing knowledge as made up of molecules and learning as synthesising knowledge
molecules. In this chemical like metaphor (see the Appendix), one knows something or one
is in possession of some knowledge if one has synthesised and is in possession of a specific
knowledge molecule.

Learning for answering a question

In an m-course the definition of the nodes or molecules of the knowledge domain covered
by the course is made by the course designer. For each node a learning process and
questions should be provided. Questions have the double role of motivating and checking
learning. This means that when a user gives a ‘passing’ answer, the m-system assumes she
or he has learnt the knowledge molecule the question refers to and proceeds to another
node. It is favourable that the definition of the knowledge molecule gives rise to a
satisfactory learning process as referred above.

1 The number of letters of the alphabet in a m-system may be greater than 2.

2 Too easy the process may impoverish an adequate synthesis of what is being learnt overall. Too
difficult the process may push the learner to give-up or to lessen interest.
3 In this case, an activity graph for learning.

2
Paulo Garrido Question Based Learning in Computer

Pre-requisites for anything to learn made explicit

It is a well-known fact of Chemistry that synthesising some molecule can only happen if a
given set of precursor molecules is present. Also in teaching one knows well that learning
or building the knowledge of a given matter is often not possible without being in
possession of pre-requisite knowledge. One may put this notion at work defining a
learnability relation among nodes, sets of nodes and learning processes. A node κ has a building set in
set B of nodes if (the course designer believes that) it is necessary and sufficient for the
learner to know all the elements of B to be able to learn κ through learning process L Bκ .
In an m-course, the designer should define for each node or molecule a set of building
nodes or molecules and a learning process. For the purposes of this paper it will be
assumed that at most one building set is defined for each node. This simplifies things
because the learnability relation becomes a function, but should not be taken as a
restriction for m-systems in general.
The learnability relation as defined should embody the condition that knowing a
molecule cannot be a pre-requisite for learning it, explicitly or implicitly. This condition and
the uniqueness of building sets structure the knowledge domain as a directed acyclic graph.
Knowledge molecules reside at the nodes, and an arrow goes from node ν to node κ, when
ν belongs to B, the building set of κ.
This structure for knowledge derives from a basic property of the learning process: it is
a constructive process. It must go from what the learner knows to what she or he does not
know. This raises the requirement that the learner had access to all knowledge4 sufficient
for learning something, before he or she is given the goal of learning that something. Any
well-written textbook is in accordance with it. Curricula respect it. Virtually, it is found
implicitly everywhere one teaches. M-systems enforce that this requirement be made explicit, by
requiring the designer to specify a building set for each node.
For the learner, visualising the graph of the knowledge domain to be learnt makes
explicit what should be learnt first. This clearly helps in forming in the user’s mind a
determinate idea of the learning task. Furthermore, at each node information on the
accomplished building set may be given, as well as information on the building set that is
being learnt and which nodes proceed from there on. The nodes passed or accomplished
by the learner may be depicted on the graph, creating a map of the user’s learning progress.

Enforcement of satisfaction of pre-requisites

But more than visualisation aids, imports the m-courses criterion of not proposing a
learning goal before the user gives evidence of being in possession of the goal’s pre-
requisite knowledge internal to the course5: no node κ is presented for learning before all
nodes in its building set B, belonging to the course, are passed. This way, it is enforced that
no time is spent trying to learn what is assumed not to be possible to learn. Of course, an
m-system cannot hinder a user from engaging in trying to learn the matter of a question
with failing pre-requisite knowledge external to the m-course. But it can give warning on
required external knowledge for learning a node. Should it become evident that the learner
is lacking the external knowledge, explicit advice can be given6.
A presentation of the nodes of the course, which respects the ‘building set before
molecule’ requirement, is given by the sequence of nodes being in a class of sequences,
known as topological sorts of a directed acyclic graph [6]. In general there are several
topological sorts for a directed acyclic graph. The m-system engine can generate sorts and
these may be selected to satisfy additional criteria or any choices the designer intends

4 Inside the domain to be learned.

5 This does not mean not to show questions about matters the learner cannot yet tackle.
6 The explicit advice could be to follow or revise an area of a previous m-course.

3
Paulo Garrido Question Based Learning in Computer

sensible. For example, the graph representation may lead to several equivalent sequences.
In this case the m-system engine can only choose at random, but the course designer can
give the most desirable solution. Arrived at a sort that satisfies his or her goals, the designer
can make it the learning presentation script for the course. Actually the designer may eventually
define not only one but several presentation scripts. This raises the possibility that the
learner follows a course according different presentations.
The existence of sorted presentations does not oblige the learner to begin a course at
the first node of a presentation. Learners may be allowed to begin the course at a node of
their choice. The automatic embedded capabilities for revising should take care in the event
the learner fails the node chosen. This can accommodate different learning styles,
maintaining satisfaction of pre-requisites.

Learning and preparation for evaluation in one system

M-systems can function in two ways. In learning mode, an m-system engine will guide a
learner along the knowledge structure according to a learning presentation script. The
script will take the learner from the initial nodes of the graph to the terminal ones. When the
learner answers successfully to a question of the last terminal node, the taking of the course
comes to an end. An interest of m-courses is that questions posed to the user can be
variants of questions used in examinations, or parts of examinations. This way learning and
preparing for examinations (or parts of it) become a unified activity of the student.
An m-system engine should be able to generate sequences of questions according to
evaluation scripts. Evaluation scripts are used in evaluating mode. In evaluating mode, questions
are posed, as they would be in an automatic test. Evaluation scripts can range from a fixed
sequence of questions to a sequence parameterised by the answers given by the user and
exploring the graph information. As a result of the evaluation, a map of user’s knowledge
may be created and displayed. This can be done by default as the initial activity of the
course, checking in particular the possession by the user of pre-requisite knowledge
external to the m-course. Evaluating mode may be used for a learner to train, revise or
exercise his or her knowledge over the course’s entire domain or over selected areas of it.

Flexibility for design

M-systems engines are to support specialisations of the above default question sequencing
strategies through guiding directives. For example, if a learner submits an answer with a
pattern that indicates a special difficulty, that could be given special attention by revising a
pre-requisite node, rather then staying at the node waiting for the user to give the passing
answer. Guiding directives constitute a kind of meta-system guiding information to be
stored in the learning description of each node7.
To sum up things for the moment, one will say that, in learning mode, an m-system
guides a learner by enforcing that he or she progresses according to a basic criterion of
mastered pre-requisite knowledge, to which other criteria may be added. Also, at each
node, information is made available on how to learn what the learner is ready to learn. So,
guiding learning happens not only along the time sequence of the course, but also at each
step of the sequence. A possibility not referred yet is that one-step ahead or n-step ahead
learning goals may be presented, stressing what immediate learning effort at the current
node is directed to.

7 In fact, the reaction pattern to answers of an m-system engine is to be so that the course designer can
ignore any explicit pre-requisite knowledge information and define paths for exploring the domain knowledge
all of its own.

4
Paulo Garrido Question Based Learning in Computer

1.3 Plan of the paper

In the next section, a review of related systems is made, simultaneously showing how the
concept of m-systems can also be developed from automatic multiple-choice tests. In
section 3, the model of knowledge structure and its learning description will be presented
formally. The full contents of an m-description will be discussed in section 4. In section 5,
the different components of an m-system engine are considered.
It appears that implementing an m-system engine does not pose special problems for
today’s software technology. Assuming the existence of m-system engines, an m-course
development amounts basically to the design of an m-learning description for the
knowledge domain. This design seems a key factor for the success of a m-course and some
issues will be discussed in section 6, which also concludes, indicating open questions and
lines of development of the concept.

2 Questions in Computer Assisted Teaching and Learning

Computers are being widely used to support and facilitate teaching and learning. Here a
non-exhaustive review of concepts and systems where question posing plays a key role is
made.

2.1 Automated multiple choice tests

A basic use for a computer is in easing the burden of written tests generation and
correction. Supposing that one has a large database of questions for multiple choices
answering, test forms can be easily generated, presented at a computer monitor in adequate
timings and the student answers recorded, corrected and graded.
If learning evaluation stands only on this type of tests, the use of computers is
enormously advantageous. It eases the burden of the teacher and can give the student
immediate information about the grade obtained. Given the adequate hardware
environment, such a system allows for making intermediate term examinations a standard
practice, even for large classes. Intermediate term examinations are deemed as a tool for
enforcing learners to study steadily along the semester, augmenting the chances for deep
learning.
This basic application already raises some requisites. The length of the questions
database must be enough to prevent students from memorising correct answers by rote, in
the event that they know the contents of the database. Furthermore, the algorithm for
picking questions from the database must lead to a reasonable test form content, with
questions covering the matters to be evaluated in a consistent way. This implies that the
database be at least minimally structured in areas of knowledge that the questions refer to.

2.2 Providing feedback to the teacher and learner

The possibility of automatically processing the answers to tests, may give the teacher
detailed feedback on his/her students progress. In this perspective, one is interested in
performing tests, not (or at least not only) for examination or grading purposes, but for
getting information on learning progress and directing study or teaching accordingly.
With this usage in mind, it makes sense that several tests are performed along the
course. For courses on matters whose concepts are strongly serially connected, this may
prove an invaluable tool to early detect and prevent students being unsuccessful due to
failure of learning in the appropriate timing. In such courses, failure to learn a single, early
presented concept, may result in the student wasting all subsequent time spent in
classroom.

5
Paulo Garrido Question Based Learning in Computer

Given that the database of questions is structured, the requirements for giving
feedback to an individual learner are of pinpointing the concepts, competences or skills
which missing may have led to the wrong answers and of giving advice on how to study or
learning them.
In this view, the usage that a teacher may intend to have from automatic processing of
questions is manifold. They allow for easy characterising a class in terms of mean capacity,
standard deviation and so on. Refinement of analysis – eventually over the disciplines of a
year, or along a student’s history – allows for creating indicators of individual learners
preferences and learning competences.
The analysis may pinpoint matters that need further classroom or laboratory work. In
addition, it may give advice about how far to push the exposition of matters, with realistic
expectations on effective learning. Alternatively, thinking the other way round, it may
reveal which cautions and preliminary work are necessary to introduce new matters in a
profitable way.
Such usage requires statistical analysis tools for the different populations of answers
generated, as well as means to chain them along time, with relation to the areas of
knowledge at hand.

2.3 Guiding learning

The drive for learning may come from necessity, from curiosity or from both. In any case,
the learning goals, be them explicit or implicit to the learner, may be formulated as a
question or group of questions. In addition, learning is obtained in the process of searching
for answers and is confirmed if the answers show right. These two assumptions lie at the
heart of the m-systems proposal.
In m-systems question posing and answering may serve evaluation purposes, but
another goal is basically pursued. This goal is not to grade students, nor to give only
feedback about a learner’s progress. The goal is to guide the learner along study and,
hopefully, along the learning process.
For such goal, the selection of questions to be presented to the student must be more
sophisticated than in the evaluation case. It is of no use to present a question for which
there is not a reasonable expectation of getting a good or right answer, on the grounds of
the assumable learner’s knowledge.
One notes that this requirement is approached by adaptive tests [1]. Adaptive tests do
not go into a more advanced, n+1, level of questions until the learner is not proficient in
answering level n. Adaptive tests are a good training tool for students providing implicitly a
guide for study. They are realizable in m-systems, exploring the knowledge structure
conveyed by the course designer in the m-learning description.
To guide learning m-systems propose as the basic criterion the learnability relation
presented above. This learnability relation is of course present in any textbook,
presentation or computer course. Making the relation explicit as in m-systems is a
facilitating tool for teaching and learning. The question form of interaction in m-systems,
serves down-top, i.e., initial to terminal necessary learning, but also allows top-down
presentation of the knowledge domain to the nodes where learning can effectively be
rooted in previous knowledge.

3 The Model for Knowledge and Learning

In this section the model for structuring knowledge assumed in this paper for m-systems is
developed. As already said the basic intuitions for the model are conveyed through a
chemical like metaphor.

6
Paulo Garrido Question Based Learning in Computer

One will see a knowledge body Κ as a structure made up of knowledge molecules

κ 1 ,..., κ j ,..., κ N . One will take K, the knowledge domain or knowledge set of Κ, as given
by:
K = {κ 1 ,..., κ j ,..., κ N } . (3.1)

One will not attempt to define what a knowledge molecule is, although arguments for
taking this metaphor are given in the Appendix. For the purposes of this paper, one will
assume that if an entity knows something, then this is synonymous to the entity being in
possession of some knowledge molecule κ.
Viewing knowledge as made up of molecules (or knowing as being in possession of
knowledge molecules) leads naturally to viewing learning as the construction, building or
synthesizing of knowledge molecules one does not possess. Yet, this construction does not
happen in a vacuum, nor is random. One will assume that the construction (learning) of
some knowledge molecule κ can proceed and can only proceed, if a given building set of
molecules, or pre-requisite set of molecules, B , is in possession of the learner and this one
undergoes with success a learning process, L Bκ , proper to B and κ. The knowledge of B is
necessary and sufficient condition for being possible that the learner builds κ by way of
L Bκ . This can be expressed as
L Bκ
B → κ. (3.2)
The special case, where B is the empty set is to be admitted. The construction process
Lκ
∅  →κ (3.3)
refers to a knowledge molecule for which a learning process suffices for construction. The
reason for allowing empty building sets for a given molecule is practical rather than
conceptual. For virtually any knowledge body, it is conceivable that there are buildable
knowledge molecules, which have as pre-requisites only cognitive capabilities of the learner
such as reading, writing, make some experience. These are pre-requisites that one does not
want to indicate explicitly, that is, by necessity or convenience one assumes they are in
possession of the learner.
Interpreting expression (3.2) one has in mind that:
L
– The symbol  → is used with the primary meaning of “construction”, not of
“transformation”, as a learning process does not change the pre-requisite
knowledge molecules the learner already possesses.
– It is not to be interpreted as “association” or “mapping” from the point of view
of the learner before he or she learns κ, because that would imply that κ would be
in possession of the learner before L would occur. This restriction does not
apply after the learner has learned κ. The memory of the learning process also
has information on the knowledge structure.
The assignment of building sets to molecules over a domain has to satisfy a consistency
condition: the assignment should not state that learning some κ depends on the previous
possession of itself. This could happen if the assignment of building sets would generate
learning loops. In fact the learnability relation expressed by (3.2) requires this consistency
condition for its definition.
It is conceivable that a knowledge molecule may be constructed from different
building sets B1 , B2 ,... As referred in the introduction, in this paper one will assume that
for each molecule at most one building set is defined. The extent to which this is restrictive
is an open question, which should not hinder one from going on. Allowing more than one

7
Paulo Garrido Question Based Learning in Computer

building block may be interesting, but it complicates the m-system engine. It is hoped that
further research clarifies this issue.
Having one only building set for each set makes the building set a function of the
molecule. In this context, (3.2) can be rewritten as
Lκ
Bκ  →κ , (3.4)
Where the indication of dependency of Lκ in Bκ has been omitted for simplicity.
For the purposes of this paper, (3.4) summarises conditions upon which to base
design, which one will express as the “possibility” and the “necessity” readings of (3.4),
respectively:
– It is believed that if a learner is in possession of knowledge set Bκ and
undergoes learning process Lκ then he or she will possibly learn κ.
– It is believed that if a learner is not in possession of knowledge molecule κ, then
either he or she is not in possession of Bκ , or did not undergo Lκ or had not
success in undergoing L κ.

3.1 Structural descriptions of knowledge bodies

Different speakers may refer a knowledge body by the same name and yet give different
descriptions of it, as well as of the ways to learn it. This fact is acknowledged, in the
context of m-systems, by learning structure descriptions. A description D of a learning
structure for a knowledge body Κ is understood to be a pair:
D( Κ ) = < K D , L D > , (3.5)
where KD is the knowledge set assigned by D to Κ and LD is the learning map assigned by D
to the elements of KD.
The learning map LD assigns to each knowledge molecule in KD a pair:
L D : κ 6 < Bκ , L κ > , (3.6)

where Bκ is a building set for κ and Lκ is the corresponding learning process for κ.
A building set Bκ may be a subset of KD or it may have elements belonging to
knowledge bodies other than Κ.
The relationship “knowledge molecule ν belongs to the building set of knowledge
molecule κ” allows one to represent partially a learning structure description of a
knowledge body as a (monochromatically edged) directed acyclic graph (DAG). The nodes
of the graph are knowledge molecules. An arrow exists between two nodes when one is in
the building set of the other. As a hypothetical example one may consider the graph in
Figure 1.

8
Paulo Garrido Question Based Learning in Computer

κ5

κ3 κ4

ν2

κ1 κ2

Knowledge body Κ Pre-requisite knowledge body Ν

Figure 1. Graph representation of a hypothetical learning description of a knowledge body.

In the example, the particular knowledge body K is described as having knowledge

domain K = {κ1 , κ 2 , κ3 , κ4 , κ5 } . The learning map makes the following assignment of
building sets for the elements of K:
κ1 6 ∅
κ2 6 ∅
κ 3 6 { κ1 , κ 2 }
κ4 6 {κ1 , κ 2 , ν 2 }
κ 5 6 {κ 3 , κ 4 }
A directed graph representation of a learning structure as in the figure is partial because
it makes no room for the indication of the specific learning process for each knowledge
molecule. With this restriction in mind, it is clearly useful as a basis for visually presenting
the description of the knowledge body. It also helps on reasoning about generating
presentation sequences and on checking conditions one wants or needs to impose on a
learning map as the consistency condition. This condition can be checked automatically by
inspecting the directed graph induced by a learning map LD for the existence of cycles.
Besides assigning to each knowledge molecule a building set, a learning map also
implicitly associates to each κ two other sets. One will call the support set of κ the set S(κ)
which elements are those knowledge molecules, which have a path to κ in the directed
graph representation of LD :
S (κ ) = {ν |there is a path from ν to κ } . (3.7)

One remarks that, with appropriate restrictions, if a learner knows κ, then he or she
should also be trusted to be in possession of all the knowledge molecules in S(κ).
One will call the continuation set of κ the set Cκ which elements are all the knowledge
molecules for which an edge goes to from κ in the directed graph representation of LD :
C κ = {ν |an arrow goes from κ to ν } (3.8)

Cκ is the set of all knowledge molecules which building sets have κ as an element.
If Bκ ∩ K = ∅ , that is if the building set of a molecule κ has no pre-requisite
molecules or nodes internal to K, then one will say that κ is initial in Κ. If Cκ ∩ K =∅ , the
continuation set of κ restricted to K is empty, then κ will be said terminal in Κ.

9
Paulo Garrido Question Based Learning in Computer

The natural way to state a learning structure D is extensionally as a set of triples. Each
triple lists a node and correlatives building set and learning process. So, one writes:

{ }
D( K ) = < κ 1 , Bκ1 , L κ1 >, < κ 2 , Bκ 2 , L κ 2 >,..., < κ N , Bκ N , L κ N > . (3.9)

3.2 Learning maps and learning sequences

Taking apart learning processes, the structure induced in a knowledge body by the type of
learning map described above amounts to organize the knowledge molecules in a DAG. A
presentation sequence of the nodes of the graph, which respects the “building set before
molecule” requirement, is a topological sort of the DAG. A DAG may have several
topological sorts. The example in Figure 1 has four sorts:
S1 =< κ1 , κ 2 , κ 3 , κ4 , κ5 >
S 2 =< κ1 , κ 2 , κ4 , κ3 , κ5 >
S3 =< κ 2 , κ1 , κ 3 , κ 4 , κ5 >
S4 =< κ 2 , κ1 , κ3 , κ 4 , κ5 >
Additional criteria may be devised for restricting the sorts to become learning
presentation scripts, for example a “bigger support cone first” criterion. Considering the
knowledge body in Figure 2, one sees that the following topological sorts are possible:
S1 =< κ1 , κ 2 , κ 3 , κ4 >
S 2 =< κ 3 , κ1 , κ 2 , κ 4 >
The support cone of κ2 is { κ1} whilst the support cone of κ3 is ∅. In principle,
learning κ2 (which implies learning κ1) takes longer than learning κ3. So, all other things
equal, when it takes to learning κ4, sort S1 allows a shorter interval between learning and
recalling κ2 than sort S2 does for κ3.

κ4
κ2 κ3
κ1

Figure 2. The support cones of the elements of the building set {κ2,κ3} have different cardinality.

It does not appear obligatory to use the facilities for defining a presentation script
provided by the m-system. A presentation script may be automatically chosen by the m-
system or the designer may specify the presentation sequence when introducing the m-
learning description. In this case the statement of the description in (3.9) would become:
D( K ) = < κ 1 , Bκ1 , L κ1 >, < κ 2 , Bκ 2 , L κ 2 >,..., < κ N , Bκ N , L κ N > , (3.10)

and the m-system engine would make the sequence of nodes < κ 1 , κ 2 ,..., κ N > the
presentation script, only checking the consistency of the introduced information.

10
Paulo Garrido Question Based Learning in Computer

A “building set before molecule” principle fixes behaviour for effective learning, but
should not limit8 the possible moves of the learner inside the DAG or the ways questions
are presented. It may be useful to allow the user to view the contents of nodes other than
the current node – the node which matters he or she is currently learning to give a passing
answer. More than that, a facility for the learner to revise nodes (back along the
presentation script or back along the support cone of some node) should be made available
in an m-course.
The mechanism for revising is also useful for presentation. Suppose a learner attains
some node in a session and then closes the session. According to the time elapsed till the
learner initiates a new session9, a revising of the nodes passed in the last session could be
made automatically, before presenting again the node attained. Revising the (passed) nodes
of the building set for the node to learn may be favourable in some situations and makes
for added flexibility of the presentation mechanism.
The general idea developed till now sees an m-system presenting a question as stimulus
for the learner immediately learn the molecule referred. It could be interesting to allow
variations in this basic strategy, by distinguishing sequences of presentation of questions
from sequence of nodes to learn. For example, one could present a question referring to a
node which the user is not prepared to learn, then its building set or any chosen questions
in the support cone till one gets to a node that the user is prepared to learn. This “going
back” through the questions may be put at use with potential pedagogical advantages, in
special if questions are designed with that intention in mind.
One advantage of the method indicated in the last example is to suggest, create or
enforce a sense of purpose or goal-orientedness in learning what is at the current node.
Other means can be put at use. For example, one can consider the following. The “building
set before molecule” principle implies that – terminal nodes excepted – a learner is always
in the situation of learning not only the current molecule but also a current building set (which
the current molecule belongs to) for another, more ‘advanced’ node10. At any point along
the course or the learning presentation script the user can be given information of the
current building set and of the more advance’ nodes, which one could term the goal node
for the current building set. This information can include which nodes the learner have
already passed and even be structured in more than one level.

4 The M-Learning Description

An m-learning description should not contain only a learning structure description that can
be stated as in (3.9) or as in (3.10). It must also contain information specific to each node11.
Formally, one can get to an m-learning description by interpreting the components
κ1 , κ 2 ,..., κ N in (3.9) or (3.10) as complex ones, rather than atomic. With this interpretation
one can write for an m-learning description, mD:

{ }
mD( K ) = < κ 1 , Bκ1 , L κ1 >, < κ 2 , Bκ 2 , L κ 2 >,..., < κ N , Bκ N , L κ N > , (4.1)

8 And in fact it is conceivable to configure an m-system engine to present the nodes in any order chosen
by the designer irrespective of building set considerations.
9 And so the probabilities of learning damage by natural forgetting.
10 In the frame of a DAG, it is possible to give a rigorous account of the notion of a node being more

advanced than other.

11 Although it may seem that the learning process is specific of the node and so should be treated, the

fact is that the definitions of a knowledge molecule at a node, respective building set and respective learning
process are all correlative. So it makes sense to state learning processes as components of the learning
structure description.

11
Paulo Garrido Question Based Learning in Computer

understanding that the nodes κ 1 , κ 2 ,..., κ N include a label L or description of the

knowledge molecule to be learnt, a set Q of questions, a set M of messages and a set G of
guiding directives, say for each node:
κ i = Lκ , Qκ , M κ , Gκ
i i i i
(4.2)

4.1 The label or description

Nodes refer to knowledge molecules. The label or description of the knowledge molecule
should be a convenient way of naming it, perhaps of adding a glance at the origin and
applications of it. It could even be a general scope question, which answer is not amenable
to ticking choices. But it should not have the role of a resource for learning.

4.2 The set of questions

In the form assumed for m-systems here, questions are to be written in a multiple sentence
format. To answer a question the user ticks one choice for every sentence of the question.
One will refer such a question as a multiple response question (MR-q). Despite possible
different variants, one will consider a MR-q, in abstract form, as a request for the learner to
assign truth and false values to the sentences in the question. Multiple response questions
are an extension of multiple answer questions, widely used in tests, where, typically, only
one sentence is true.
One will define the standard intention of a MR-q to test or promote the learning of some
knowledge molecule κ. As already said, one lets a knowledge molecule κ undefined. But
one will assume that a knowledge molecule κ may be represented by true sentences
s 1T (κ ), s 2T (κ ), … One will say that each s iT (κ ) expresses (eventually partially) κ . The
standard technique under a MR-q is to present one or more true sentences, together with some
false sentences s 1F (κ ), s 2F (κ ),... , which apparently could represent also κ . The possession
by a student of the knowledge molecule κ , is then evaluated from his or her assignment of
truth-values to the s 1T (κ ), s 2T (κ ),..., s 1F (κ ), s 2F (κ ), …
A node κ has a set of questions Qκ and the questions in the set will be called variants.
The reason for this term is that one intends that questions in the set are, in principle,
equivalent in an evaluation sense. This means that one assumes pragmatically that if a
learner gives a passing answer to any of the questions, then, as it would happen in a test,
one will judge that the learner did learn the knowledge molecule at the node, and one can
proceed accordingly. This also means that, although perfect equivalence of questions is
delicate matter, the designer should feel comfortable in choosing any question in the set for
evaluating the corresponding knowledge in a test or examination.
Having several questions for a node, rather than one only question, is advantageous in
several ways. When the node is presented first time, one question is chosen randomly by
the m-system engine or indicated by the designer. The learner is also given the learning
process, which can encompass experiments, readings, exercises or whatever the designer
indicates. Going through the process of learning can be combined with viewing other
questions of the same node. The user is free of submitting answers to any question of the
set. This facilitates that the learner becomes aware of her or his sensitivity to varying verbal
expressions of some knowledge, both in a warning way “what I do not know’” and in a
constructive way “what I know that I can apply to”.
Having several questions may also give sounder learning and evaluation, compensating
for bias or partiality of questions. Suppose a user is going a process of revising (some part
of) what has been learnt, i.e., nodes he or she has passed. One defines the revising process
so that each time a node is revised, the set of answerable questions will be the complement

12
Paulo Garrido Question Based Learning in Computer

of the set of questions already answered with success in that node. The confrontation of
the learner with different questions may spot weaknesses in learning or enhancing the
comprehension of matters. Of course this is not possible with one only question by node.

4.3 The set of messages

By messages one understands anything the designer wants to tell the user, be it in entering
or leaving a node, at the presentation of a question, or upon the submission by the user of
a specific answer pattern. They may embody felicitations, comments on the matter to be
learnt, indications for the learner suggested by an answer pattern submitted, or whatever
the designer may find useful.

4.4 The guiding directives

Guiding directives are mainly deemed as a means for making possible variations to the
presentation script defined. A form for guiding directives may be “if…then” rules:

If condition then action.

With guiding directives a designer may instruct the m-system to take actions adequate
to specific answers of the user, for example to initiate immediately a revising process of
some prerequisite knowledge, which the answer could indicate that is missing or
incomplete12. Or the m-system may put at focus a particular aspect of the matter being
learnt.

4.5 The building set

The case of a building set that contains molecules belonging to the domain of other
knowledge bodies must be taken in account. Let one suppose that Bκ = {κ a , κ b , κ c ,ν a } ,
with κ a , κ b , κ c belonging to the domain of knowledge body Κ, and νa belonging to the
{
domain of knowledge body Ν. Then, one can write Bκ = Qκa , Qκb , Qκc , I ( Ν : ν a ) , with }
I ( Ν : ν a ) being a descriptor of the molecule νa residing in Ν. What can be done upon the
information carried by the descriptor, if the need arises to assess the knowledge of the
learner about νa, depends on the m-system scope. An alert message may be issued advising
the learner to review the knowledge in question. On the other hand, questions about νa can
be posed, if the m-system includes, besides the m-description of Κ, an m-description of Ν.

4.6 The learning process

M-systems are neutral with respect to the definition of the learning processes the designer
assigns to molecules. A learning process may constitute of any combination of reading
texts, assisting presentations, making exercises, making simulations, making laboratory
experiments or whatever the designer finds useful. Computer materials for learning may be
embedded in the m-system or they may be available trough the Web.
Learning processes to use in an m-course may eventually be classified in a restrict
number which will ease its definition for the designer.

12One should not forget that the fact of the learner having given a passing answer means that the
corresponding knowledge molecule has been learnt in principle, not for sure.

13
Paulo Garrido Question Based Learning in Computer

5 The M-System Engine

For an m-system engine one understands a structure of software modules or computer
environment functioning at the most general level as an interface both for designer and
learner to the m-learning description including communication with the exterior.
Communication with the exterior allows an m-system engine to become also an interface
between teacher and learner. Such view is depicted in Figure 3.
Web contents of
designer / teacher

Learner

M-learning M-system
description engine

Designer / teacher

Web contents of
learner

Figure 3. General interface functions for an m-system engine. For Web contents one understands usage of
the Web relevant to the m-course.

For the learner using an m-system engine it will function as an “instructional

transaction shell”, as cited in [3]. It enables her to enter a learning interaction with
knowledge. For the designer it will allow to create, edit, visualise m-learning description.
And to create presentations and evaluation scripts for the course. It should also allow for
testing the course in the design and gather information from learners’ use in order to get
feedback for improving the course. An object-oriented analysis of an m-system engine
appears in Figure 4.
The analysis sees an m-system engine as objects communicating over a bus with
connections to Internet. The system accepts two types of users, designer/teacher or
learner. The functionalities and privileges of each user are enough different that the system
shell is spread between two objects: The dt-shell for designers or teachers and the l-shell
for learners. In fact they are enough different to justify 2 versions of m-systems: a
(complete) dt-version and a (learning only) l-version.
Interaction with an m-system is made through its shell. The functionalities available for
each type of user configure coordinated computerised interfaces with knowledge. In one
case the interface is of the teaching type and in the other it is of the learning type.

14
Paulo Garrido Question Based Learning in Computer

DAG M-description Teacher

M-learning Processor Manager Communications
description
DT-
Shell

User log(s) Evaluations Presentations Tester

Editor Editor

Evaluation Node Session Course

script(s) Presenter Monitor Manager

L-
Shell

Course
presentation(s) Reviser Evaluator Learner
Communications

Data-like objects Processor-like objects

Figure 4. An object oriented decomposition over a bus communication structure of an m-system engine.
Communication modules establish gateways with Internet. The arrow from Tester to Course Manager
expresses the possibility of the designer log in the system as a student and test design.

5.1 The designer/teacher interface

After logging in the system, a designer should have available at least the options of creating
or changing an m-description, creating or changing presentations, creating or changing
evaluations and testing a course or a piece of it. The m-description manager mediates the
creation and changing of m-descriptions. It acts as an interface to a database holding the
m-description and invokes the DAG (directed acyclic graph) processor to check data
consistency with respect to the learnability relation and for visualisation of the knowledge
structure.
One understands the DAG processor as a set of algorithms for checking that the data
in the m-description configures a DAG, for visualising it and for generating topological
sorts. Algorithms for performing these tasks are readily available, as in [2,5,9].
The presentations editor allows for generating topological sorts of the m-description
nodes, by invoking the DAG processor. These sorts are candidates to be selected or
changed by the designer to become presentations for the course. Desirable properties can
be built in the sorter.
It should be useful, although not mandatory, that a designer or teacher creates
evaluations or evaluations scripts for the learner to assess its capabilities in evaluation-like
conditions. Evaluations are fixed assignments of questions to be solved in a fixed time
interval. Evaluation scripts may be seen as evaluations with parameters, sensitive to user

15
Paulo Garrido Question Based Learning in Computer

profile and answers. The evaluations editor allows creation and management of evaluations
and evaluation scripts.
An m-course is an m-description together with presentations and evaluations run by an
m-engine. The tester object allows the designer interact immediately with its design by
allowing her to login as a learner and experiment the system from his point of view. This
means that a designer needs a complete version of the m-system engine.
The teacher communications module is sought to add value and functionality to the
assumed Internet connection and software available. It may help in organising questions
from learners, keeping track of answers, gathering data from the users logs of learners in
order that the teacher may assess the design and use of a course.

5.2 The learner interface

A learner enters an m-course through a registering session. From then on she has learning
sessions, which may be interspersed with evaluations. The course monitor takes in charge the
tasks necessary to follow the learner along to completion, logging the user, open and
closing sessions, showing progress and possibly providing general feedback for the learner
advance.
The session monitor manages the interactions between learner and system during
sessions. It registers the status of the learner (learning a new node, revising, evaluating) and
dispatches user choices, like going to a node, initiate revising or make an evaluation. It also
should take care of guiding directives that affect the user status. For this the session
monitor should invoke the node presenter, the reviser or the evaluator.
The node presenter introduces the learner to a knowledge molecule, displaying
questions of the node, the learning process and eventual feedback messages on no passing
answers. The reviser sets up a sequence of backtracking over the support cone of a node,
exploring all possibilities for learner failing. Such a sequence terminates when the learner
comes back to the departure node having passed every question presented. The evaluator
chooses a prepared evaluation or generates one on the basis of an evaluation script.
The learner communications object has a role analogous to the teacher communications. It
may help the learner in organising questions sent and answers received, communicate with
colleagues or participate in a forum about the course or prepare Web searches.

5.3 Common resources

The two possible versions of m-systems engines, the full mlt-system for designers and
teachers and the restricted ml-system for learners will share resources. At this point of
analysis the most obvious to appear is the DAG visualisation tool. But m-systems where
learners would be allowed to follow presentations generated under their command are
conceivable. Then the presentations editor and the DAG sorter would be included in the
ml-system.

6 Conclusion and lines of development

This paper as presented a proposal for conceptual foundations and a methodology for the
design of computer courses, to be named m-courses and provided by m-systems. It is the
belief of the author that m-courses have several characteristics that may improve the
efficiency of computer courses. M-courses may function on a stand-alone basis, integrated
in a learning group or as a complement to traditional courses with instructor’s physical
presence. By its concept they require an active learning attitude, specify what is to be learnt
at each step, enforce explicit knowledge domain components and their precedence, give

16
Paulo Garrido Question Based Learning in Computer

feedback to the learner about progress in several ways, unify studying and preparation for
evaluation and are flexible for designers13.
To implement prototype m-system engines with current software technology appears
not to present major problems. Assuming that such prototypes are available, the important
issue is the design of m-learning descriptions giving rise to m-courses. Such design is
crucial for the success of m-courses.

6.1 Designing m-courses

M-courses break with the classical attitude of centring teaching in expositions. This attitude
is fine as it allows converting one’s knowledge easily into texts, exercises or design of
laboratory experiments. In m-systems the coherent presentation of a knowledge domain
should be complemented with explicitly breaking the domain in manageable learning units
–knowledge molecules–, making clear the dependencies among units from a learning
perspective and providing for each unit a learning process and s set of questions assumed
to be able to test the learning of the unit.
The design of an m-course need not proceed from scratch. As a rule, teachers dispose
for their courses of materials as texts, exercises, laboratory experiments and a set of
questions for evaluation. The design of an m-course can proceed from this material. Either
by first taking the texts, exercises and laboratory experiments for an analysis conducing to
the definition of knowledge molecules, their dependencies and learning processes. And
then adding sets of questions for each molecule. Or by first taking the set of questions for
evaluation, analysing from it the diverse learning objectives of the course, the assumptions
made on how the students attain them and, on the grounds of the analysis, explicit the
knowledge domain in terms of molecules and the dependencies among molecules. These
two approaches are not opposite but complimentary and as a means to ease the eventually
necessary change of teaching attitudes. Compared with the traditional way of preparing
courses, the proposed methodology appears more demanding. Not only because of the
computer set-up, but in special, because it demands a deeper reflection on what is to be
learnt and how it is to be learnt. The reward may come in better pedagogical tools.

6.2 Lines of research and development

The central question to be solved by experimentation is how m-systems compare to other
computer aided learning systems in terms of learning efficiency versus cost14. Such
experimentation requires at least an actually functioning m-system engine and several m-
courses. Established techniques of assessment can be applied then. Test groups of students
can be made undergo m-systems and more traditional computer courses. Data can be
gathered then to compare the development costs, attractiveness, easy of use, rate of
learning and deep learning of both types of courses. This is an area for applying
experimental Psychology methods and that interests software developers of learning
systems.
If it turns out, as this author expects, that m-systems reveal potential to become an
established concept for teaching tools, then several lines of research are of interest.
M-systems are being proposed assuming that no natural language processing devices
are available for the task at hand. Questions or messages can not only be displayed, but also
spelt with voice synthesisers, which contributes to a more humanised interface. But as it
stands now, learner answers remain being given by choosing values of truth for sentences.

13 By all this, it seems very difficult to confuse m-systems with some sort of Frequently Asked Questions

(FAQ) document.
14 Actually this question raises a more fundamental aspect. How question-based teaching, in a maieutic

spirit, compare with other pedagogical methods? Prealable studies in this matter could be done in the absence
of implemented m-systems.

17
Paulo Garrido Question Based Learning in Computer

It would be of interest to study the usefulness of multiple sentence questions with

admissible fuzzy values of answer. For example, “almost true” besides “true”, “almost
false” besides “false”. This is of general interest in teaching and evaluation.
The functionalities for m-systems described allow several possibilities in giving
feedback to the learner during the learning process of a node, by choosing different
reaction patterns to the answers submitted. An information analysis of the reaction patterns
and relating them to learner culture seems important for efficiency.
M-systems may be understood as components for e-learning systems. A learner may
use an m-system in a pure autodidactic fashion, communicating with no one, and the
specifications given here respect that possibility. But m-systems are being conceived as
tools for bettering teaching and learning, not to suppress teachers. The natural cell for use
of an m-system is a group of learners following an m-course with support of a teacher..
Learners enrolled in the course beneficiate of asking questions to the teacher and having
answers from this. Other group techniques of learning may co-exist. The m-course may
even be a part of a course with lectures, tutorials laboratory experiments and evaluation. In
this case the m-course promises to be invaluable in providing a prepared path of learning
and immediate auto-assessment of the learner.
Most of the time a group of learners will be enrolled simultaneously in several courses.
If these are m-courses the natural cell for the group takes the aspect depicted in Figure 5.
Each learner has (at least) access to an individual computing facility running the learner
interface of the m-system engine and several m-courses. Each teacher has an individual
computer running the designer / teacher interface of the m-system engine and the m-
course she is lecturing. Teachers and learner are connected in a bus topology through
Internet. A server stores files and takes care of administrative and accountant tasks. For
realising such group learning, it seems necessary the application of computer
communications.
The possibility of aggregating and structuring m-systems in a network raises the
possibility of constituting m-knowledge bases, i.e., knowledge bases directed for learning. It
is also possible that m-systems may be built to be learning shells for existing knowledge
bases.
Learner 1 Learner 2 Learner N

m-courses m-courses m-courses

ml-system ml-system ml-system
Teacher 1

m-server m-course1
mlt-system

Teacher 2

ml-system : realises the learner interface of an m-system engine m-course2

mlt-system: realises the designer/teacher interface of an m-system mlt-system

Figure 5 Group of learners and teachers. Learners are provided with individual computing facilities to
follow m-courses. Teachers can observe, facilitate learning and gather elements for revising their courses.
The connection between the agents is assumed over Internet. A local server may be useful.

18
Paulo Garrido Question Based Learning in Computer

Acknowledgements
Pedro Pimenta for inspiration. Young people lead the way. Armando Machado for support
at a critical moment of doubt and other non-critical moments. Renato Morgado for long
conversations about how to solve teaching problems in a difficult context. The anonymous
people who supported me. Algoritmi Research Centre funded this research.

Appendix
The chemical metaphor proposed for knowledge and learning is useful, but must be taken
with due care. In a chemical reaction, the components of a synthesised molecule disappear
as individual entities. Differently, learning a knowledge molecule does not mean that the
learner looses what can be called the knowledge components of the molecule. The logic of
learning is not linear.
A building set for a molecule can contain components of the molecule, but it needs
not. For example, the learning process could specify to build a molecule, which would be
the negative of some pattern present in the building set. This shows that learning is not
reducible to one-step chemical reactions. The one-step knowledge increment operation is
conceptually wider than its chemical counterpart. This taken in account, the metaphor is
excellent and already used by [4] in the context of artificial intelligence. The constructive
aspects of language, for example, can be seen through a chemical lens and language is a
most common mean for conveying or describing knowledge.
The expression “knowledge molecule” captures the discretisable character of
knowledge but also suggests the active nature of a knowledge molecule. A knowledge
molecule is not a passive record, but an active entity able to react with other molecules for
the production of new knowledge.

References
[1] Carslon, R. (1994). Computer-adaptive testing: A shift in the evaluation paradigm. Journal of Educational
Technology Systems, 22, 213-224.
[2] di Battista, G., Eades, P., Tamassia, R. & Tollis, I. G. (1999). Graph Drawing: Algorithms for the
Visualisation of Graphs. Prentice-Hall.
[3] Duffy, T. & Jonassen, D. (1992). Constructivism and the technology of instruction: A conversation. Hillsdale,
NJ: Lawrence Erlbaum Associates.
[4] Gianantonio, P. D., Guglielmi, A. & Levi, G. (1993). Chemical logic programming? In P. Tarau (Ed.)
Proceedings of the ICLP'93 Post-Conference Workshop on Blackboard-Based Logic Programming, Budapest.
[5] Herman, I., Melançon, G. & Marshall, M. S. (2000). Graph Visualization and Navigation in
Information Visualization: A Survey. IEEE Transactions on Visualization and Computer Graphics, 6(1),
24-43.
[6] Joldus, M. (2002). Lecture Notes on Data Structures and Algorithms. Technical University of Cluj-
Napoca. (http://uiorean.cluj.astral.ro/cursuri/dsa//7_Sda.pdf).
[7] McInerny, R. (1963). A History of Western Philosophy. Jacques Maritain Center, University of Notre
Dame, IN. (http://www.nd.edu/Departments/Maritain/etext/hwp.htm).
[8] Schank, R. & Cleave, J. (1995). Natural Learning, Natural Teaching: Changing Human Memory. In J.
Singer (Ed.)^(Eds.), The Mind, The Brain and Complex Adaptive Systems. Reading, MA: Addison-Wesley.
[9] Sedgewick, R. (2002). Algorithms in C++. Boston, MA: Addison-Wesley.

19