The Use of E-learning Tools in a Basic Logic Course During the COVID-19 Lockdown

. This paper describes a study of the development and use of e-learning tools in the context of a basic logic course that was taught during the COVID-19 lockdown in 2021. The tools were used to compensate for learning and teaching limitations brought about by the lockdown. We compare a course that was taught in February and March 2020 (before the lockdown in Denmark) with a similar course that was taught in 2021 during the lockdown. In terms of exam results, the students from the 2021 course did signiﬁcantly better than those in the 2020 course. This paper considers possible explanations for this difference. Among other things, we analyse the data collected from the students via a questionnaire.


Introduction
Since the 1980s, various versions of a basic logic and argumentation course have been offered to students studying communication and digital media at Aalborg University, Denmark.This course emphasises the study of the logical validity of arguments from basic propositional logic and Aristotelian syllogistics.One important learning goal in this context is that students should obtain the needed skills to analyse an arbitrary propositional or syllogistic argument that is formulated in natural language in terms of symbolic logic to evaluate its logical validity.For this purpose, students should use truth tables and semantic trees to analyse propositional arguments and Venn diagrams and basic inference rules to analyse syllogistic arguments.
One of the authors of this paper (Peter Øhrstrøm) has taught versions of the course throughout this entire period from the 1980s to 2021, whereas two of the other authors (Thomas Ploug and David Jakobsen) have only been involved as teachers for a few years.A joint textbook [3] was used for the course, along with two learning tools, Syllog and Proplog, which were developed by Ulrik Sandborg-Petersen specifically for this course.These tools were employed during logic exercises to make the learning experience gamelike and enjoyable (see [4]).The interface of the present version of the Syllog tool is shown in Fig. 1.Interface of the Syllog tool.Note that in the case of a valid syllogism, the system will give the classical medieval name of the argument (see [2] and [3]).The student may compare these names with the Aristotelian theory presented during the course lectures.The interface of the Proplog tool is similar.In both Syllog and Proplog, valid and invalid arguments occur with the same frequency.
The Syllog user can click on 'New syllogism' to get a new syllogism presented on the screen.The user must then decide whether the syllogism presented is valid or invalid (i.e.whether the conclusion follows the premise in any possible/thinkable scenario).Proplog works in a similar manner with a similar interface.See [3] for more information on the course.
In both 2020 and 2021, Syllog and Proplog were not only used along with the teaching during the courses, but it was also used to establish the individual exams after the courses.The students were asked to produce 15-20 random propositional and syllogistic arguments using Syllog and Proplog and then follow with analysing and discussing the arguments to document their validity or invalidity carefully.Ultimately, students of a basic logic course should not only be able to identify a valid/invalid argument but should also be able to understand and explain why a particular argument is valid/invalid.
The 2020 exam results revealed that several students needed a clearer understanding of what it takes to demonstrate that a syllogistic argument is valid, and they appeared to have an even weaker understanding of how to show that a syllogistic argument is invalid.
In 2020, the course's teachers had to invest significant energy into presenting and training students on the use of Venn diagrams as well as basic inference rules in syllogistic reasoning because many students seem to find these topics rather difficult.This was done in lectures and traditional exercises to be carried out and discussed in groups of 2-3 students under the teacher's supervision.For the 2021 version of the course, the COVID-19 lockdown made it unrealistic to apply a strategy involving exercises in small groups under the supervision of a teacher.Instead, two learning tools, Proof and Venn, were developed and used.The tools should make it possible to obtain the same knowledge and skills alone while only using a personal computer.The Proof tool should support the user in the construction of a proof documenting the validity of a syllogism, and the Venn tool should support the documentation of the syllogisms' invalidity.
In Sects. 2 and 3, we discuss the design of the two tools and their theoretical background as well as the practical use of the tools in the course.In Sect.4, the results of the exams in 2020 and 2021 are considered and compared.Section 5 focuses on the results from a questionnaire designed to study the students' learning experiences during the course.Finally, in the conclusion, we offer some perspectives on possible compensation from the digital tools and teaching problems that can arise if teachers are limited to online communication with their students.The conclusion also answers some open questions and suggests interesting topics for further research.

The Proof Learning Tool
The Proof tool makes it possible for the user to construct indirect syllogistic proofs using a formal language involving four types of syllogistic propositions: Additionally, we may reduce the number of proposition types if we allow for the negation of terms because the following may be argued: Using the Proof tool, the user may be able to reason within the framework of a modern version of classical Aristotelian syllogistics (see [1,2]).The three straightforward inference rules mentioned in Fig. 2 (i.e.TRANS, MUT and EX) are available.TRANS can be presented as: MUT can be presented as: By contraposition and renaming we obtain: e(X, Y) → e(Y, X).
EX can be presented as: As shown in Fig. 2, the user may prove the validity of a syllogism with two syllogistic premises and one syllogistic conclusion indirectly, demonstrating that the assumption of the premises along with the negated conclusion will lead to a contradiction.The present example shows that the combination of the propositions o(M, P), a(M, S) and a(S, P) leads to a contradiction.This follows the inference rule TRANS.Thus, we have demonstrated the validity of the syllogism from the premises o(M, P) and a(M, S) to the conclusion o(S, P), which is the negation of a(S, P).It is well known that any valid syllogism formulated in this classical way may be proved in this manner.
Notably, the user may play with the various possible applications of the inference rules to obtain practical experiences with what it means to prove something in syllogistic reasoning.The student may carry out this kind of playful and game-like activity alone, using their computer as a kind of dialogue partner.This possibility became quite relevant for learning during the COVID-19 lockdown.However, it seems that such tools may also be useful when the learning situation is more 'normal' because at least some students would like to study alone, even if traditional group work under supervision is offered.

The Venn Learning Tool
The Venn learning tool allows the user to construct a Venn diagram designed to show that a certain syllogism is invalid.The diagram corresponds to a set of elements belonging to some universe of discourse.Seven subsets of this basic set are parts of the basic set.The user makes the diagram online using ' +' to indicate that a particular subset is non-empty and '−' to indicate that the subset is empty.
It should be mentioned that while John Venn (1834-1923) suggested another way of marking the subsets in the diagram (see [6]), we find his original approach inadequate as a basis for building a practical and interactive digital tool.For this reason, we chose a more useful but still equivalent method for constructing the Venn tool: -It should be noted that this kind of diagrammatical reasoning makes it possible to reason on the basis of incomplete knowledge -something which was in fact very important to John Venn himself (see [6]).
In the beginning all the subsets in the diagram are marked with '?', because it is as a start unknown whether or not they contain elements.Then the user can add the information that some of the subsets are empty (marked as '−'), whereas others are non-empty (marked as ' +').When at least some of the subsets (or some of them) have been marked with ' +' or '−', we may evaluate any syllogistic proposition (a, i, e or o) based on M, P and S as 'true', 'false' or 'unknown' relative to the diagram.This makes it possible to look for a diagram that documents that a certain syllogism is invalid (i.e. it has true premises and a false conclusion relative to the diagram).
In the example shown in Fig. 3, the propositions i(M, P) and e(S, M) both turn out to be true, whereas e(S,P) is clearly false given the signs indicated in the diagram.This means that this Venn diagram documents the invalidity of the syllogism from the premises i(M, P) and e(S, M) to the conclusion e(S, P).It is well known that if a classical syllogism is invalid, then there is a Venn diagram according to which the premises are true and the conclusion is false; the student just has to find it.
As with the learning tool Proof, this approach to Venn diagrams can easily lead to a playful and game-like activity that the students can carry out alone, with their computers as their only counterpart.

Exam Comparisons
The exams in 2020 and 2021 were organised in almost the same way, making it interesting to compare the results.The course lectures in propositional logic and syllogistics were offered to second-year students studying communication and digital media at Aalborg University in Aalborg and Copenhagen during February and the first week of March 2020 (i.e., before the COVID-19 lockdown).In 2020, 121 students participated in the exam.In 2021, all the lectures were given during the COVID-19 lockdown, and 132 students participated in the exam.After the evaluation of the exam, each student received a written statement explaining how well they had done.These statements were rather formal and brief; thus, they were easily categorised as 'Good', 'Acceptable' and 'Weak'.The results of the exams in 2020 and 2021 are shown in Table 1.
Table 1.A chi-square test of independence showed that there was a significant association between year and result.The chi-square statistic (2, N = 253) is 6.292, p-value = 0.0430.The result is significant at p < 0.05 in favour of the year 2021.Importantly, all assignments were evaluated by the same person, making the comparison of the results from the exams in 2020 and 2021 extremely reliable.

Good
It is interesting that the learning results were significantly better in 2021 during the lockdown than those obtained in 2020, when teaching occurred before the COVID-19 lockdown.Multiple factors may have contributed to this difference in performance.We conducted a brief focus group interview with 8 students from the Copenhagen group.
A consensus was reached on three interrelated explanatory factors, namely the uninterrupted time spent on doing logic exercises in direct continuation of classes.The students stressed the importance for their learning process of being able to do exercises in direct continuation of classes without being interrupted by fellow classmates.They estimated that they ended up having spent more time doing logic exercises than they would otherwise have done, due to the unimpeded time right after the logic classes and the readily available exercises.The tools Venn and Proof played a key part in this change in the students' study behaviour, as they provided the students with platforms for conducting logic exercises.The changes in study behaviour partly enabled by E-learning tools indicates that the serious challenges to traditional university teaching caused by the COVID-19 lockdown can be overcome.

The Questionnaire
A group of 84 students (the Aalborg group) was asked to complete a questionnaire.Unfortunately, only 40 responded.For this reason, we should use these results with some caution.On the other hand, they may give some indication of the situation, and we may still use the results as some sort of descriptive statistics.In particular, the students' answers to two of the questions should be noted, because they relate directly to the two tools we have introduced during the COVID-19 lockdown.Table 2 refers to the Proof tool: Although around half of the students who have responded don't think that the use of the tool has given them a better understanding of syllogistic validity, there is still around 40% of the students who find that the tool has helped them in this regard.Furthermore, the Proof tool is very new, and its user interface may not be fully satisfactory.It should also be noted that its use of indirect proof may be a complication.As a consequence, some of the students may not fully have understood what precisely the tool does.In fact, this may be the reason why the majority of the students were unable to benefit from the use of the Proof tool.Table 3 refers to the Venn tool, and it appears that the use of this tool has been helpful to an even bigger group of the students working with the course material.It appears from Table 3 that 52.5% of the students stated that they could benefit to some extent from the use of Venn.Graphically, this may be illustrated in the following manner (Fig. 4): The difference between the numbers in Tables 2 and 3 is in fact rather close to what we expected.At least it is fully consistent with our impression during the 2020 course and in particular during the 2020 exam, according to which many of the students found it relatively easy to prove the validity of a syllogism, whereas they felt more uncertainty about making a Venn diagram, by which we may document that a syllogism is invalid.

Conclusion
The finding that the 2021 students did significantly better on the logic exam than the 2020 students is a bit surprising considering that the 2021 students had more challenges due to the COVID-19 lockdown.One possible explanation may be that the students who were locked down were rather isolated and had few social distractions, which may have motivated them to study more.While likely part of the explanation, it is probably not significant enough to explain the notable difference between the exam results in 2020 and 2021.However, it is quite likely that the better exam results in 2021 were due to the introduction of the new learning tools Proof and Venn, the latter in particular.It is difficult to draw definite conclusions due to the low number of responses, but the answers seem to lend some support to the impact of the tools, with around half the students answering that the use of the tools led to a deeper and better understanding of the validity and invalidity of syllogisms.
In conclusion, this study clearly documents that, at least in some cases, when traditional teaching procedures and methods are or must be abandoned, being creative and innovative may allow us to design new teaching tools that can give rise to significantly better learning results than those occurring before the teaching situation changed.The outcome of an innovation sparked by change may even be useful and relevant when the normal teaching situation has been re-established.Thus, it will be meaningful to take advantage of all innovative ideas that emerge during a special and changed teaching situation, such as the one generated by the COVID-19 lockdown.As indicated by the present case, it will be relevant to create tools that can support students' work with propositional arguments, even if such tools have to be designed and developed after the COVID-19 lockdown.The analysis of the propositional arguments shows that many students lack a clear understanding of how propositional validity and invalidity can be demonstrated.This indicates that there is a need for an even stronger emphasis on the analysis of arguments in terms of truth tables and semantic trees.We may even consider building a tool based on the so-called existential graphs suggested by C.S. Peirce (see [7, pp. 165-181]).
The material for our basic logic course has been very much inspired by the works of A.N. Prior (1914-69).In particular, we have found his Formal Logic [8] from 1955 very useful.(More on the tools and topics chosen for the course can be found on our site on Prior's basic ideas on logic, www.logic.aau.dk).However, it is still an open question how many of Prior's ideas should go into our basic logic course.
No matter what, we should consider the development of tools for the part of the course material dealing with propositional logic as well.Such new tools should be carefully tested and evaluated.It is very likely that the use of such tools may improve the learning results even more, whether or not there is a lockdown.

Fig. 1 .
Fig.1.Interface of the Syllog tool.Note that in the case of a valid syllogism, the system will give the classical medieval name of the argument (see[2] and[3]).The student may compare these names with the Aristotelian theory presented during the course lectures.The interface of the Proplog tool is similar.In both Syllog and Proplog, valid and invalid arguments occur with the same frequency.

Fig. 2 .
Fig. 2. Interface of the Proof tool.Note that the user can apply three different inference rules, TRANS, MUT and EX, to obtain a contradiction in order to produce an indirect proof.It is well known that all classical syllogisms can be proved in this way.

Fig. 3 .
Fig. 3. Interface of the Venn tool.Note that the user can change the indications regarding the seven subsets in the diagram from '?' to ' +' or '−'.Then, the user can evaluate the propositions involved in the syllogism in question.If the premises are true and the conclusion is false, then we have documented the invalidity of the syllogism.-Itshould be noted that this kind of diagrammatical reasoning makes it possible to reason on the basis of incomplete knowledge -something which was in fact very important to John Venn himself (see[6]).

Fig. 4 .
Fig. 4. Graphical illustration of the results included in Table 3.The students were asked to consider the following statement: 'Venn has given me a better understanding of what it means to document the invalidity of a syllogistic argument with a Venn diagram'

Table 2 .
The students were asked to consider the following statement: 'Proof has given me a better understanding of what it means to document the validity of a syllogistic argument with a direct proof'.

Table 3 .
The students were asked to consider the following statement: 'Venn has given me a better understanding of what it means to document the invalidity of a syllogistic argument with a Venn diagram'.