Enhancing student engagement in multidisciplinary groups in Higher Education: A case study of a module taken by Computer Science, Electrical and Electronics Engineering and Mechatronics Engineering students at University of Northampton

Michael Opoku Agyeman, Senior Lecturer in Computing, FAST, UoN

Statement of the problem

Computer Science and Engineering students often disengage from modules because of excessive, irrelevant or forced examples that do not relate to real-world applications (Bédard, Lison et al. 2012). While some students are interested in hands-on experience, others prefer theory (Prince 2004). This is even exacerbated in cases where a curriculum/module from a particular discipline is taken by multidisciplinary groups of students. There is a higher risk of disengagement of student groups from one or more of the disciplines. Studies have shown a positive correlation between student engagement and improved academic performance (Trowler, Trowler 2010). However, there are a range of different points of views reported in literature about effective student engagement techniques.

Microprocessor Systems (CSY2015) is a Level 5 module taken by Computer Science, Electrical and Electronics Engineering and Mechatronics Engineering Students within the Department of Computing. It has been observed over the years that the engagement of the students within the first teaching block is low. Student engagement is however, better in the second teaching block, possibly because they realise the significance of engagement with the face-to-face sessions after the first teaching block.

A common issue is some students believe that other students are more skilled in some aspects. For instance, engineering students expect computing students to be better at programming, while computing students expect engineering students to be better at building circuits and mathematics. This belief however, inhibits the learning experience of the students and creates a disjoint in the engagement with some aspects of the module.

Over the years, there has been an imbalance in the number of Computing and Engineering students enrolled onto the module. Though the total number of students varies each year, normally, the ratio of the number of Computing to Engineering students is about 4:1. This can create a sense of being a minority group amongst the Engineering students.

Due to the disconnect between the significance of the module and the discipline/department in which it is delivered, some students do not see the direct relevance to their programme of study and career. Most Computing and Engineering dissertations in Level 6 depend on Microcontrollers (the core of CSY2015). However, in Level 5, students do not seem to see the relevance to their individual programme of study.

The module runs for 24 weeks with two summative assessments: an interim computer-based MCQ (marked at the end of first teaching block) and an end of module report (AS1) (marked at the end of second teaching block). Most students feel they can disengage with the face-to-face sessions, read a book or online resources and/or try their luck with multiple-choice questions and at least pass.

This case study evaluates the outcomes of an intervention aiming to enhance student engagement in this multidisciplinary module, starting from the beginning of the first teaching block.

Literature review

Many studies confirm that students’ engagement is essential to their success (Schmoker 2018, Eccles, Wigfield et al. 1998). According to Willis (1993), student engagement consists of academic engagement and institutional engagement. Similar to Renninger (2006), academic engagement is defined by Ketele (2000) as the ability allocate metacognitive, cognitive and affective resources to a learning task. This case study focuses on how to enhance academic engagement of the students. Various literature has identified problem-based learning, active learning, collaborative learning and cooperative learning techniques as effective in engaging students academically (Pirker, Riffnaller-Schiefer et al. 2014, Attle, Baker 2007, Burden, Byrd 1994, Freeman, Eddy et al. 2014) .

Teaching via problem-based learning technique normally involves putting students into small groups to work on tasks in an independent manner. Norman et al. (2016), based on several meta-analysis, highlighted that the small groups help improve the academic achievements while the independent element of problem-based teaching has some negative effects on student learning. Some literature on the application of problem-based learning in teaching medicine (Prince 2004, Albanese, Mitchell 1993) claim that there is some level of improvement in the clinical performance at the expense of exams which had negative correlation. Study by Bédard et al (2012), suggests that both medicine and engineering students find problem-based learning rigid and stressful (J. Reeves, Brian J. Arnold, Prince 2004, Bédard, Lison et al. 2012). In their study, students relate this rigidity to the requirement of having to work in groups and keeping up with the group.  The increase in stress levels were mostly related to the fear of failing a group work as an individual and resit. In fact, among the four determinants of students’ engagement identified by Bédard et al (2012) (self-efficacy, stress, new cognitive tasks, theories and beliefs about knowing), stress is the most predominant element of students’ engagement. Vernon et al. (1993) after investigating 35 studies between 1970 and 1992 concluded that problem-based learning improves students’ attitudes. Moreover, work by Albanese et al. also confirm that both students and tutors have positive attitude towards problem-based learning approach, while Norman et al (2016)  confirms that problem-based learning approach does not only challenge students but is also more exciting and motivating. In summary, problem-based learning has been extensively supported by literature to increase student engagement and attendance (Prince 2004, Albanese, Mitchell 1993, Qin, Johnson et al. 1995). However, strategies must be put in place to reduce the level of stress and possible negative effects on assessment results.

Active learning techniques involves engaging students through meaningful learning activities. A common approach is to occasionally (twice or thrice in an hour) break the lecture for students to discuss their notes in pairs. A study by Hartley et al. (1978) revealed that students’ attention and retention reduce drastically with the length of lecture. Interrupting lectures with relevant activities help refocus the minds of students and keep them engaged (Ruhl, Hughes et al. 1987, Prince 2004). Work done by Qin et al. (1995), suggest that cooperation is more effective in producing high quality individual problem solving compared to competition. Their conclusion however, is based on the findings that individuals in teams had better solutions to challenges compared to individuals working competitively. These findings do not necessarily conclude that students working in cooperative groups were more engaged or developed stronger transferrable skills or long-term problem-solving techniques. Problem-based learning helps resolve the issue of long lectures and students’ disengagement. The activities give the students the opportunity to collaborate and cooperate in groups. Due to technology advancement, online learning and/or blended learning, where synchronous and/or asynchronous technology enhanced strategies are used as active learning techniques, have become increasingly popular. As highlighted by Reeves (2015), depending on student’s learning preference, synchronous and asynchronous sessions have their advantages and disadvantages in students’ engagement. Some students feel safer to engage anonymously online while others engage more during face-to-face sessions. Reeves (2015) presented an interesting case of teaching hands-on laboratories online to multidisciplinary engineering students where the Virtual Learning Environment (VLE) was depended on extensively for student engagement. Their findings show that both asynchronous and synchronous sessions are useful in engaging students online. Kemp (2014) however, revealed that students feel more engaged with face-to-face discussions with peers and instructors compared with online, though they prefer to do the activities online.

Though the findings above may help in improving the students’ engagement of CSY2015, there are some associated challenges such as low grades and stress that need to be considered.

Preliminary study

In order to gain a deeper understanding of the student engagement issues and associated challenges from students’ perspective, a study was conducted in-class, with the experimental group (CSY2015 2018/19), using an adopted empathy map Ferreira et al. (2015) (Table 1 and Figure 1). This showed that, while most students want to “gain” good programming skills and good grades in order to graduate and get good jobs, they fear the “pain” of not having enough prior programming skills and failing. As shown in Figure 1, one student (“Student A”) suffers from the fear of anxiety and depression, and feels overwhelmed by the impact of the study responsibility involved with taking several modules at the same time. This was taken into consideration in implementing the interventions discussed in this case-study, the details of the additional advice and support provided to “Student A” is beyond the scope of this report (and hence not discussed). This study reveals also that, students see the relevance of the module to their future job opportunities. They want high grades; however, they do not have enough confidence in their own programming skills. Therefore, an intervention that gives students the opportunity to apply their theoretical knowledge to practical problems in order to help improve their programming skills while given them a feel of the work environment may be the right solution. Problem-based learning provides such an opportunity. Moreover, if given the opportunity to work with others, students do not only get to learn from each other but to also understand that they are not alone in some of the challenges encountered during the learning curve. A lesson for the tutor gained from the study is to not overly complicate the problem-based learning activities, as this may rather demotivate the students or hinder their learning. Also, the use of interim but flexible deadlines will help ease the level of anxiety on students with regards to not being able to complete the tasks in time.

Student Empathy Study

Table 1. Results of Student Empathy Study of CSY2

Sample from student empathy study

Figure 1. Sample from student empathy study

The intervention

One of the strategies was to use problem-based learning with various hands-on laboratory activities to encourage experiential learning where students learn by doing. CSY2015 involves physical computing where hardware (microcontroller) development tools are used. In order to help engage the students and to increase the relevance of the module to the individual disciplines, the most popular microcontroller programming hardware and software resources used across both engineering and computing students for the dissertations in Level 6 were selected for the activities of CSY2015.

Another idea was to make effective use of groups as a strategy to enhance student engagement. An adapted version of the 7 steps strategy proposed by Plymouth University is used and outlined below:

Set clear expectations
A key strategy was to help students become familiarised with each other during the early weeks of the module. The practical problems used in the early weeks were only formatively assessed, in order to encourage students to get engaged in group activities without the fear of risking grades. Students were encouraged to work in groups of 4 of their choice, or individually. They were given the flexibility to either stay in the same group or form different groups for each session. Their progress was monitored during class and through group discussions (and through the project journal on NILE).

Group size and composition
In the past, a variety of strategies have been used to form working student groups (4 per group) during the second teaching block, with varied effects. For example, students have previously been allocated as follows:

  • A mixture of Engineering and Computing students: It was observed that, engineering students either ended up dormant or relying too much on computing students for programming; while computing students relied on engineering students for circuit building and report writing. There was not much interdisciplinary learning involved. Student from each discipline mostly concentrated on what they felt they were good at. This may not be an issue in the first teaching block which emphasizes individual learning, however, interdisciplinary learning is essential in the second teaching block which emphasizes transferable skills and students’ ability to work in real-world industrial settings.
  • A mix of engaged (or promising) students and non-engaging (or at-risk) students: It was observed that non-engaging students depended too much on the “promising students” for the completion of the tasks, and this sometimes slowed down their learning curve.

In response to the above challenges, the new strategy was to help students form teams, rather than just groups, to enhance engagement. The early weeks, where students work individually or in larger groups, were used to monitor engaging and non-engaging students. This helped to identify students at risk early enough to help put the right intervention in place. Afterwards (around week 5-6), smaller groups, facilitated by the tutor, were formed. Engaged students who had made significant progress could either work alone, with other engaged students, or with students of their choice. Progress could easily be monitored by the quality of their contribution to the NILE journals.  Students with low engagement either got to work on their own or with other non-engaging students.

Though this strategy may seem like a bad idea, when it was implemented (in Week 7), the non-engaging students suddenly started showing some willingness to learn. They demonstrated a sense of responsibility to complete the tasks in their own creative way, knowing that they could not hide behind the mask of other students. Moreover, after the implementation of this strategy, classroom attendance, student participation and interaction with the tutor (asking for clarification and feedback) also improved greatly.

Encourage intercultural/interdisciplinary group work
In this context, the idea was to design the group activities in such a way that no particular discipline had a disciplinary advantage over the other. Thus, to encourage interdisciplinary group work, each task required building a circuit on a solderless breadboard and programming the circuit to work. This required both Engineering and Computing skills to collaborate without deviating from the learning outcomes of the module.

Innovative group work activities
As emphasized by Gibbs (1995) group activities should be of a complexity suitable for the collective effort and knowledge of all the group members. During the first few weeks where groups of 4 were used, students had to work collaboratively and creatively employ problem solving skills as well as subject knowledge to solve the problems. Groups were introduced into the early teaching weeks to encourage student engagement. Hence, the focus of the activities in this stage was to help students identify which group members they worked best with, and whether they engaged better with the module working alone or as a group. The goal here was not to use overly complex activities to ensure that each activity could be completed in within a week. This allowed student to form new groups in different weeks in order to find the best group members for them.

Managing the logistics of group work: monitor group activities and managing assessed group work
Using the group features on NILE, students can exchange files, create wikis, blogs, use journals and more importantly, these activities can easily be monitored and graded by the tutor. Students were advised to work within their groups and upload the findings of their completed task (by following a template) to their group page on the NILE site. Most students could not complete this during the class when it was first introduced, as they had no prior experience with the group tool on NILE. However, all the groups participated and additional time was given to complete the task outside the face-to-face session. Statistics reports from NILE showed a high level of student engagement with these activities both during and outside the face-to-face session.

Another strategy used was to assign a percentage (10%) of the assessment of the second teaching block (AS1) of the module to the problem-based activities, which are designed to map to the learning outcomes of the module, whilst ensuring that this remained true to the assessment as laid out in the module specification.

Peer Observation

Mr. Umair Tanveer observed an in-class delivery as well as the online contents of CSY2015. The observation occurred during teaching Week 8, two weeks after the implementation of the second stage of the intervention (use of smaller groups). The aim of the observation was to receive feedback on the suitability of the new mode of delivery of the module as well as the content of the NILE site activities for student engagement in the multidisciplinary context and level 5. The total number of students was 25 (23 male students and 2 female), including 8 engineer students.

I had three main dimensions of the implemented intervention which I wanted feedback as discussed below.

The effectiveness of the groups/teams in engaging students  

My goal was to create groups where students can either work effectively as teams or individually to actively engage in the learning process. The idea is to start with groups of 4 (to work on formative assessment) to help students familiarise each-other and eventually form a more focused groups of 2 to work towards summative elements of the assessment. My concern was that, the groups of 2 or students working individually will rather inhibits interdisciplinary learning as well as increase the chances of losing the other learning advantages associated with groups.

The complexity of the problem-based learning activities

The idea of using problem-based learning activities seems like a feasible approach. My concern however, was that the problems may be too complex or simple. To encourage students to engage in the module, the problems needed to be of a complexity appropriate for the level 5 students to complete within the set deadlines. Moreover, the problems should not only be interesting to each discipline but relevant to the learning outcomes.

The suitability of the NILE site in encouraging and monitoring student engagement

The idea was to use the features available in the group tool of NILE e.g. Wiki, journal, file exchange, collaborate etc to both motivate and monitor student engagement in the activities. In addition to making the activities available online for students to access outside the classroom, I have made several materials available on the NILE site, intended to give students extra help outside the face-to-face sessions. Therefore, I requested for the observer to evaluate how suitable the help files of the NILE site were for self-study. Moreover, I asked the observer to evaluate how the students input on the NILE tools, particularly the use of journal and file exchange, reflects on their level of engagement.

The observer was interested in evaluating the challenges associated with teaching multidisciplinary groups of students. He was also interested in how the problem-based teaching could be used to improve student engagement. Furthermore, the observer was interested in evaluating the challenges associated with simultaneously using small groups of 2 and students working alone as a student engagement strategy.

Observation outline

The observer attended a 2-hour face-to-face session of CSY2015 during week 8. Additionally, he was given full access to the module’s NILE site as an Instructor. Prior to the observation, we had a meeting where I gave a brief introduction to the activity for that week and also an overview of what the students have been doing in previous weeks. We explored the NILE site together and I explained the available teaching resources with particular emphasis on the group features and self-study materials. Bearing in mind the focus of the observation, the observer had the opportunity to perform the activities on his own and also to observe the students during the face-to-face session. The students were very positive about the practical approach to learning the module. They showed eagerness get involved in the practical activities and felt confident in asking for pointers from the tutor. By week 8, their focus had shifted from just reading about theory to how they could apply their skills and knowledge to solve practical problems. However, the students who did not engage in the early weeks of the module did raise concerns about the complexity of the activities. I arranged for the classroom assistant to spend extra time with those who were enthusiastic to reengage in the module to bring them up to speed.

The observer’s experience from a department which is different from mine and as a senior lecturer were resourceful in his appraisal of the effectiveness problem-based learning approach to engaging multidisciplinary groups of students. Feedback was given in a relaxed atmosphere over coffee. The informal nature of the meeting helped to encourage more constructive discussions.

Discussion of outcome of observation

The effectiveness of the groups/teams and the suitability of the NILE site in engaging students

Initially, the observer shared my concerns about the possible problems associated with assigning two people per team or letting students work individually as it is smaller than the ideal size reported in literature (Best, Kahn 2016, Keller 1986, Kitzinger 1995). However, after observation, he highlighted that based on his evaluation of the NILE content, assessment strategy and interaction with the students, this strategy rather motivated the students to engage in the module. With the help of the statistics tracking tool of NILE, he identified that the activities in the early weeks where students worked in larger groups had lower student access rate in contrast with when students worked in smaller groups. Also, the quality of work submitted by students when they worked in smaller groups had greatly improved. This could be because the activities conducted by the smaller groups were part of the summative assessment. Moreover, students were aware that though they could work in smaller teams, they would be assessed based on their individual contributions. The tasks were designed in such a way that students could work either as a team or individually. The NILE tool made it easy to identify students who have not engaged. Besides, if a student missed a session, he could catch up either on his own or with the help of the member and his progress could be tracked online. He reassured me that, this was an evidence that NILE was being used effectively to motivate and monitor student engagement.

Moreover, the observer praised the statistical evidence of the fact that non-engaging students started engaging when given the responsibility to work on their own or paired with similar students. He identified that this could be because the grouping helped both the tutor and the classroom assistant to give the right support for each group. He was particularly impressed with the learning balance that the groups helped to form. Fast learners who engaged with the module had the opportunity to work on more advanced problems while slow learners or non-engaging students got the opportunity to reengage and the support to catch up. Furthermore, he applauded that this showed an element of trust to the students that I had the confidence in their ability to complete the activities in order to meet the learning outcomes before the end of the module.

The complexity of the problem-based learning activities

The observer praised the amount of effort that has been made to towards the intervention of students’ engagement in the module. Specifically, he emphasized on the novelty used in embedding the theory and learning outcomes in the problems. He expressed his appreciation to the strategy used in given students feedback and related it to how it encouraged their learning and engagement. He identified that the problem-based approach gave the opportunity for tutors to give feedback and support to small groups. However, wherever an issue was common among teams or the whole class would benefit from a feedback, the feedback was given to the whole class and an open discussion was had. He appreciated how the students were given an opportunity to complete the tasks outside the classroom. He liked the idea of given students the opportunity to work on more advanced problems if they were ahead of the class. He highlighted that he found the time he spent on his own with one of the activities exciting. This gave students both synchronous and asynchronous learning advantage and to work at their own pace with reduced stress levels (Ruhl, Hughes et al. 1987, Prince 2004). He expressed that the NILE materials were enough to get him started and the more he engaged with the activities the more he wanted to see the code work on the physical platform. However, he raised concerns that though it was exciting after the problems were solved, it may be frustrating when the implemented solution does not work. In physical computing, students can easily see whether their solution worked or not. Hence, he recommended that in order to keep the majority of the students engaged, I should go through the help files (that I have made available on the NILE site) with the students during the face-to-face sessions.

Findings from preliminary study

In order to evaluate the impact of the intervention, effect sizes, a well trusted quantifying tool for evaluating the significance of an improvement, is used. The aim is to evaluate the effect of the intervention on the student engagement and academic achievement. The percentage difference between the number of students in 2018/19 (experimental group) and 2016/17 is 83.3% (see the table in Figure 2) which is too significant to give a fair comparison of effect sizes. Therefore, grades if students from 2017/18 which has only 15.8% difference in student numbers with the experimental group is selected as the control group. Besides, the time overlap between the two academic years makes it more suitable to be selected for quantitative evaluation.

Assessment results

Assessment results

Figure 2. Assessment (MCQ) Results of First Teaching Block of CSY2015

The average score was 64.4 in 2018/19, and 50 in 2017/18 with a standard deviation of 18.8. Hence the effect size is (64.4-50)/18.8 = 0.8. This implies that the grade of an average student in 2018/19 after the implementation of the intervention outperforms the grades of 79% of the students in the previous year. Effect sizes of 0.5 or higher (in this case 0.8) is much higher than most interventions reported in literature (Prince 2004). Moreover, it has been confirmed in literature that findings of effects sizes of 0.8 are rare as it demands significant gains (Prince 2004). This therefore confirms that the intervention has a significant impact on the students grades (Trowler, Trowler 2010) .

As shown in Figure 2, students’ grades in the MCQ test have greatly improved in 2018/19 compared to that of the previous two years. Though 2018/19 has the highest recorded number of students taking the test, 41% of the students had A while only 5% got F grade in contrast of the previous year which recorded 21% and 32% in A and F, respectively, with 19 students. A detailed look at the grades in Figure 3 reveals that 2018/19 recorded more A+ grades than any of the previous years. Also, the lowest grades have shifted from F- and F in the previous years to F+ which represented only 5% of the students. MCQ is computer accessed and not biased to the tutors marking. An interesting observation however, is that, the 5% of the students who got F is in fact a student who did not engage with the face-to-face sessions and only turned up on test date. The students’ engagement with the online content and “possibly” his peers outside the classroom might have however, contributed to the F+ instead of F or F-.

Detailed assessment results

Figure 3. Detailed Grades of Assessment (MCQ) Results of First Teaching Block of CSY2015

Prince (2004) and Albanese (1993) suggested a negative correlation between problem-based learning and test results. However, the results of the interventions prove that problem-based learning helps improves students’ grades. The intervention in this case-study promoted competition through rewards, however, this did not have a negative effect on the quality of individual problem solving as implied by Qin et al. (1995). It rather had a positive effect on the quality of individual problem solving, student engagement and grades. The improved engagement and grades after implementing engagement strategies in the face-to-face session agrees with Kemp’s work (Kemp 2014) which shows that students have higher engagement preference in face-to-face discussions.

Implications for future development and teaching practice

The intervention proved to be a feasible strategy for engaging students in CSY2015. The use of larger but variable groups to work on formative assessment, and eventually forming smaller groups to work on elements of the summative assessment, was a particularly good strategy in encouraging interdisciplinary learning, enhancing student engagement and giving effective feedback. The problem-based learning approach to teaching is effective in engaging students of varied abilities and disciplines.

Since the interim summative assessment is MCQ, it will be good to introduce MCQ-based knowledge checks on the NILE site. It will also be beneficial to highlight the related learning outcomes in the activity briefs. The MCQs should be designed to draw on the experience and understanding gained through the problem-based learning activities, without breaching the learning outcomes assessed in each item.

It will be interesting to get external parties, such as employers, involved. For instance, the problems or activities could be linked to particular industrial projects, and employers could be invited to give the briefing or help set the real-world scenarios. Labs and problems that cut across the disciplines could be explored.

References

ALBANESE, M.A. and MITCHELL, S., 1993. Problem-based learning: A review of literature on its outcomes and implementation issues. ACADEMIC MEDICINE-PHILADELPHIA-, 68, pp. 52.

ATTLE, S. and BAKER, B., 2007. Cooperative learning in a competitive environment: Classroom applications. International Journal of Teaching & Learning in Higher Education, 19(1).

BÉDARD, D., LISON, C., DALLE, D., CÔTÉ, D. and BOUTIN, N., 2012. Problem-based and project-based learning in engineering and medicine: determinants of students’ engagement and persistance. Interdisciplinary Journal of Problem-based Learning, 6(2), pp. 8.

BEST, J.W. and KAHN, J.V., 2016. Research in education. Pearson Education India.

BURDEN, P.R. and BYRD, D.M., 1994. Methods for effective teaching. Allyn and Bacon Needham Heights, MA.

DERMO, J., 2009. e‐Assessment and the student learning experience: A survey of student perceptions of e‐assessment. British Journal of Educational Technology, 40(2), pp. 203-214.

ECCLES, J.S., WIGFIELD, A. and SCHIEFELE, U., 1998. Motivation to succeed.

ENGLAND, R.E., MEIER, K.J. and FRAGA, L.R., 1988. Barriers to equal opportunity: Educational practices and minority students. Urban Affairs Quarterly, 23(4), pp. 635-646.

FERREIRA, B., SILVA, W., OLIVEIRA JR, E.A. and CONTE, T., 2015. Designing Personas with Empathy Map. SEKE 2015, pp. 501-505.

FINN, J.D. and VOELKL, K.E., 1993. School characteristics related to student engagement. The Journal of Negro Education, 62(3), pp. 249-268.

FREEMAN, S., EDDY, S.L., MCDONOUGH, M., SMITH, M.K., OKOROAFOR, N., JORDT, H. and WENDEROTH, M.P., 2014. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), pp. 8410-8415.

GIBBS, G., 1995a. Learning in groups: Tutor guide.

GIBBS, G., 1995b. Learning in Groups: Tutor Guide
. Oxford: Oxford Centre for Staff Development.

GIBBS, G., 2009. The assessment of group work: lessons from the literature. Assessment Standards Knowledge Exchange, , pp. 1-17.

HARTLEY, J. and DAVIES, I.K., 1978. Note‐taking: A critical review. Programmed learning and educational technology, 15(3), pp. 207-224.

HIDI, S. and RENNINGER, K.A., 2006. The four-phase model of interest development. Educational psychologist, 41(2), pp. 111-127.

J. REEVES and BRIAN J. ARNOLD, Applying student engagement techniques to multidisciplinary online engineering laboratories, ASEE Annual Conference and Exposition, 2015.

KELLER, R.T., 1986. Predictors of the performance of project groups in R & D organizations. Academy of management journal, 29(4), pp. 715-726.

KEMP, N. and GRIEVE, R., 2014. Face-to-face or face-to-screen? Undergraduates’ opinions and test performance in classroom vs. online learning. Frontiers in psychology, 5, pp. 1278.

KITZINGER, J., 1995. Qualitative research: introducing focus groups. Bmj, 311(7000), pp. 299-302.

LIVINGSTONE, D. and LYNCH, K., 2000. Group Project Work and Student-centred Active Learning: Two different experiences. Studies in Higher Education, 25(3), pp. 325-345.

MCCRACKEN, M., ALMSTRUM, V., DIAZ, D., GUZDIAL, M., HAGAN, D., KOLIKANT, Y.B., LAXER, C., THOMAS, L., UTTING, I. and WILUSZ, T., 2001. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students, Working group reports from ITiCSE on Innovation and technology in computer science education 2001, ACM, pp. 125-180.

NORMAN, G.R. and SCHMIDT, H.G., 2016. Revisiting ‘Effectiveness of problem‐based learning curricula: theory, practice and paper darts’. Medical education, 50(8), pp. 793-797.

PARNAS, D.L., 1990. Education for computing professionals. Computer, 23(1), pp. 17-22.

PIRKER, J., RIFFNALLER-SCHIEFER, M. and GÜTL, C., 2014. Motivational active learning: engaging university students in computer science education, Proceedings of the 2014 conference on Innovation & technology in computer science education 2014, ACM, pp. 297-302.

PIROT, L. and KETELE, D., 2000. L’engagement académique de l’étudiant comme facteur de réussite à l’université Étude exploratoire menée dans deux facultés contrastées. Revue des sciences de l’éducation, 26(2), pp. 367-394.

PLYMOUTH UNIVERSITY, 2017-last update, 7 Steps to: Using Group Work in Your Teaching. Available: https://www.plymouth.ac.uk/uploads/production/document/path/2/2398/7_steps_to_using_group_work_in_your_teaching_March_2013__1_.pdf [24/11/18], .

PRINCE, M., 2004. Does Active Learning Work? A Review of the Research. Journal of Engineering Education, 93(3), pp. 223-231.

QIN, Z., JOHNSON, D.W. and JOHNSON, R.T., 1995. Cooperative versus competitive efforts and problem solving. Review of educational Research, 65(2), pp. 129-143.

RUHL, K.L., HUGHES, C.A. and SCHLOSS, P.J., 1987. Using the pause procedure to enhance lecture recall. Teacher education and special education, 10(1), pp. 14-18.

SCHMOKER, M., 2018. Focus: Elevating the essentials to radically improve student learning. Ascd.

SEETHAMRAJU, R. and BORMAN, M., 2009. Influence of group formation choices on academic performance. Assessment & Evaluation in Higher Education, 34(1), pp. 31-40.

TALBOTT, E., FLEMING, J., KARABATSOS, G. and DOBRIA, L., 2011. Making Sense of Minority Student Identification in Special Education: School Context Matters. International Journal of Special Education, 26(3), pp. 150-170.

TROWLER, P. and TROWLER, V., 2010. Student engagement evidence summary.

VERNON, D.T. and BLAKE, R.L., 1993. No title. Does problem-based learning work? A meta-analysis of evaluative research., .

WILLIS, D., 1993. Academic involvement at university. Higher education, 25(2), pp. 133-150.