How I used the principles of software carpentry to teach a class on Epidemiology and Evidence based health

I argue here that it is possible to use software carpentry inspired teaching to teach evidence appraisal and clinical epidemiology. I will describe how I used a “software carpentry approach” to teach an undergraduate course on evidence-based health. I used one module in that class and converted a traditional lecture style teaching to a more interactive software carpentry approach to teach use of a web app (Gradepro, link here: Gradepro.org) how to critically appraise and develop evidence portfolio and summary of findings.

Epidemiology is an applied discipline where the teaching focuses on the application of theory to solve real-world problems rather than using dry didactic exposition. Besides, in the critical appraisal of research and in clinical epidemiology, we use computers and computer programming. For example, we use Epi Info, or other web based calculators for conducting data analysis in Epidemiology. A number of modules in R, or Stata are available to conduct epidemiological calculations. Despite this, formal programmes and classes in epidemiology and evidence based health do not necessarily cover the details of working with these programmes, nor do they cover issues around using computer programmes. Yet, for day to day work, students or professionals who will have to work with evidence or appraisal of evidence will need to work with software tools such as Epi Info, or Revman, or Gradepro.

In this presentation, I will narrate my experience of converting a class on critical appraisal of evidence using gradepro from a traditional class to one that was based on or shaped on or inspired by software carpentry. In 2016, I was teaching critical appraisal of evidence using Gradepro. The task involved the following:

1. The students were provided a set of questions.
2. For each question, the students were asked to select five primary studies that would address the outcome in the question. For example, a question might be “What is the effectiveness of Nicotine replacement therapy for smoking cessation?”
3. The students would identify an outcome of their choice and five primary studies that would best address that outcome and appraise the evidence presented in these articles for that outcome and intervention pair.
4. The students would use the GRADE approach and use GRADEPro webapp to abstract information from the studies, appraise the level of evidence presented in the studies and would construct an evidence portfolio and a summary of findings for the studies combined.
5. Construction of this evidence portfolio would involve that the students would have to construct a summary of findings based on the effect estimate and effect size, the 95% confidence interval band, and depending on the type of effect estimate (absolute effect or relative risk), comment on the use of each effect estimate in terms of their magnitude, the quality of the evidence (experimental studies would be awarded a higher grade of evidence than observational epidemiological studies). This process would take about two hours for a body of five primary studies and the process would involve understanding of the study designs but also that hte students would be facile working with the GRADEPro programme.

In the first few years I taught this programme and this module, I used a standard lecture format. I would lecture the students on GRADE, assign them readings on GRADE and construction of evidence, then in the class, I would demonstrate how to use the GRADEPro webapp to start with a study question and work through the process of critical appraisal of evidence. The students would then be expected to replicate the processes and steps that I showed them in their own computers using another set of problem questions. The report generated from the GRADEpro software would then be used for writing their assignment on critical appraisal of a body of evidence for an outcome. Till 2016, my approach was that, I would demonstrate the process first, or hold a tutorial class where I would show them the process again. Some students would bring a computer or a notebook to the class and replicated the process with me using either their problems or my problem and then reproduced the steps in their assignment.

I used to find that this process was not working well. Specifically,

1. The students would complain that the webapp GRADEPro was not easy to work with and was a bloated software as it would take several minutes to hours to boot up and store data locally, and it was not very intuitive to understand if they were left to themselves to work with it.
2. While the students seem to understand how to do the critical appraisal using GRADEPro during demonstration and I even used a webinar I conducted with an international agency to supplement my teaching materials, yet, when it came to provide feedbacks on this module, the students would complain that it was tedious and the quality of the final output of their work was not very satisfactory. I was confused. Some students seemed to understand the theory well, but when it came to produce the results, the results were poor. Most students would get a B or lower grade in this assignment and that was perhaps due to their unfamiliarity with the software. Many students had also problems interpreting data results from individual articles they would select, and this also added to the complexity.
3. Third, students would set up appointments with me so that I would have to go one on one training with the students to use this tool. This used up considerable time for me, and I frequently found several students would wait outside my room and I had to work several hours with the students teaching them how to work with the tool and how to conduct critical appraisal using GRADEpro, despite showing them in the class and sharing teaching and study materials.
4. Several students would ask for extension and report that they were unsure as to how to work with the assignment and use Gradepro despite several rounds of working with them. Several of them reported assignment anxiety.

These suggested that the system of teaching them how to use GRADE pro to conduct a critical appraisal and grading exercise was not working to the best advantage of the students. I had to find another way to do the teaching. Searching for a solution, I came across an article by Greg Wilson, titled, “Software Carpentry: Lessons learned. In this article, Wilson outlined the steps and processes of teaching principles of software tools such as basic terminal commands, using R, Python, and other programming languages. The courses and the teaching would be directed to scientists or students who were not familiar with these principles even though they had to work with software programmes on a day to day basis and the following would be the principles:

1. A short two-day intense programme of hands-on teaching would be preferable to five day or longer workshop format of teaching. This two day teaching would be workshop format anyway.
2. The purpose would be to teach computational thinking rather than the nitty gritty of working with programming languages or BASH scripting or Python or R (these were all open source software that would run in any operating system).
3. The instructor and the students would work together at the same time on the same problem or the same programming challenge; this would slow down the instructor. This mode of teaching was referred to as “live coding”. Live coding not only slowed down the instructor but it also made the teaching more interactive. More importantly, the students would learn from the mistakes even the instructor could make.
4. Such a mode of teaching would encourage the students to help each other and form peer teachings; for example, if the students were to work in pairs using a single computer, then one student could help another student as they worked on the problem and would take turns.
5. A system of repeated feedbacks were introduced. At each gap, the students were encouraged to leave feedbacks and their learning needs for the subsequent session. This enabled the instructor to tailor instructions.
6. At different points in the lecture or instruction or working with the students, the instructor would stop and check the level of understanding of the students. This provided an opportunity for the instructor to assess how much of the instruction got transmitted.

I used these principles and transformed my module on teaching critical appraisal and gradepro class after the inspiration of software carpentry principles. Gradepro was a free app, although it was not open sourced, it was based on javascript and provided a free web based app for the students to install on their browsers: it was available for all the different types of browsers except for Internet explorer 6 or later. Most students used Firefox or Google Chrome; I tried the app on Vivaldi and Safari for Mac, it worked. Gradepro also comes with extensive documentation and uses and links to the GRADE working group website that has posted tutorials and documentation on the use of the GRADE approach and use with GRADEpro, besides, it has use cases and videos on the usage of this app.

I converted my previous course in the following manner:

1. I changed the seating of the students from a more “stadium style seating” to a more “dinner table style seating”; now the students sat around a table
2. Each student was asked to bring a computer (not a tablet or a chromebook, although this would work on a tablet or a chromebook). If a student did not have a computer, one would be arranged for the student as a loan laptop borrowed from the IT department.
3. A problem was distributed to the students and detailed instructions on how to use GRADEPro and the GRADE approach readings were distributed a week in advance to the students. They were expected to read this before coming to the class. In this practice, it was similar to what was done in the previous years.
4. I would work with the students on the same problem.
5. Each student was provided with a red and a green sticky note or post it note. If the student completed the task assigned to him or her during the interactive session, the student would stick the note on the back of their laptop screen (or the lid); if the student was stuck for some reason or did not progress or could not understand, then the student would stick a red sticky note on the back of the computer. I would then troubleshoot. This was done as I did not have a teaching assistant with me in the class.
6. From time to time, I would explain a key topic and ask a question. The students raised their hands and answered the questions. However, it was different from a typical software carpentry session in the sense that in a software carpentry workshop, we use a webpage hackmd.io or etherpad; in our case, we just used ora question and answers.

The class met twice for two-hour sessions in two consequent weeks. The students worked with me in a topic I worked with them. There were questions and some students ran into problems for the app installation. Several students did not understand concepts about relative risk and odds ratios, and in assessment of quality of evidence; these doubts were clarified. Some other students ran into problems working on some aspects of GRADEPro. When they put up their red sticky notes, their friends and neighbouring students would help them, in this way, although I did not have helpers in the classroom, other students filled in their position and the class grew into a community of practice of sorts. At various points during the class, I obtained feedbacks from the students.

Following the class, the students were to work on their own problem sets using Gradepro and turn in their assignments. All students turned in their assignments on time and no one asked for extension. Informal feedbacks from the student obtained through the University’s system of post course feedback suggested that the students liked this new form of interactive teaching. Several students wrote personal emails to me expressing the idea that they liked this format of teaching and asked if I were to offer similar classes and tutorials. Overall the student satisfaction was higher than in the previous years. The quality of the assignments turned in was better than the previous years and the average grade for this component of the assignment also was better (more B or B+s compared with B-s and Bs in the previous years). These informal assessment of the impact of the teaching suggested that this mode worked for this class format.

I have subsequently converted a postgraduate research methods course using the principles of software carpentry where I have taught R for statistical computing for health sciences, and doing data science, critical thinking in health research along the principles of software and data carpentry. The course is ongoing but the initial results and informal feedbacks from the students suggest that this method of teaching is well accepted by the students.