Principles of teaching in the Carpentries and those that can switch over to regular classroom teaching as well

…. where I write the key lessons I learned teaching The Carpentries instructor training and you too can use these in your own teaching

I learned a lot over the last three years being in The Carpentries community and teaching The Carpentries instructor training workshops. I have learned how to run workshops, how to teach university semester level classes and a lot more about ‘pedagogy’ (the science and art of teaching children but also adults). This is a good time to write them down. I’d love to hear from you about your own experience.

A good starter about the Carpentries is to learn about what they are. Greg Wilson and Brent Gorda cointed The term “The Carpentries” for enabling researchers to do “computational thinking” and use tools of computation effectively. You can learn more about The Carpentries on their site and explore and join. I am not going to cover the details as this post is about my lessons in being a Carpentries trainer, but I am sure you can pick up the vibes if you work through the documentation on The Carpentries at their site:

These are a set of my lessons in no particular order, I have noted as they come along. I recommend you to read two core books:

Here are my takeaways:

Be where your learners are. — I’d like to start with this. In the carpentries style of teaching, you should be seen where your learners are. This means you will be _physically_ with the learners in the same room, you travel to where the learners are situated, or if you are organising a learning context using say Zoom or an online environment, you will be with the learners. It is the same thing with the classroom as well. Your presence, and your active work with the learners where you are by their side is a salient feature. This is same when you port your carpentries style of teaching in a university or school classroom setting. Be with the students and be one with them. This is the spirit of ‘guide by the side’.

Start with a learner persona. — When you plan a class or a carpentry session, think of the people who will take your class. For a university class, think of the student profile you get to see (this gets a little easier). Construct the persona on the basis of demographics (age, gender, social status, etc), and prior learning experience, work challenges. Say the following is an example of a learner persona:

Chen is doing post-doc in Epidemiology and is keen to learn how to use R in his data analysis. He has earlier used spreadsheet and a GUI software for statistical data analysis. Chen works with data sets and needs to organise data and conduct statistical data analysis

Note how the learner persona is constructed:

  • Chen is the name of a person
  • post-doc in Epidemiology tells you that Chen has significant background knowledge in the domain of Epidemiology
  • That Chen wants to learn R in depth tells you that when teaching Chens in your class on R, you need to pay attention to provide relevant content
  • That Chen has worked with GUI based stats software tells you that you will need to transition Chen from a GUI based data analysis to R’s style, so go slow

Construct a concept map of the topic(s) you want to teach before proceeding. — This is really important. One of the things we do in the carpentries is to follow the Dreyfus Model or discuss Patricia Benner’s research on nursing as to how people can move from a state of novicehood to being competent practitioners. Before we get into the nitty gritty of this details, let’s take a look at the Dreyfus Model first and then Patricia Benner’s research. Then we will review the role of mastery rubric, and finally come back and take a look at the role of concept maps in that order.

Review of Dreyfus’ skill acquisition model. — In 1980, the US Armed forces funded Stuart and Hubert Dreyfus to characterise a model of how people acquire skills [1]. Dreyfus’ based their observations on the performance of real life individuals and not artificial experiments that was vogue at the time, as they claimed that people alter their behaviours when they are aware that they are watched. In any case, they came up with a theory that when people acquire complex skills starting with no experience and they provided illustrative examples of people acquiring skills in language (speaking), chess playing and flying aircrafts, they described that people acquire such skills in five progressive stages and they develop four different levels of competencies.

The five stages are the stages of novice, competence, proficient, expert, and master. A novice would be a person who would not have any prior _experience_ of the task that person is called upon to perform. So, initially, the person’s world view will be based on learning the basic rules and the person would then use analytical skills to decide the course of action in learning the basic skills and gain experience. The knowledge and the tasks as this stage, according to Dreyfus’ would be termed as ‘non-situational’, i.e., tasks that are NOT contextualised to a real world action but time well spent in learning the key principles. Then, as the person gains experience, the person faces more ‘aspects’ or practical experiences of the authentic tasks that person will face in real life, the person gains understanding in recognising ‘patterns’ in which the principles present themselves in real life; in turn the instructor for such a learner would not only introduce authentic tasks but will also introduce ‘guidelines’ to understand and negotiate such challenges. The competent practitioner then gains practice (often hours of practice honing and fine tuning real life challenges that he or she can handle with the learned heuristics or guidelines); each situation builds on his competency. One of the things here is to understand that with repeated real world experiences or aspects, the learner builds his or her own ‘maxims’ and decides actions to guide his responses. At this stage, the learner becomes a proficient practitioner, ready to solve real world problems. This practice for the proficient practitioner is at the stage of analyses when faced with challenges while the practitioner builds up a repertoire of patterns and guidelines and heuristics of solving them. At some stage, with repeated practice, and the situations subtly vary, the proficient practitioner builds tacit knowledge ‘diving’ into the vast repertoire of the patterns, guidelines and maxims that he or she has learned over time. Much of the actions become unconscious and not point to point analyses all the time. This building of intuition differentiates an expert from a competent practitioner but also there are couple of issues:

  • Expertise is not a matter of only time so the notion of 10, 000 hours of practice needs to be taken with a pinch of salt: at its worst it will reinforce our errors, but what we need is slightly altered and more complex challenge each time as we practice rather than the same thing over and over again (see the Makeuseof piece that has addressed Anders Ericsson’s rule of 10, 000 hours of practice)
  • Experts often have what is referred to as ‘expertise awareness gap’ [2](alternatively expert blindness) that the expert does not know what he knows as he has developed intuition where he can jump from one node on the decision tree to another without bothering to go through the analytical phases that he went through when he _was_ a competent or even a proficient practitioner. In real problem solving world, this ability to jump from one point in the decision node to another is often referred to as ‘fluid representation’. As a result, when an expert is called upon to teach a class on which he or she is well-versed, and has developed intuition, learners may find it difficult to follow his train of thought. This is something to keep in mind as we as experts are called upon to teach a class and in particular with the Carpentries.

Dreyfus’ then describe, somewhat vaguely, a stage of ‘Mastery’ where they state that a master is a person who can ‘immerse’ oneself in a state or achieve a state of absorption such that time flows by. Mihalyi Cziksentmihalyi termed this state as a state of flow [3]. In The carpentries, our idea is to move people who are relatively novices to a stage of making them competent practitioners, not so much as Masters. This may be similar with the situation where I am teaching an undergraduate class. But the challenge with say, university courses where I am called upon to supervise an advanced student for PhD or Master degree is different: here, my challenge is to move this person beyond the stage of competent practitioner and make him an expert. So, this is an area worth exploring.

In summary, Dreyfus’ came up with a four by five matrix (see the following table):

| Skill       | Novice  | Comp  | Prof | Expert | Master |
|-------------|-------- |-------|------|--------|--------|
|Recollection | Non-sit | Sit | Sit | Sit |Sit |
|Recognition | Decom | Decom | Holi | Holi | Holi |
|Decision | Analy |Analy | Analy| Intuit | Intuit |
|Awareness | Monit |Monit | Monit| Monit | Absorb |

Adaptation of Dreyfus Model of skill acquisition in technical education (Patricia Benner and nursing training). — As with the Dreyfus’ model of skill acquisition, this too came out of the 1980s [4]. Here, Patricia Benner is talking about nursing training and use of Dreyfus’ model in the context of nursing training. She conducted a survey among 67 nurses (at various levels of training and experience) and used the Dreyfus model of novice, advanced beginner (Dreyfus’ did not write about them, Benner did, competent practitioners, proficient practitioners, and experts; Brenner omitted the Masters). As before, Benner provides examples how they would train a nurse who was a novice: start with context-free rules, they termed as ‘objective attributes’. This would mean that for a novice the idea for a trainer would be to use ‘measurable things’ that would determine what they would do in case of a challenging situation. She cites the example of training a nurse trainee on assessing the fluid balance of a patient, and lays down rules that the trainee would need to follow (measure weight of the patient, measure intake and output of the previous day, and if the weight gain was more than 500 cc for the day restrict fluid intake, and so on). As before, in level II (advanced beginner), we use ‘aspects’ that are context-sensitive and to use them, the learner need to have some prior experience. Part of advanced beginners is to develop understanding the ‘aspect’ of training: what to recognise. In level III (competent practitioners), the nurse is able to set long range goals of the practice, although he/she has still not reached the level required for proficient practitioner (level IV). A proficient practitioner moves out of working on the basis of aspect-based learning and has now developed his/her maxims or rules to live by and with practice, will develop intuition to guide his/her actions. The Proficient practitioner relies on the maxims. At the highest level, the expert, no longer relies on analytical (guidelines, rules, maxims) but more on developing intuitions after years of practice and fine tuning of proficiency.

Mastery rubric. — The mastery rubric borrows on or continues from this motif and has been applied to bioinformatics, epidemiology, and indeed other areas of study as well. A rubric is a set of principles to grade a student. Teachers provide students with the grading rubric and expect that the student will understand what is expected of them in the rubric to attain a certain grade in terms of performance, and this helps in assigning a grade to the work of the student. In the mastery rubric, these cognitive and performance tasks appropriate for a training programme or a class or course is set in advance and the teachers use such rubrics to teach and assess students (expand on this one)

Mental models. — Each of these discussions bring to focus the role of starting out with a mental model of the learners in the class. The term “mental model” here denote a set of units and interconnections between and among the different units of knowledge laid out in the shape of a map. Think of this map in the form of nodes and edges, and you can draw it out like a concept map. The nodes represent the quantum of knowledge or units of knowledge or facts or isolated facts that are stored in the minds of the learners. The edges connect the different nodes and together this complement of nodes and edges complete the mental model.

The mental model of a novice is different from the mental model of a competent practitioner and likewise, the mental model of a competent practitioner is different from that of an expert. The points of difference is not so much in the number and intensity of individual nodes per se, rather in their inter-connections. A novice and an expert may have the same number of nodes in the map, but the expert has many more connections compared with the novice, who may not be able to make the connections explicit. This means:

  • The aim of training/teaching will be to reinforce these connections, so even if you teach diverse facts, your aim will be to develop the interconnections among them. How well you can build those connections will determine the success of your teaching.
  • The way to foster those connections will be by deliberate practice. By this, I mean that you will provide learning and practice opportunities in the class so that the learners can practice over and over again subtly changed and challenging situations that will reinforce these connections.

But what does novice, competent practitioner, and expert mean in real world? According to the Carpentries, a novice would be that individual whose mental model is so sparse that the person may not even know what he does not know, or may not be able to ask the right kind of questions to clarify his doubts or move on. In other words, for a novice, the state of knowledge is that, he does not know what he does not know.

So for a novice, it is important that we do not push too many facts all at one go (but aim at somewhere between 7–9 facts no more but reinforce connections). So the way to start with a novice according to the Dreyfus model would be to start with facts and heuristics that are context-independent. We provide the rules and ways to solve problems BEFORE we lead them to solve problems. This is where guided instructions as opposed to problem based learning works with novices. A competent practitioner, on the other hand, has forged some connections, so for a competent practitioner, the case is often that, he knows what he does not know. So when a competent practitioner learns, or executes actions, these are context dependent so that out of heuristics and rules, he carves out strategies for new challenges, and builds a repertoire of challenges. For a competent practitioner, perhaps, an unguided challenge or problem based learning may work.

Then, an expert moves on from being a competent practitioner and has such a rich repertoire of experience and intuition that he often jumps around from one node to another, quite unconscious as to how such node jumping occurred in search of solutions to problems and this does not matter. But this introduces a couple of new ideas:

  • Expertise awareness gap. — Often referred to as expert blindness, the expert fails to see why or how a novice fails to see what to him seems such obvious. So, if such an expert is asked to take a class of novices or undergraduates, that experience can be frustrating as the expert has no notion of or at least forgotten by now, what it takes to go through solving a problem going step by step. This is the expertise awareness gap and needs to be addressed. The expert’s blindness makes him use words such as ‘just’, “Oh it is so simple”, which can be perceived as dismissive languages in a classroom setting and one must be aware of these tendencies.
  • Flow. — This is what differentiates an expert from a competent practitioner, where challenge and ability are matched with each other so much that time seems to disappear during the task (see the optimum zone in the figure below):
Zone of flow (source: http://deliveryimages.acm.org/10.1145/1240000/1232769/figs/f1.jpg)

So in the end, an expert is a person who does not know what he knows. Somewhere in the process people start suffering from impostor syndrome where the teacher senses that he knows less than what he actually knows. This happens quite often with newcomers to the training once they have qualified. On the other hand, with the novices, it is the other way round: people suffer from Dunning Kruger Effect, where many people tend to think that they know more than they actually know. Each of these are results of faulty mental models that would need to mature over time.

So, the factor that moves a novice from a specific mental model to an expert is repeated deliberate practice of skills. However, just any form of repetition is not going to work because quite often there are issues around errors or conceptual or view based errors can propagate. Generally, there are three different types of errors:

  • Factual errors. — These are easily remedied
  • Errors in mental models. — Most teaching and training can be addressed here, so that with repeated formative assessments these can be corrected.
  • Errors in worldview. — These are more complicated and correcting this type of error is beyond the scope of a single class/course.

10,000 hours to expertise is not accurate. — Malcolm Gladwell is said to have written in Outliers (link??) that it takes about 10, 000 hours for anyone to become an expert in the field. However, the truth is more complex than that. If you are a novice and have a faulty mental model to start with 10, 000 hours of practice will only make it worse, as the same errors will be propagated. The clue is to do sufficient practice with subtle changes in the challenges one faces with repeated and deliberate practice that tests the concepts learned in various stages. Gradually, over time, people generate enough experience to identify the patterns of problems and issues that can move them from the stage of context-free rubric based heuristics based reasoning to more complex intuition based solutions to expertise where overall gestalt of the problem become the source of finding solutions. This is where the following concepts become useful:

Limit the amount of information (cognitive load). — The cognitive load theory states that higher amount of cognitive load hinders understanding. About seven items of disjoint or orthogonal or disconnected pieces of information can be provided at any given setting (where does the number seven started?). Sweller (1988) stated to limit cognitive load during informatio processing in classroom setting, and to make things easier, ‘chunking’ similar pieces of information is useful to minimise cognitive load.

Use Bloom’s taxonomy to set teaching learning objectives. — Benjamin Bloom (1956) came up with a classification of writing educational objectives based on six related and hierarchically connected steps (see Figure)

Bloom’s Taxonomy (from: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/)
  • Knowledge. — Factually recollect and can repeat or recite basic concepts. This is the first step to move to the next step
  • Understand. — Explain ideas or concepts in one’s own terms
  • Apply. — When the student understands or grasps facts and related concepts, he starts applying them in real world situations to solve problems and apply them in new domains or areas
  • Analyse. — This is the step that moves one step from applying the concepts that are learned where the student then compares and contrasts with similar concepts elsewhere or starts forming the connections. This is where there is an overlap between mental model of the competent practitioner from the novice. This is a point that can be targeted at various stages in writing learning/teaching objectives for classes and courses.
  • Evaluate. — Once the skills of analysis are learned, the student then uses these skills to evaluate diverse concepts and ideas and critically appaise diverse body of information.
  • Create. — This is the apex where learned and memorised concepts, understanding, applying to new situations, and evaluations then pave the way for creating new knowledge artefacts.

For a good discussion of Bloom’s taxonomy, see

Repeat formative assessment to diagnose and reinforce learning points. — Formative assessments are teaching strategies where you will design tests as you teach to (a) diagnose whether students understood the concepts you taught; and (b) what misunderstandings or misconceptions remain. The recommendation is to use one formative assessment every five or 10 minutes. The formative assessments are in the form of multiple choice tests with a stem and five or four choices. Students select only one answer as the correct answer. After you collect the responses, analyse and discuss the correct responses. If you see that students have missed a key concept, it is a good time to reinforce and teach the key concept before moving on. The question must be concept based and not fact based. The following is a good resource:

Use live coding to teach. — While this is specific to Carpentries, with some modifications, this can be used for any classroom teaching context. You can create simple formulae and use a software or spreadsheet to reinforce these points while teaching simple calculations for instance. We have used live coding for teaching R, or other software programming applications, but this can equally be used to demonstrate and solve real world problems. This principle is based on ‘maximally guided instruction’ as opposed to ‘problem based learning’. In maximally guided instruction, the teacher imparts lessons to the students before asking them to solve problems, and live coding or participatory live coding is one of the best ways to reinforce several things:

  1. Participatory live coding enables students and teachers to participate together in a learning activity and fosters active learning. Active learning is a teaching strategy where students and teachers together participate in the learning activities as equal partners, so students do not passively listen to lectures.
  2. When you use participatory live coding, you can build in ‘errors’ deliberately in the process of teaching, so that students can ‘see’ that even experts can make mistakes and such mistakes or errors is part of learning. How to troubleshoot is equally important
  3. Participatory live coding forces you to slow down as you deliver the contents, so you can cover lessons repetitively and go deeper instead of covering a lot of ground in a lecture — this reinforces learning

The best practices and what happens in participatory live coding:

  • You should stand up and move about
  • You should speak as you demonstrate the code or write the code
  • Students should repeat after you
  • You will use illustrations and slowly comment on every action you perform
  • Wait till all students have moved up to where you are
  • Use red and green sticky notes to check students’ learning
  • You can and should use different formative assessment strategies to reinforce learning

Obtain and provide feedback. — At each step, obtain feedback from the students you teach and likewise, when you provide them tasks to perform so that you can test how or what they have learned, provide them feedback as well. The goal of any teaching endeavour will be to foster a growth mindset to learning and this is best achieved with feedback real and live inside a classroom.

Carol Dweck coined the term ‘Growth mindset’ to indicate a mental pattern of some learners who are resilient and would want to learn more. From the website https://www.mindsetworks.com/science/, we see they define growth mindset thus:

After studying the behavior of thousands of children, Dr. Dweck coined the terms fixed mindset and growth mindset to describe the underlying beliefs people have about learning and intelligence. When students believe they can get smarter, they understand that effort makes them stronger. Therefore they put in extra time and effort, and that leads to higher achievement.

The goal of any class will be to foster a growth mindset in the students. Be aware that some students will still be in the fixed mindset mode where they will set up a internal dialogue that they are not good enough or that they do not want to learn anything new. If you want to foster the sense within your students that their efforts will make them stronger learners, then provide them challenges and then provide them feedback. Equally, you will also solicit feedback from them periodically to check that you are in the right track. But what type of feedback will you provide?

There are three types of feedback you can provide — performance-based, effort-based, and improvement-based. Performance-based feedback is a generic feedback that reflects on the person or a comment such as “you have done well or you are a good student”; effort-based feedback is one where you comment on the effort and more specific on the task as you provide feedback, such as ‘add extra material on topic X’, and improvement-based feedback is feedback where you note what change did the student bring about, example, ‘you have considerably improved between second term and this term in writing about X’.

John Hattie and Helen Timperley⁶ have classified four stages of feedback. This is what they write in their paper (I have highlighted the relevant parts):

“… We claim that there are four major levels and that the level at which feedback is directed influences its effectiveness. First, feedback can be about a task or product, such as whether work is correct or incorrect (FT). This level of feedback may include directions to acquire more, different, or correct information, such as “You need to include more about the Treaty of Versailles.” Second, feedback can be aimed at the process (FP) used to create a product or complete a task. This kind of feedback is more directly aimed at the processing of information, or learning processes requiring under-standing or completing the task. For example, a teacher or peer may say to a learner, “You need to edit this piece of writing by attending to the descriptors you have used so the reader is able to understand the nuances of your meaning,” or “This page may make more sense if you use the strategies we talked about earlier.” Third, feedback to students can be focused at the self-regulation level (FR), including greater skill in self-evaluation or confidence to engage further on a task. For example, “You already know the key features of the opening of an argument. Check to see whether you have incorporated them in your first paragraph.” Such feedback can have major influences on self-efficacy, self-regulatory proficiencies, and self-beliefs about students as learners, such that the students are encouraged or informed how to better and more effortlessly continue on the task. Fourth, feedback can be personal in the sense that it is directed to the “self,” (FS) which, we argue below, is too often unrelated to performance on the task. Examples of such feedback include “You are a great student” and ‘“That’s an intelligent response, well done.”

As much as possible, keep your feedback at the level of effort-based or improvement-based, or FT, FP, and FR according to Hattie-Timperley framework, and never ever on FS or performance-based. Use encouraging words as much as you can, and if you have to provide critical feedback, sandwich this between two positive constructive feedback and then, when you provide feedback, suggest what you want to see differently rather than attack (this is often termed as ‘poop sandwich’ method where you deliver a non-delectable item between two acceptable pieces). So, here are the lowdown when teaching a class or carpentries:

  • Use red sticky and green sticky notes. Provide each learner a green and a red sticky note that they stick on the lid of their laptops and use these to draw attention if they are stuck in course of the lesson. When the lesson is over, ask them to use these green and red sticky notes to provide you with feedback (one up one down — one thing they liked and one thing they did not like)
  • Use a poop sandwich method to provide feedback to students — layer a critical comment in between two good comments and in the critical comment, state what you want to see done differently
  • All your comments must be on the basis of effort-based or improvement-based, and NEVER performance-based

Use faded examples and Parsons problem to check learning. — This is particularly useful for teaching programming or complex information processing such as while teaching Epidemiology say you want to teach students how to calculate standardised mortality rates. Work step by step first, then take out one step at a time and ask them to fill in the missing step, till you recreate the problem entirely and ask them to work on it. See the following example for writing a function:

# first write the full function
addtwo <- function(x){
y <- x + 2
return(y)
} ....................[1]
# now take out one part at a time
addthree <- function(_){
y <- m + 3
return(y)
} .................... [2]
addthree <- function(_){
y <- _ + _
_(y)
} .....................[3]multiplytwo <- ....................[4]

As you can see in the above example, in subsequent steps 1 through 4, we have taken ‘off the training wheels’ so that the participants can fill in the missing bits. You can mix this with multiple choice questions if you want, but often this is good enough to illustrate the key learning points.

— — —

[1]:Dreyfus, S. E., & Dreyfus, H. L. (1980). A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition: https://doi.org/10.21236/ADA084551
[2] See http://mnav.com/expert-blindness/ for a nice explanation of what is meant by expert blindness or expert awareness gap
[3] Read the book by
[4]: Deakin University, Faculty of Education, & Open Campus Program. (1994). A collection of readings related to competency-based training. Geelong, Vic.: Deakin University.
[5]: for a discussion of the mastery rubric in the context of clinical training, see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4664514/
[6]: See Power of Feedback by John Hattie and Helen Timperley for a more detailed discussion here, http://www.columbia.edu/~mvp19/ETF/Feedback.pdf

Associate Professor of Epidemiology and Environmental Health at the University of Canterbury, New Zealand. Also in: https://refind.com/arinbasu

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store