About the Transforming Technologies course
Transforming Technologies was a new postgraduate award run by the Centre for Lifelong Learning, University of Warwick. It was part of a wider programme offering an 'access point' to those who might be considering further study, who already have a degree or equivalent professional qualification. The course was, therefore, designed to equip students with the necessary critical, academic skills befitting of postgraduate/Masters study. However, the primary aim of the course is to develop and enhance students' understanding of technology for teaching and learning.
More detail about the course can be found in the YouTube video below.
Promotional video
Creating the matrix
During four of the ten sessions of Transforming Technologies, students looked at the e-portfolio marking matrix, thinking about what should be included, discussing it and testing it (see diagram below). I combined their ideas with my own, the University assessment criteria at level 7 and some things I had come across in the literature to produce the final marking matrix.
Marking using the matrix
Students used the matrix to anonymously peer assess each other's digital posters and I did the same. Students gave qualitative feedback, but also highlighted on the matrix where they thought their peers' work indicated skills against the assessment criteria (which were divided into knowledge and understanding; analysis and critique, presentation and digital skills). An example of a comparison between tutor and student marking on the same digital poster is shown in the diagram below (yellow = highlighted indicators; blue lines = indicators which both identified). Also attached here is the final matrix.
Critique of the project
What this project achieved was to create and implement a new matrix for assessment, but it has not been tested or critiqued by enough people yet. The sample of students was small, but did indicate that this was something which could be used. I have not carried out detailed analysis on the marking exercise, and this could be an interesting direction.
I do think the matrix is quite detailed; maybe it can now be refined. The process itself may need more refinement too; I like the fact that this matrix values students' digital skills, but in the sample I marked this may have been at the expense of analysis and critique. I also wonder if there was a clear enough distinction between the intentions of a 'digital poster' and an e-portfolio' in general.
I would certainly like to gauge the views of others and evolve this project further.
Introduction to the project
I have long been interested in alternative forms of assessment that might recognise a range of student skllls and experiences, but still be rigorous enough for Higher Education. This project saw me developing an e-portfolio marking matrix, in conjunction with students, for the assessment of a digital poster they created as part of their summative assessment on the course. Students completed their posters and then peer assessed each other's using the matrix. I also used the matrix and compared the peer and tutor marking, so see if there was consistency. Finally students evaluated their experiences.
Some pedagogical themes I was interested in with this project were:
- alternative assessment methods in HE, specifically e-portfolios.
- the notion of 'student as producer'
- curriculum as a process
- student agency in curriculum design decisions
Presentation slides: 08/09/16
Drawing on the literature
There is definitely a current trend towards investigating alternative forms of assessment in Higher Education - JISC have had a substantial Assessment and Feedback Programme with many projects and the HEA have recently published a Framework for Transforming Assessment. There is also a growing body of academic literature around e-Portfolios, for example, in the International Journal of e-Portfolio.
I read two papers which influenced my thinking for this project, in particular. The first, by Trevitt et al (2013), discussed the used of e-portfolios for both summative and formative assessment in HE. They take a historical view, looking back to a time before the public exam in HE, where a mentor coached a degree candidate in preparation for an oral exam. Whilst not practical now, due to student numbers, I like the notion of mentoring and supporting the 'process' and ongoing development of the student and this dialogue can be perhaps be achieved in e-Portfolios. I also found the paper by Springfield (2015) fascinating: they propose an e-Portfolio marking rubric which assesses those 'affective' areas of development: confidence, pride, identity etc, Students can (and should) go through these rich transformative changes in HE and e-Portfolios have the potential to capture this much better than a summative assessment or final exam.
Leading on from this, I think e-Portfolios can offer a more holistic assessment of the individual's skills and experiences, valuing not just cognitive development, but affective elements and digital skills. The diagram I have created above, based on Bloom's Taxonomy, attempts to express this.
Indicative reading
Brookfield, Stephen (1998) Critically reflective practice, Journal of Continuing Education in the Health Professions; Fall 1998; 18, 4
http://www.anitacrawley.net/Resources/Articles/Brookfield.pdf
Higher Education Academy (2015) Framework for transforming assessment in Higher Education, The Higher Education Academy Framework series, HEA.
https://www.heacademy.ac.uk/sites/default/files/downloads/transforming-assessment-in-he.pdf
JISC (2015) Digital Futures: Expert Briefings on Digital Technologies for Education and Research (eds: , and ), Chandos Publishing, Kidlington.
Springfield (2015) Transformation Rubric for Engaged Learning: A tool and method for measuring life-changing experiences
Trevitt, C, Macduff, A and Steed, A (2013) eportfolios for learning and as evidence of achievement: scoping the academic practice development agenda aheadThe Internet and Higher Education, Vol.20, January 2014, 69-78 http://www.sciencedirect.com/science/article/pii/S1096751613000250
Wuetherick et al (2015) Why e-portfolios? Student perceptions of eportfolio use in continuing education learning environments, International Journal of ePortfolio, Volume 5, Number 1, 39-53 http://www.theijep.com/pdf/IJEP135.pdf