These resources provide an overview of materials presented at ALT-C 2016,  discussing work with students on the design and implementation of a marking matrix for the assessment of an e-Portfolio (specifically, in this course context, a 'digital poster' built in MyPortfolio). The longer term aim, after students created their digital posters, was to make them a part of the Technologies for Academic Practice (TAP) repository shared with Warwick staff and students.

Please get in touch if you have any questions or comments: S.Hattersley@warwick.ac.uk.      Twitter: @sarahattersley

About the Transforming Technologies course

Transforming Technologies was  a new postgraduate award run by the Centre for Lifelong Learning, University of Warwick. It was part of a wider programme offering an 'access point' to those who might be considering further study, who already have a degree or equivalent professional qualification. The course was, therefore, designed to equip students with the necessary critical, academic skills befitting of postgraduate/Masters study. However, the primary aim of the course is to develop and enhance students' understanding of technology for teaching and learning. 

More detail about the course can be found in the YouTube video below.

Promotional video

Curriculum Design and Transforming Technologies

Introduction to the project

I have long been interested in alternative forms of assessment that might recognise a range of student skllls and experiences, but still be rigorous enough for Higher Education. This project saw me developing an e-portfolio marking matrix, in conjunction with students, for the assessment of a digital poster they created as part of their summative assessment on the course. Students completed their posters and then peer assessed each other's using the matrix. I also used the matrix and compared the peer and tutor marking, so see if there was consistency. Finally students evaluated their experiences.

Some pedagogical themes I was interested in with this project were:

  • alternative assessment methods in HE, specifically e-portfolios.
  • the notion of 'student as producer'
  • curriculum as a process
  • student agency in curriculum design decisions

Considering theory

E-Portfolio matrix

Presentation slides: 08/09/16

Creating the matrix

During four of the ten sessions of Transforming Technologies, students looked at the e-portfolio marking matrix, thinking about what should be included, discussing it and testing it (see diagram below). I combined their ideas with my own, the University assessment criteria at level 7 and some things I had come across in the literature to produce the final marking matrix.

Marking using the matrix

Students used the matrix to anonymously peer assess each other's digital posters and I did the same. Students gave qualitative feedback, but also highlighted on the matrix where they thought their peers' work indicated skills against the assessment criteria (which were divided into knowledge and understanding; analysis and critique, presentation and digital skills). An example of a comparison between tutor and student marking on the same digital poster is shown in the diagram below (yellow = highlighted indicators; blue lines = indicators which both identified). Also attached here is the final matrix.

Critique of the project

What this project achieved was to create and implement a new matrix for assessment, but it has not been tested or critiqued by enough people yet. The sample of students was small, but did indicate that this was something which could be used. I have not carried out detailed analysis on the marking exercise, and this could be an interesting direction. 

I do think the matrix is quite detailed; maybe it can now be refined. The process itself may need more refinement too; I like the fact that this matrix values students' digital skills, but in the sample I marked this may have been at the expense of analysis and critique. I also wonder if there was a clear enough distinction between the intentions of a 'digital poster' and an e-portfolio' in general. 

I would certainly like to gauge the views of others and evolve this project further.

Drawing on the literature

There is definitely a current trend towards investigating alternative forms of assessment in Higher Education - JISC have had a substantial Assessment and Feedback Programme with many projects and the HEA have recently published a Framework for Transforming Assessment. There is also a growing body of academic literature around e-Portfolios, for example, in the International Journal of e-Portfolio.

I read two papers which influenced my thinking for this project, in particular. The first, by Trevitt et al (2013), discussed the used of e-portfolios for both summative and formative assessment in HE. They take a historical view, looking back to a time before the public exam in HE, where a mentor coached a degree candidate in preparation for an oral exam. Whilst not practical now, due to student numbers, I like the notion of mentoring and supporting the 'process' and ongoing development of the student and this dialogue can be perhaps be achieved in e-Portfolios. I also found the paper by Springfield (2015) fascinating: they propose an e-Portfolio marking rubric which assesses those 'affective' areas of development: confidence, pride, identity etc, Students can (and should) go through these rich transformative changes in HE and e-Portfolios have the potential to capture this much better than a summative assessment or final exam.

Leading on from this, I think e-Portfolios can offer a more holistic assessment of the individual's skills and experiences, valuing not just cognitive development, but affective elements and digital skills. The diagram I have created above, based on Bloom's Taxonomy, attempts to express this.

Indicative reading

Brookfield, Stephen (1998) Critically reflective practice, Journal of Continuing Education in the Health Professions; Fall 1998; 18, 4
http://www.anitacrawley.net/Resources/Articles/Brookfield.pdf

Higher Education Academy (2015) Framework for transforming assessment in Higher Education, The Higher Education Academy Framework series, HEA.
https://www.heacademy.ac.uk/sites/default/files/downloads/transforming-assessment-in-he.pdf

JISC (2015) Digital Futures: Expert Briefings on Digital Technologies for Education and Research (eds: Martin Hall, Martyn Harrow and Lorraine Estelle), Chandos Publishing, Kidlington.

Springfield (2015) Transformation Rubric for Engaged Learning: A tool and method for measuring life-changing experiences

Trevitt, C, Macduff, A and Steed, A (2013) eportfolios for learning and as evidence of achievement: scoping the academic practice development agenda aheadThe Internet and Higher Education, Vol.20, January 2014, 69-78  http://www.sciencedirect.com/science/article/pii/S1096751613000250

Wuetherick et al (2015) Why e-portfolios? Student perceptions of eportfolio use in continuing education learning environments, International Journal of ePortfolio, Volume 5, Number 1, 39-53 http://www.theijep.com/pdf/IJEP135.pdf