Two strains of thought coming together
First, I was in a meeting with Associate Deans L&T and Heads of Program yesterday from one of our Faculties, and I was talking about the TEQSA publication Assessment reform for the age of Artificial Intelligence and one of the suggestions in the paper that, “Assessment should emphasise…a systemic approach to program assessment aligned with disciplines/qualifications” (p.4). In it, they suggest a program-wide approach, where assessment design considerations should span a whole program, or at least a sub-structure within it. In the meeting, it was suggested to me that ‘this would be difficult as not all students take the same study path’, and that is true. I think I want to challenge that here.
Second, I was trying to finish off a journal paper on ePortfolio use in Australasia, and I was thinking through a use-case for the use of Generative AI and ePortfolio, but more importantly, how ePortfolio could guard against the widespread use of Gen AI (if that were desired or possible).
Then it occurred to me, what happens if I put 1 and 2 together? Then I may come up with 3 (funny that).
1 + 2 = 3
Third, what if we were asking our students to reflect on how the assessments they are doing in the individual units are linked to the standards for the profession they are emerging into (or graduate attribute if no standards are linked)? It wouldn’t matter at that point if they were from a different discipline or not, there would be some alignment possible. Most professions have standards (Engineering, Nursing, Education, Psychology, etc. etc.), and many of the Programs/Courses we offer already have to align with the standards for that profession. In fact, the University gets certified by the professional body as meeting the standards for our students to be registered into that body once they graduate. And if not professionally accredited, most universities align to their Graduate Attributes (some do both). But how do students make that link between what they study and those standards? For example, how does the report they wrote, the essay they submitted, the presentation they gave, the infographic they designed, and the plan they drew up help them meet the standards they have to meet to be registered as a professional in their field?
Interestingly, disciplines (faculty) have already had to map this out so their programs could be accredited by the professional bodies, and they already have aligned the assessments to meet the learning outcomes, to then meet those standards. So, we are more than halfway there already.
Enter Generative AI and ePortfolio.
ePortfolio thinking at a program level is not a new concept, we have been using ePortfolio for over a decade, probably closer to two decades now. We have asked students to do reflections in their ePortfolio, to record their practice and provide evidence of placements and the like. It has been very popular in Education and Nursing for many years, just to name two professions. In some cases, we have asked them to even link this to their professional competencies.
At this point, Generative AI is not great at personal reflection. It could, if using ChatGPT 4 or one of the other paid versions, do a reasonable summary of all the stuff you have done and if you feed it all the things you have done and the standards, it could make some link between the two, but that would not be a trivial undertaking and you would have to be pretty cluey to do that effectively. Having said that, if someone was to use it in this way, they would learn something about themselves anyway. But let’s presume that they are not prepared to go to those lengths to provide a personal reflection on their learning, which may only be worth 10% in a unit of study (arguably) just so they can make the link between what they have learned to one or more of the professional standards they need to attain. Not that this would be a bad use of Generative AI 🙂
Program-wide assessment
Now, if the faculty has done its mapping, if they make explicit how each assignment might be linked to a professional standard or graduate attribute (if no professional standards exist), then surely, we can ask our students to reflect on this as part of their learning journey through the program. This may only be worth 10% (arguably) in an individual unit of study but it could be used to build towards (articulate too) creating a much larger piece of assessment that could be contained in a final year Capstone Unit, where the student pulls all those pieces of string together to create their professional profile. Arguably this final piece could make up 40, 50, or 60% of the mark in the capstone unit, but it has been something they have been building towards over the period of their studies. So, the student has not only reflected on each step of the journey but also now has the evidence that they are worthy of registration into their profession.
As I said, it is not a new idea regarding ePortfolio, but it is a good case for using ePortfolio in a program, not just a unit. Could they use ChatGPT to help them? Maybe, but what would be the point, as we are talking about an iterative piece of assessment over the life of the program, not in an individual unit, of which they may be doing 4 at any time? It’s a big ask to pull all those bits together in a Generative AI engine. But if Chat GPT or the like can help them make the mental and emotional connection to link what they have learned to their professional standards then surely;
Michael Sankey 