Category: Tools

Recently I gave a talk on Augmenting pedagogical writing support with contextualizable learning analytics at the CRLI seminar series in the University of Sydney.  It was a great opportunity to share and discuss ideas from my PhD research, and indeed a privilege to be invited to present at this seminar. Long time slot means less time constraints, so I enjoyed doing the 1 hour+ session. The talk is recorded and available for viewing on Youtube, and the slides are here. This post is a summary of the key ideas from this talk and an upcoming paper on ‘Contextualizable Learning Analytics Design (CLAD)’.

Big data, learning analytics and education:

Big data and artificial intelligence are changing many ways we do things to improve our lives (for better or for worse). Companies around the world including Facebook, Google, Apple and Amazon use data everyday to get big insights to support us. What can the more traditional organizations like educational institutions use data for? Can we harness this technology and data to improve learning? To answer these questions, Learning Analytics (LA) emerged as a field to attempt tackling huge amounts of data in education. Although data was previously available in education research for decades, different granularities of data from multiple sources in authentic scenarios and technical affordances of new tools can now support many causes which were not previously plausible. This root cause for the inception of the field has probably been a reason for its emphasis on ‘big impact’ and generalizable solutions that can cater to and scale up to huge numbers. Massive Open Online Courses (MOOCS) are a classic example of how we can scale teaching to a large number of learners using technology. However, the problem with scalable, generalizable solutions in learning analytics is that education is inherently contextual, and a one-size-fits all approach would not work in all contexts the same way. This has led to the argument on moving from big data to meaningful data for learning analytics.

Bringing in the context:

To bring the educational context to Learning Analytics (LA), it must be coupled with pedagogical approaches. This involves the integration of LA in pedagogical contexts to augment the learning design and provide analytics that are aligned with the intended learning outcomes. Learning Design (LD) describes an educational process, and involves the design of units of learning, learning activities or learning environment which are pedagogically informed. LA can provide the necessary data, methodologies and tools to test the assumptions of the learning design, and LD can add value to the analytics by making it meaningful for the learner. By bringing LA and LD together, they can contribute to each other and close the gap between the potential and actual use of technology.

Contextualizable Learning Analytics Design:

We introduce the Contextualizable Learning Analytics Design (CLAD) model in a forthcoming article by bringing together the elements of LA and LD for context. The educators are involved with LA developers to co-design this contextualization. This involves LD elements of assessment and task design, and LA elements of features and feedback working dynamically and in sync for different contexts, rather than being rigidly fixed. The CLAD model is demonstrated by implementing the Writing Analytics tool ‘AcaWriter’ in different learning contexts (Law essay writing, Accounting business report writing). AcaWriter, developed by the Connected Intelligence Centre, UTS provides automated feedback on student writing based on rhetorical moves. To contextualize the use of this LA tool for students, the elements of the CLAD model were employed as follows:

• Assessment formed the basis of contextualization to align AcaWriter with the intended learning outcomes.
• The features of data that are important for the context were picked so that AcaWriter can bring them to the attention of the learners.
• The feedback from AcaWriter was tuned to make it relevant for the context of writing by mapping it back to assessment criteria.
• Task design ensured that AcaWriter activities are relevant to the learner and grounded by pedagogic theory.

With such contextualized LA, the educator has agency to design learning analytics that is relevant to the learning context, and the learner finds it meaningful due to its embedding in the curriculum. This ensures that LA contributes to learning in authentic practice by augmenting existing good pedagogic practice. The approach scales over multiple learning contexts by transferring good design patterns from one learning context to another (for example from law essay writing to accounting business report writing).

More details on the above can be found in the following article, and related resources are available on the HETA project website.

References:

• Antonette Shibani, Simon Knight and Simon Buckingham Shum (Forthcoming). Contextualizable Learning Analytics Design: A Generic Model, and Writing Analytics Evaluations. Full paper accepted for publication at the Ninth International Conference on Learning Analytics & Knowledge(LAK ’19), Tempe, Arizona
• Simon Knight, Antonette Shibani and Simon Buckingham Shum (2018). Augmenting Formative Writing Assessment with Learning Analytics: A Design Abstraction Approach. Full Paper presented at the London Festival of Learning cross-over track at ICLS’18.