The forum ran as an online webinar where speakers from India and Australia shared their thoughts on the evolving role of generative AI tools such as ChatGPT in higher education. The full recording of the session is below, and a transcript is also available:
I am delighted to see emerging initiatives that promote and foster collaboration between India and Australia, and I’m glad to be a small part of it. I was also previously selected as one of the AIRS Fellows in the Australia India Research Students Fellowship program that supported short-term research exchange of scholars between the two countries, and have published with Indian colleagues. I hope to continue my research collaboration with colleagues from various institutions in India on related topics.
I attended the prestigious Human-Computer Interaction (HCI) conference CHI’24 at Hawaii, Honolulu in May 2024. While I’m quite familiar with the field of HCI, it was my first time attending the conference because it is much wider than my main area of research (Learning Analytics, AI in education, and Writing Analytics). The sheer scale of the conference (~2K to 3K attendees) and the broad range of topics it covers (check out this full program) is almost impossible to fully grasp!
TLDR; Go to the end for the list of paper from CHI’24.
My personal highlight was the Intelligent Writing Assistants Workshop, which was running for the third time at CHI, organized by a bunch of fun people who are all super keen about researching the use of AI to assist writing. Picture from our workshop below (thanks, Theimo, for the LinkedIn post)
Pictured: Participants of the Intelligent Writing Assistants CHI’24 workshop at the end of the session
This was actually a deep dive into the Ecosystem aspect of a larger piece of work we presented at CHI on A Design Space for Intelligent and Interactive Writing Assistants. The full design space from our full paper mapped the space of intelligent writing assistants reviewing 115 papers from HCI and NLP, with a team of 36 authors, led by Mina Lee.
Figure: Design space for intelligent and interactive writing assistants consisting of five key aspects—task, user, technology, interaction, and ecosystem from our full paper.
An interactive tool is also presented to explore the literature in detail.
— Shibani Antonette (@ShibaniAntonett) May 14, 2024
I also had a late-breaking work poster presentation on Critical Interaction with AI on Written Assessment (I have a seperate post about it!) where we explored how students engaged with generative AI tools like ChatGPT for their writing tasks, and if they were able to navigate this interaction critically.
A cherished memory to hold on to was also the time I spent with my friend Vanessa, who is currently a Research Fellow at Monash university during this trip in Hawaii. Vanessa and I started our PhD together at th Connected Intelligence Centre at UTS ~8 years ago, and it was really nice to catch up after a long time (along with few others). I had also just visited Monash university’s CoLAM a week before for a talk and meeting fellow Learning Analytics researchers, hosted by her and Roberto. The group do interesting work in Learning Analytics that is worth checking out.
6 years apart… On the left: Vanessa and I in 2018 while attending AIED/ ICLS 2018 in London; On the right: Us while attending CHI in 2024 in Hawaii.
TLDR -> Research publications:
Here are all the papers from the work we presented at CHI’24:
Mina Lee, Katy Ilonka Gero, John Joon Young Chung, Simon Buckingham Shum, Vipul Raheja, Hua Shen, Subhashini Venugopalan, Thiemo Wambsganss, David Zhou, Emad A. Alghamdi, Tal August, Avinash Bhat, Madiha Zahrah Choksi, Senjuti Dutta, Jin L.C. Guo, Md Naimul Hoque, Yewon Kim, Simon Knight, Seyed Parsa Neshaei, Agnia Sergeyuk, Antonette Shibani, Disha Shrivastava, Lila Shroff, Jessi Stark, Sarah Sterman, Sitong Wang, Antoine Bosselut, Daniel Buschek, Joseph Chee Chang, Sherol Chen, Max Kreminski, Joonsuk Park, and Roy Pea, Eugenia H. Rho, Shannon Zejiang Shen, Pao Siangliulue. 2024. A Design Space for Intelligent and Interactive Writing Assistants. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24), May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 33 pages. https://doi.org/10.1145/3613904.3642697
Antonette Shibani, Simon Knight, Kirsty Kitto, Ajanie Karunanayake, Simon Buckingham Shum (2024). Untangling Critical Interaction with AI in Students’ Written Assessment. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI ’24), May 11-16, 2024, Honolulu, HI, USA. doi.org/10.1145/3613905.3651083
I’ve had a longstanding interest in exploring how students engage critically with automated feedback and develop their AI literacy. In our LAK22 paper, we argued why it is so important that we develop these skills in learners. There is a heightened necessity in today’s educational landscape for learners in the age of generative AI (Gen AI) to engage with AI critically.
Our upcoming CHI publication investigates the fundamental question: Why do students engage with Gen AI for their writing tasks, and how can they navigate this interaction critically? In our paper, we define in concrete terms and stages how criticality can manifest when students write with ChatGPT support. We draw from theory and examples in empirical data (which are still unbelievably scarce in the literature) to understand and expand the notion of critical interaction with AI.
A pre-print version is available for download on Arxiv [PDF]. Full citation below:
Next week, I’m presenting my work in the First workshop on Generative AI for Learning Analytics (GenAI-LA) at the 14th International Conference on Learning Analytics and Knowledge LAK 2024:
Antonette Shibani, Faerie Mattins, Srivarshan Selvaraj, Ratnavel Rajalakshmi & Gnana Bharathy (2024) Tamil Co-Writer: Towards inclusive use of generative AI for writing support. In Joint Proceedings of LAK 2024 Workshops, co-located with 14th International Conference on Learning Analytics and Knowledge (LAK 2024), Kyoto, Japan, March 18-22, 2024.
With colleagues in India, we developed Tamil Co-Writer, a GenAI-supported writing tool that offers AI suggestions for writing in the regional Indian language Tamil (which is my first language). The majority of AI-based writing assistants are created for English language users and do not address the needs of linguistically diverse groups of learners. Catering to languages typically under-represented in NLP is important in the generative AI era for the inclusive use of AI for learner support. Combined with analytics on AI usage, the tool can offer writers improved productivity and a chance to reflect on their optimal/sub-optimal collaborations with AI.
The tool combined the following elements:
An interactive AI writing environment that offers several input modes to write in Tamil
Analytics of writer’s AI interaction in the session for reflection (See post on CoAuthorViz for details, and related paper here)
A short video summarising the key insights from the paper is below:
Generative AI (GenAI) has captured global attention since ChatGPT was publicly released in November 2022. The remarkable capabilities of AI have sparked a myriad of discussions around its vast potential, ethical considerations, and transformative impact across diverse sectors, including education. In particular, how humans can learn to work with AI to augment their intelligence rather than undermine it greatly interests many communities.
My own interest in writing research led me to explore human-AI partnerships for writing. We are not very far from using generative AI technologies in everyday writing when co-pilots become the norm rather than an exception. It is possible that a ubiquitous tool like Microsoft Word that many use as their preferred platform for digital writing comes with AI support as an essential feature (and early research shows how people are imagining these) for improved productivity. But at what cost?
In our recent full paper, we explored an analytic approach to study writers’ support seeking behaviour and dependence on AI in a co-writing environment:
Using keystroke data from the interactive writing environment CoAuthor powered by GPT-3, we developed CoAuthorViz (See example figure below) to characterize writer interaction with AI feedback. ‘CoAuthorViz’ captured key constructs such as the writer incorporating a GPT-3 suggested text as is (GPT-3 suggestion selection), the writer not incorporating a GPT-3 suggestion (Empty GPT-3 call), the writer modifying the suggested text (GPT-3 suggestion modification), and the writer’s own writing (user text addition). We demonstrated how such visualizations (and associated metrics) help characterise varied levels of AI interaction in writing from low to high dependency on AI.
Figure: CoAuthorViz legend and three samples of AI-assisted writing (squares denote writer written text, and triangles denote AI suggested text)
Full details of the work can be found in the resources below:
Is autonomy (self-writing, without AI support) preferable to better quality writing (with AI support)?
As AI becomes embedded into our everyday writing, do we lose our own writing skills? And if so, is that of concern, or will writing become one of those outdated skills in the future that AI can do much better than humans?
Do we lose our ‘uniquely human’ attributes if we continue to write with AI?
What is an acceptable use of AI in writing that still lets you think? (We know by writing we think more clearly; would an AI tool providing the first draft restrict our thinking?)
What knowledge and skills do writers need to use AI tools appropriately?
