2019 Year in review

Welcome 2020! A new year is the perfect time to reflect on the past year, so I wanted to take a step back and think about it. 2019 was one of the most successful years for me professionally (and personally) with a range of experiences and productive outcomes. Quite a few achievements I’m really proud of happened this year. This post is mostly a note for myself to remind me of all those 🙂

I started the year on a positive note – I allocated quality time for me to do some coding for a novel graph analysis method I developed for writing analytics. Recovering from my laptop loss from the previous year (noting how important backing up your work is), I redid it from scratch, and made a version better than what I had last time. Coding up those interactive automated revision graphs was probably the first successful outcome in the year for me.

My biggest achievement this year was completing my PhD from the Connected Intelligence Centre. Even at the start of the year, I hadn’t started writing my thesis and I was still finishing up data analysis. Even when I started writing my thesis in February, I was unsure if I could complete it before the August deadline. The main chapter seemed like a monster job since most of the analysis had to be done newly and I hadn’t written it up before. The best decision I made at that point was to start off with this hard chapter instead of the starting or the easier ones, where I had already written stuff (like a lit review or an introduction). A pat on the back – I stuck with the deadline of completing it before I flew out to LAK19 in March- it was quite intense, both emotionally and physically taxing, but I made it! I emailed the first version of this chapter with an overall skeleton of the thesis to my supervisors when I was on a bus home – I was literally making use of every minute I had before flying out to the conference.

My participation in LAK19 was quite a success. I’ve written a whole post on it before, so I’m not gonna dive into details. But I presented a full paper and got some amazing comments, facilitated a workshop (almost solo since my co-organizers couldn’t make it at the last minute) and joined the SoLAR executive committee. I had received the ACM-Women in Computing Scholarship to attend this conference.

While I was writing the rest of my thesis, I applied for a Lectureship at UTS Faculty of Transdisciplinary Innovation and got it! I decided to go for this one over postdoctoral research positions to stay long term in academia. Searching and applying for jobs are such an ordeal and my skills were dusty; I’m super glad mine went smoothly since it was the only job I applied for, and the timing worked out perfectly.

I had to start the lectureship in July, which pushed my thesis submission deadline a month earlier. I couldn’t take a break after thesis submission, so I took a small break after sending out the full draft of my thesis to my internal reviewers in June. I went home to India for 2 weeks, which just flew by. I worked super hard to submit the final thesis after my return, to the point where I didn’t really want to take another look at it anymore! Finally, I submitted my thesis on the 25th of July, 2019.

I started lecturing right from the first month of me joining the Faculty of Transdisciplinary Innovation. It was pretty hard, truth be told, as I was trying to juggle between a few different things. First time teaching a subject from preparation to delivery, handling student queries, the admin, the mentoring, managing difficult students – it was a handful. I even dropped my plans to take part in the 3MT competition coz my schedule was so tight.

In the meantime, the reviews for my thesis came back. I passed with flying colours and the reviews were extremely positive, with appreciation of it being one of the best theses the reviewers had reviewed! Both reviewers accepted the thesis for publication without any changes. I did make some minor changes for final publication based on their comments and my degree was conferred on the 12th of November 2019.

I also got a few invitations (both internal and external to my university) to take part in events, which went really great. I was invited as a panel speaker at Intel, Sydney where we discussed ‘Artificial Intelligence Today for Our Tomorrow‘ with some great minds. I co-organised a workshop with our Faculty staff on Data in September for the Festival of Learning Design. I gave a short talk at UTS TeachMeet “The Future Starts Now“ in October,  hosted by the School of International Studies and Education, UTS – Video of Highlights here. I visited the Centre for Research in Assessment and Digital Learning (CRADLE) at Deakin University, Melbourne in October to participate as an invited delegate at the “Advancing research in student feedback literacy” international symposium – had good conversations and set plans to move the research forward in our upcoming work.

I received the Future Women Leaders Conference Award and visited Monash for two days in November for the conference, where there were a series of workshops and talks supporting future women leaders in academia from engineering and IT. I also created from scratch and published a podcast (Episode 3 of SoLAR Spotlight) that month – lots of learning happened in putting it together, from preparation to editing. I do have regrets in turning down some good opportunities that came my way, just because I was not having enough hours in a day to manage everything. But I guess it is a part of growing as an academic, since you prioritize and decide what is more important, and try to achieve work-life balance. In the end of November, I co-organized a workshop at ALASI. That was the end of work-related events in 2019, but the best was yet to come.

I went to India in December for my long-awaited wedding with my sweetheart. It was a big fat south Indian wedding, so lots of prep and stress, but loads of fun! Here’s a picture from the wedding 🙂

Notes: The calibration of student judgement through self-assessment

Reference: Boud, D., Lawson, R., & Thompson, D. G. (2015). The calibration of student judgement through self-assessment: disruptive effects of assessment patterns. Higher Education Research & Development, 34(1), 45-59.

Background:

  • Effective judgement of own work is an important attribute for HDR students
  • Focused on Self-Assessment (also represented as Self-regulation and Metacognition in few works)

Problem:

  • Self assessment is not facilitated using systematic activities, but rather thought to be achieved from normal tasks.
  • Students are not given feedback on their judgement.
  • Ensure that different assessment methods don’t distract students’ ability of learning and judgement.

Purpose:

  • To investigate if students’ judging improves over time through criteria-based self-assessment in the given units of study in two parts:
    1. Replicate improvement of student judgement over time with more data from different disciplines (Repeat questions 1- 4)
    2. Investigate improvement of judging skill over sequential modules and analyze based on assessment type, assessment criteria (New questions 1-4)

Method/ Context:

  • Voluntary self assessment of students in authentic settings using an online assessment system ReView™.
  • Percentage marks from continuous sliding scale stored – of both students and tutors.
  • Data from 5 year period in Design, and 3 year period in Business course from two Australian universities.
  • 182 design students, 1162 business students.

Results and Discussion:

Repeat Q1: Does accuracy in students’ self-assessed grades vary by student performance?

  • Ability level – High (distinction or high distinction), low (fail) and mid (pass or credit)
  • p values
  • Yes. Low ability students did not improve judgement over time. Significantly higher improvement for mid ability students (Students matched tutors grading)

Repeat Q2: Does students’ accuracy in self-assessment relate to improved performance?

  • Accuracy levels: Over-, accurate and Under-estimators
  • One way ANOVA
  • Accurate estimators had increased scores over time

Repeat Q3: Do students’ marks converge with tutors’ within a course module?

  • Series of paired t-tests
  • Convergence found in design data, but not in business data
    • Design assessment tasks are scaffolded to lead one to other, but business uses different modes of assessment within a course module.

Repeat Q4: Does the gap between student grades and tutor grades reduce over time?

  • Yes. Students’ judgement improves over the time of the degree programme, but not very useful as the convergence happens almost at the end of the degree programme.

New Q1: Does the gap between student grades and tutor grades reduce across modules designed as a sequence?

  • No data for design and reduction in gap (erratic patterns with no gradual reduction) between student and tutor grades for business with sequential modules
  • Leading to examine the type of assessments

New Q2: Does mode/type of assessment task (e.g., written assignment, project, and presentation) influence students’ judgement of grades?

  • Data was inconsistent, despite showing earlier convergence for few assessment types. Most assessment types saw convergence in iterations 2 or 3 (Refer Table 1 from the original paper).
    • Could be due to difference in tasks within each assessment type

New Q3: Does analysis of criteria that relate to type of assessment task influence students’ judgement of grades?

  • Consistent and related criteria for particular assessment type fosters faster calibration in iterations 1 or 2 (Refer Table 2 from the original paper).

Limitations:

  • Criterion is provided for assessment since students are not experts, however a holistic evaluation of own work is recommended.
  • Not possible to identify the cause for improvement from independent measures.
  • Whole population of students is not included, especially the less engaged students who might be low achieving.
  • There might have been other informal factors (not measured in this study) to influence the results like the following: comments received from staff, discussions with peers, and students’ own aspirations.