yumiao-zhang profile photo

I am an empathy-driven UX designer dedicated to crafting human-centered digital experiences that make a real difference in people's lives. I thrive on challenges and finding creative solutions, with a strong focus on design details to ensure seamless user journeys.

Exploring the Impact of Conversational AI Behaviour on Cognitive Engagement in Foreign Language Learning

This project investigates how different levels of proactivity in conversational AI chatbots affect cognitive engagement, emotional reactions, and overall learning experiences in foreign language acquisition, providing insights to optimize AI-driven language learning tools.

Design process

The design thinking process for this project followed a user-centered approach, starting with a focus group to deeply understand user needs and challenges in language learning. Based on insights from user research, task scenarios were developed, and multiple prototypes were iterated upon through evaluations. Finally, a controlled user study was conducted to assess the impact of different AI chatbot behaviors on language learners’ cognitive load and emotional reactions

Research aim and questions

  1. How does AI chatbot’s initiation strategy (proactive, hybrid and reactive) impact on language learner’s cognitive load?
  2. How does AI chatbot’s initiation strategy (proactive, hybrid and reactive) impact on foreign language learners’ emotional reaction?

Prototype development

By formative evaluation I turned to use a customised version of ChatGPT as a testing tool. This allowed ChatGPT to respond dynamically to user input without predefined constraints.
I employed prompt engineering techniques, crafted specific prompts based on three initiation strategies – proactive, hybrid, and reactive – and provided detailed scripts to guide the chatbot’s responses.

User Testing

For study design, I used a within-subjects controlled design, involving 22 non-native English-speaking students from Loughborough University. Participants interacted with three chatbot prototypes, and the order of exposure to the 3 prototypes counter balanced to avoid the effect of learning transfer effect. Then they will be asked to fill the Paas and PANAS scales.

Testing Result

Cognitive load:

Pairwise comparisons showed that the proactive chatbot significantly reduced cognitive load compared to both the hybrid (p = 0.006) and reactive prototypes (p < 0.001). However, the difference between hybrid and reactive was not statistically significant (p = 0.059).


Emotional reaction:

For PA scores, the normality test allowed me to use ANOVA, which showed significant differences between the prototypes. Both Proactive and Hybrid chatbots significantly increased positive emotions compared to the Reactive chatbot, However, there was no significant difference between Proactive and Hybrid, which indicates that both approaches are effective at boosting user motivation and emotional engagement.

For NA scores, by non-parametric tests, I found no significant differences between the prototypes. This indicates that none of the chatbot designs heightened negative emotions.

Other Works

EcoGo

Based on the brief provided by GenGame, EcoGo is a mobile app concept designed to help UK households using solar panels efficiently manage their energy consumption. By integrating smart solar panel data, the app aims to actively engage families in climate action, promoting energy saving and reducing emissions to combat climate change.

SkySports Kickstart

Sky sports Kickstart is a redesign project for Sky, the UK’s leading digital TV operator, aimed at revitalizing Sky Sports’ digital products. Currently dominated by male users aged 35+, the project seeks to broaden the audience by enhancing inclusivity and appealing to younger, diverse fans, especially women and new sports enthusiasts.

Healthy Sip

Based on the UN Goal of good health and well-being, Healthy Sip aims to tackle health issues related to excessive drinking. It provides UK drinkers with a virtual reality experience where users make choices in a sober state and feel the effects of intoxication in a simulated environment. The project also includes a mobile app that tracks drinking habits and health, encouraging long-term behavior change.