Introduction

Background

About 9.8% of undergraduate students in computer science or programming majors drop-out before finishing their course every year. This high drop-out rate of students is associated with the difficulty and critical thinking required to gain knowledge as compared to other majors. Also, the percentage of students failing in computer science or programming courses is relatively very high.

Problem Space

It is hard for instructors to provide needed in-person guidance to each student. Many online tutorials help understand the basics but do not provide the practical knowledge required to code a solution. Also, other resources such as Stackoverflow, Github, etc. are not beginner-friendly.

Design Process

I collaborated with two of my classmates for this project in researching and designing . The higher-level UX design process we followed is represented in the sketch below.

Research

User Interviews

To understand more about the teaching techniques, areas of improvement in undergraduate programming education, we interviewed 12 students, 2 professors, and 2 Teaching Assistants. We tried to understand the fundamental problems of a programming class and why some students do not score high enough in the assignments to pass the course. Some of the insights and comments from the interviewees are listed below.

Peer-to-peer programming

"I have trouble translating the theory and class exercises we learn to actual code. I can do better if I have some help."

"My friend helps me with the assignments all the time. He usually points me into the direction I should be going."

Overall performance report

"I want to see the students’ performance to the rubric level. This can help me create a proper plan of action."

"Having a higher level view of the grades of overall class can help me understand where they need my help the most."

Sharing code with other students

"If I am not with my friend, I send him my entire code and he runs it in his system to see what’s wrong. This takes time."

"I usually wait until TA hours to get a review. That is the only time I can get someone to take a look at my assignment."

Code history

"If I can see the all the steps the student has taken in writing that piece of code, I can understand where they had trouble."

"Many students don’t understand the rubric. Every time I release grades, they come and ask me where they lost points."

Alternate code

"In the beginning, students usually do not write efficient code. They need practice and help to get better at coding."

"I always provide solution code after every assignment. Using that, students understand an alternate way of solving the problem."

Desktop Study

We found many insights which corroborate our primary research findings. We realized that programming language or skill can become easier to master when learned collaboratively with other students. Also, having access to the entire history of code changes would result in understanding the students’ thinking process and hence better code reviewing. We also realized that all the collaborative programming tools available currently focused on experienced programmers only, and there are no tools or support platforms that help students.

Analysis and Insights

Using the Affinity Diagramming technique, we organized all the information collected from interviews and literature review to create patterns and insights as below

Peer-to-peer collaboration among students can significantly smoothen the learning curve of a programming language.

Usually, students like to help each other in finishing assignments, rather than going to professor or a teaching Assistant.

Students have difficulty in translating their theoretical knowledge to practical solution.

Each student is unique and the problems they face are also unique.

Feedback on the assignments are usually generic to the whole assignment, rather than specific to a piece of code.

A student need to know different ways of solving a problem to become proficient in a programming language.

Sometimes, professors have difficulty in understanding the exact problem faced by a student to offer relevant help.

Instructors need to know the performance of students to the rubric level to provide proper support to the students.

Problem Statement

The Affinity Diagram transformed the raw data into information we could use to begin framing our guiding problem statement. Through these insights we defined our design direction to focus on the user’s most pressing needs. The problem statement we followed is:

Ideation

Personas

Based on the user research we conducted, we created three personas we thought would represent our target users as shown below.

Brainstorming

Using the insights gathered so far, we brainstormed numerous ideas that could address our problem space. We then started filtering them based on technical, economic, and feasibility. We combined the best parts of some ideas so that they serve the purpose of helping students and instructors in a programming class. We tested and analyzed many ideas to decide on the final design solution.

Our final design solution is to build a code editor that allow students to write and share their code with other students for review or submit to the instructor to get feedback. The reviewers then comment on the student's code with the help of code's history. When students share their coding assignments with instructors, they receive grades and comments tagging the section of the relevant code in the editor.

This code editor was incorporated as an extension of Canvas, which is a Learning Management System (LMS) that most of the universities in the United States use. Since Canvas provides APIs to build third-party extensions, we were sure about our design solution's feasibility.

Storyboards

We created storyboards around the problem scenarios that the students and instructors would use the identified solutions.

Scenario 1: A student is getting help from his peers.

Jakoby wants to improve his knowledge in coding

Jakoby finds the code too difficult to write

Jakoby invites his friend to help with the code

His friend suggests him to share his code for review

His friend is effectively able to guide him

Jakoby feels that he learnt something useful.

Scenario 2: Student and Instructor work together to resolve doubts.

Dr. Ryan receives an email from Jakoby for a doubt

Dr. Ryan asks Jakoby to share the code.

Dr. Ryan receives a link to Jakoby's code.

Dr. Ryan helps Jakoby by coding collaboratively.

Dr. Ryan is happy that he could guide Jakoby.

Scenario 3: Student sharing code context with the instructor

Dr. Ryan came across an issue with Haylie's work while grading.

Haylie has a hard time understanding code and finishing the assignment.

Haylie approaches her classmate for a solution to her problem

Haylie meets Dr. Ryan, and he suggests her to use the new code editor.

Haylie started using the new code editor for her next assignment.

Haylie submits her work to Dr. Ryan and learns that she had incorrect steps.

Scenario 4: Professor identifying the student’s trouble.

Dr. Ryan is looking at Haylie's code history and making sense of her thought process

Haylie received a detailed feedback on her assignment with alternate code suggestions

Dr. Ryan realizes that a lot of students have similar issue and teaches the concept again in class.

Design

Low-Fidelity Prototype

We used a paper prototype to perform an initial evaluation of the product to understand the usability of the product. We created paper prototypes to provide tangible interaction with the features we had explored. The paper prototype focused on the MVP screen of the application with the core functions.

Student portal's code editor

Grouping code blocks to access features

Add code context or view code history

Adding context, rationale to the code

Sharing for collaboration or code review

Viewing history of a code block

Overall code edit of the project

Instructor's grading portal

Rubric grading for each code block

Feedback / Alternative code suggestion

Initial Usability Evaluation

After creating paper prototypes, we began evaluation right away. We used techniques such as Cognitive Walkthrough and Heuristic Analysis to evaluate the usability of our paper prototype. Based on the results of the evaluation, we made changes and iterated them. Most of the changes we made are of changing the layout of the editor and not the functionality of them.

High-Fidelity Prototype

We used Figma for designing high-fidelity screens and also for prototyping. In this stage, there were several iterations of the screens based on the workflow and the usability of the platform. Below are final screens we designed.

Splash Screen

Student platform's journey

Student's Sign In

Student's Dashboard

Select a line to access addition features ('+' icon)

Code block generation with access to features

Adding context to the code

Option to view Code History for a code block

Requesting other student or professional of review

View of the feedback received from the Instructor

Instructor platform's journey

Instructor's Sign In

Instructor's Dashboard

Instructor's grading screen

Rubric grading pill upon highlighting code

Selecting the predefined grading rubrics

Adding grade to the code block grading pill

Adding Comment/Feedback at rubric level

Adding alternative code to give a different perspective to the student

Evaluation

User Testing

After creating high-fidelity prototypes, we began working on usability testing. Six potential users and three user experience designers participated in our usability testing and evaluation sessions. We created three tasks with the editor and performed think-aloud usability evaluations with the users. Later, we asked the participants to fill a post-task questionnaire to understand their experience.

Results and Findings

The overall experience of the app was well received by the users. However, several areas of the app were identified which needed further attention.

Critical Issues

1. Some students expressed concern about using code history. They think it is difficult to understand and analyze the code as they are also beginners.

Minor Issues

1. UX experts expressed concern that there could be a lot of training involved to onboard the new users to use the editor.

Positive Feedback

1. Student participants who have used the product felt that the editor can definitely help them in learning programming skills easily.

2. Teaching Assistants liked the idea of selecting a code block and grading it using the rubric. They felt that it would help students understand feedback at a granular level.

This usability study confirmed that the basic usability aspects of the application are sensible and functional. The evaluators felt that the application is consistent in terms of design which they liked. They also said that the application would definitely solve the problem to some extent.

Reflection

Next Iteration

The current application is designed as a web based application, going further in the development we would like to integrate into the existing canvas platform for education where the students use it as a mandatory service for all the coursework. Apart from this we would like to add a few features to the existing MVP to make it more efficient.

1. Support for collaborative work for students to work in groups

2. Adding professionals as mentors for helping the students

3. Generating in-depth reports to understand the student's problem areas.

Key learnings

Involve potential users in design

Involve the potential users in the low-fidelity phase to validate the ideas. This involvement could eliminate any biases that the designer might have while creating a product. It saves the energy and time of changing high-fidelity prototypes later.

Early validation of concepts

After brainstorming, it is very important to validate and test the selected design solution in order to see if it addresses the problem statement properly and works as expected or not.

Challenge

Project Specifics

Introduction

Research

Ideation

Design

Evaluation

Reflection

Home

Next

Let's Connect!