Mastering E-Learning Analytics

Summary

E-learning platforms generate a lot of data: making sense of all of it in order to improve your own platform can be really tough.

The following white paper details the steps that you need to take in order to set up and leverage a learning analytics strategy for your e-learning business. By personalizing your e-learning platform for different user segments, you will be able to improve user engagement and stay ahead of the competition.

Introduction

Recently, UNESCO published a report that detailed how learning analytics promises ““to transform educational research into a data-driven science, and educational institutions into organizations that make evidence-based decisions.” Over the last few years, a lot of these measures have already started to take place.

The role of big data and analytics in the e-learning business has, in fact, profoundly changed how education is administered. By having access to a wealth of behavioral data, some of Cooladata’s customers have been able to personalize learning and boost user engagement.

In this white paper, we’ll discuss the step-by-step guide to wielding the true power of learning analytics to optimize user engagement, improve learning, and make your e-learning business the best that it can be.

We’ll start out by discussing the importance of data unification for behavioral and learning analytics. By unifying your data into a data warehouse and tracking the right metrics, you’ll be able to leverage learning analytics in powerful and targeted ways.

In the next section, we’ll talk about the behavioral metrics that every e-learning business has to track. Tracking and segmenting based on behavior can lead to higher-quality interpretations of what keeps your users engaged and learning well.

Along the way, we’ll discuss one of Cooladata’s success stories — ABA English — and how they were able to use behavioral analytics and a data warehouse in order to significantly improve user engagement. We’re sure that by following the steps featured in this guide, you’ll be able to gain new insights into how you can leverage learning analytics for your own e-learning business.

What is Learning Analytics?

Learning Analytics

Learning analytics is used whenever a business tracks, measures, analyses and reports on the data that learners produce when they engage with your e-learning service. At it’s core, learning analytics is a specialized version of behavioral analytics — the difference, of course, is that learning analytics is targeted towards services that specialize in education.

What all of this means, is that by leveraging your behavioral information and making judgments about whether or not certain behaviors lead to certain outcomes, you can begin to answer more complicated questions about how learning happens when users engage with your software.

Learning analytics can also show you which methods are more effective for different segments. If you’ve unified all of your data, you will be able to personalize your service and significantly improve how your users go through the learning journey.

Unfortunately, learning analytics features a lot of moving parts. By drawing on many fields — business intelligence, web analytics, operational research, and data mining — learning analytics can be tough to grasp all at once.

The three sections of this white paper break down the process of setting up and optimizing a learning analytics solution. It is also helpful to mention that Cooladata’s services are optimized for providing a quick and easy data unification and analytics solution for small teams.

1. Unify your data

The first step in improving your e-learning analytics is unifying all of the data that you have already collected.

Like any other SaaS web service, a learning platform depends on many different data sources to garner insights about their users: Google Analytics for website traffic, MailChimp for email outreach, and various ad platforms (Google, Facebook) for acquisition. Needless to say, things can get pretty messy.

Data Warehousing

Data warehousing is probably the best solution to unifying all of these sources into a coherent and comparable stream of data. By putting all of your information into one place, all of the teams in your company — product, marketing, communication, etc. — can analyze and draw insights from your learning and acquisition metrics.

Unfortunately, data warehouse can sometimes take months to set-up from the ground up. We’ve already written about the difficulties of setting up your own data warehouse and the prices associated with building your own solution.

Unify Your Data

Luckily, Cooladata offers a fully-managed data warehouse service, which takes care of all this for you. You stream your data to it, Cooladata assembles all the puzzle parts into one unified view of your business automatically, and you can spend your time asking questions and getting answers while optimizing your product and marketing in real-time.

ABA English

One of our e-learning customers, ABA English, did just this: by moving their user data from their back-end CRM to Cooladata’s data warehousing solution, ABA English gained a wealth of access to new behavioral metrics. These metrics gave them plenty of new insights into user engagement and learning that they simply could not access before.

2. Six Key Learning Analytics Metrics

Six Key Learning Analytics Metrics

Having a data warehouse solution in place is only half the battle. Now, you need to fill your storage solution with meaningful metrics that will help you understand how users learn through your platform.

After all, this isn’t just our take on the issue: many academic studies in recent years have studied the importance of optimization in e-learning. As Liang et al. point out, personalized learning through user profiling is essential in any data-driven e-learning service.

Below is a list of a different variety of behavioral metrics that we have found to be useful for measuring and optimizing user engagement. If you track the metrics below, you will more efficiently understand what your users are — and aren’t — doing on your e-learning platform, making it easier to adapt to their needs and values.

1. Time spent on your platform

Let’s start with the first, most obvious metric: time spent on your platform. Knowing how much time your user spends learning is the measure of your success, so keeping your hand on this pulse will be the way that you evaluate the success of your other optimization efforts.

Attempting to boost your time spent metric can also have huge benefits for the rest of your metrics as well. The more time your user spends on your site, the more behavioral information they generate, allowing you to respond and to adapt to how they use your platform.

2. Page exits

While many business intelligence analysts focus extensively on the moment of churn, by which we mean users that stop coming back altogether, a more useful metric is the amount of times a user has suddenly exited a learning session.

An incomplete learning session can be useful for finding out where users are running into trouble. If page exists become common in a certain lesson, that probably means the lesson is flawed or there are technical problems.

Frequent page exits can also mark users that are frustrated and ready to churn for good. Being able to find out why those users are dissatisfied and commonly quitting their lesson could help your retention strategy.

3. Premium subscriptions and renewals

Segmenting your user population by premium and non-premium users is a great way of finding out how to convert the latter into the former.

ABA English correlated their user engagement data across these two categories and found out the engagement categories that were leading users to higher conversions for the premium plans.

4. Clicks and transitions

If you know how or why learning transitions are failing, you are already ahead of the customer.

Where users click and how they interact with your platform can also show you what’s working well along with what isn’t. Are your users clicking in the right places? What button design works best for inputting answers? Which ones don’t?

Being able to answer these questions can greatly affect user experience and improve the educational metrics of your applications.

5. Learner proficiency

Are your students being placed in their correct proficiency categories? Or is the material too hard? Or is it the reverse — maybe the material is too easy?

Knowing the strengths and weaknesses of your own students should be part and parcel of your tactics for placing them in the right category. Your users need to be engaging with material of the right difficulty in order for them to be engaged properly and sustainably.

6. Learner progress

Segmenting your users by the progress that they’ve made through your application is vital if your learning platform uses leveling or gamification to track learning outcomes.

Learner progress is a unique percentage score (0-100%) of how well your user is progressing through your service.

Knowing when and where your users are learning is useful as well — is there a particular lesson that is better at getting information across than other one? Your user engagement and learner progress metrics can show when and where users are getting stuck.

Overall, this is an important metric to for any educational service to measure.

The takeaway

The six behavioral metrics above are a strong first start for any learning analytics strategy. They are, however, just that: a first start — any good analytics approach will find ways to customize the ways in which they measure user analytics.

By unifying your data and tracking many different behavioral metrics, you truly segment and personalize your educational learning platform for maximum effects.

3. Personalize learning with user segmentation

ABA English’s learning strategy was most successful when they were able to measure and track user behavior and then personalize their service to certain segments of their user base.

A fully-operational data warehouse, loaded up with some of those great metrics that we mentioned above, should give you access to a wealth of data that can be transformed, compared, and segmented for actionable insights.

In this section, we’ll discuss some ways in which you can properly segment your educational metrics in order to improve user engagement.

Measure learning engagement

As Betul C. Czerkawski points out in “When Learning Analytics Meets E-Learning” learning analytics uses pattern analysis and social network analysis in order to measure how engaged a student is with the content that they’re learning.

By tap into and compare total time spent learning, learning completion, and user page exits, you will already have a solid estimate of how engaged that user is with their learning.

Ultimately, the resulting metric will be called user engagement: a rough score that estimates how much value your users are getting from your product.

User Engagement

Engagement is incredibly important for any e-learning business. If your user is not engaged, they’re not learning, which means that your own value is at risk.

If you clump together the learning metrics outlined above and then segment your user base by their engagement scores, you will be able to draw insight from two groups: those that are engaged and happy, and those that are about to churn.

You can then cross-reference these insights. What are your engaged users doing that the unengaged ones aren’t? How can you get as many users as you can into the green?

Event-based analysis

While separate metrics can be useful at getting a bird’s-eye view of how your users fare with your learning application more generally, event-based analysis unites the whole learning journey and allows you to find out where your users are experiencing difficulties.

ABA English used Cooladata’s services in order to track and analyze all click and transitions, which allowed the company to adjust their learning units to prevent users from getting stuck.

Event based analysis

Cooladata allows you to quickly and easily drill through certain behavioral events and create cohorts out of manually defined groups.

The beauty of this solution is that it can tell you so much about what your users are and aren’t getting. Are the users that search a particular question finding the right answer? Is your search algorithm working correctly?

Personalized learning: the final frontier

As Ryan Baker points out in “Using Learning Analytics in Personalized Learning,” individualized learning has been the goal of teaching theory since at least the 1920s.

While individualized learning is still out of reach for traditional learning institutions — such as universities, colleges, and schools — adaptive online learning systems have been bridging the gap.

Personalized Learning

Baker points to one specific way that a system to adapt to student needs: software can make learner knowledge assessments and customize the difficulty of the software for the user. One specific example of this in action is “Cognitive Tutors for Algebra” which assess how well a student knows the material and then adjusts the difficulty and the content based on which questions students get right and which ones they get wrong.

If you assess your users using learning analytics, you will be able to powerfully and flexibly adjust your platform to accommodate different learning skill levels and expectations.

Conclusion

Building up a learning analytics solution isn’t easy, but the work, in our experience, pays for itself.

ABA English is a great success story, for exactly that reason. By investing into a Cooladata data warehouse and analytics solution, ABA English was able to fix their mobile conversion and identify, real-time, certain bugs in their software.

User engagement went up, and ABA English was finally able to make sense of all of their data.

We hope that the following white paper has been an insightful dive into the ways in which the data unification and learning analytics can help you customize your end-user’s learning experience.

Share this post

Leave a Reply

Your email address will not be published. Required fields are marked *