Matt Fox

7

Can you trust course reviews on Udemy?

Just looking for a quick answer? Skip to the bottom line.

You know what sucks? Getting ripped off after trusting fake reviews.

Udemy has over 100k courses and most of these have 4+ star averages… but are these reviews really legit?

I’ll teach you when you can trust Udemy reviews, and when their full of it.

Udemy has some weird nuances with their review system that make it stand out from other popular sites like Amazon or Yelp. From when they ask for reviews, to who they let write a review, there is some shady stuff going on with their review system..

Most Reviewers Haven’t Completed the Course

This is definitely my biggest complaint with Udemy’s review system. It’s also my biggest criticism with the platform in general.

Imagine you’re a hot-shot movie producer. You just made a new movie and you wanted to figure out what your IMDB rating was going to be after it hits theaters.

So you packed a movie theater with 100 unsuspecting millenials and started the film. After 7 minutes, you paused the movie and asked everyone to rate the movie on a scale of 1-10.

That’d be pretty dumb, right? How can they evaluate it fairly after 7 minutes?

Well, Udemy doesn’t think this is such a bad way to collect reviews.

This screen appears after watching anywhere from 5 to 15 minutes of course content:

Since Udemy asks for a rating so quickly, a considerable amount of Udemy’s reviews are left by people that have nearly 0 first hand experience with the course

So, instead of reviews of a course’s quality, what you’re getting instead is an indicator of how enthusiastic that student is to be taking the course.

When people first start a course, they are excited and hopeful. This positivity bias results in lots and lots of positive 5 star reviews that have nothing useful to say about the course.

This is not a smart way for Udemy to be collecting reviews. It inflates the rating counts and it hurts the integrity of the review system.

This is precisely why there are so many crappy reviews on Udemy that say nothing specific or useful, like these ones:

Now, Udemy recognizes the fact that many of these reviews are not very useful, and they sort them last with their sorting algorithm. Udemy has a pretty robust sorting system so the most useful reviews appear first. They consider things like the length of the review, how recent it was, and they even consider how much of the course the student completed before writing the review.

These low-quality reviews end up near the bottom of the sort list. But they still have an impact on the overall course rating.

This flawed collection process results in lots and lots of courses that have great reviews but in reality they’re terrible. This is a significant problem on the platform, resulting in many 4+ star rated courses that are considerably worse than what you can get for free on YouTube.

SUMMARY

Udemy asks students for a review ~10 minutes into taking the course. This tactic results in lots of meaningless reviews.

How Instructors Cheat

Let’s say you want to sell your course on Udemy. You publish it and you start at the same place as everyone else – with 0 reviews.

It’s going to be incredibly tough to sell a course with 0 reviews. If only there was some way to generate tons of 5 star reviews quickly, so you have the “social proof” you need to start selling your course.

Fortunately for the course creator (and unfortunately for the rest of us), there is an easy way to mass generate 5 star reviews. Give the course away for free.

Udemy allows instructors to generate a specialized coupon code that makes the course free. No matter how bad a course is, Udemy students are pretty reluctant to give a course a bad rating if it’s free. The expectations from these students are low, and some of these folks may feel like they owe the instructor something for giving them the course for free.

Instructors rack up 5-star reviews by giving their course away for free. Then, they jack up the price.

When some poor sap in the future checks the reviews, there’s no way to know whether those reviewers were given the course for free or if they paid for it.

Amazon gives products away for free in exchanges for a review. It’s called the Amazon Vine program. But the difference is that Amazon places a note on the review letting you know that the reviewer received the item for free.

There’s no such disclaimer on Udemy.

Who cares if reviews are left by students that didn’t pay? Those reviews should still count, right?

The fact is, if someone PAYS for a course versus being GIFTED it for free, it completely changes their perspective when writing a review.

Let’s say I gift you a new gaming keyboard as a gesture of good-will. It works ok for a while but breaks after a month.

Are you going to start trashing me online because the free gift I gave you was low quality junk? If you’re like most people, you wouldn’t. This is because:

  1. Your expectations were low to begin with, since you paid nothing for it.
  2. You feel like you owe me something because I tried to do something nice for you. (In Psychology, this is called the Rule of Reciprocation.)

But what if you bought that keyboard from me? In this transaction, you paid a fair price for a product. You expect the product will last a reasonable amount of time. And you don’t feel indebted to me because it was a simple transaction of commerce where we both benefited.

In this scenario, you’re probably going to be (rightfully) angry, and you may just write a review giving me a piece of your mind.

Instructors are well aware of this phenomenon and constantly use it to generate 5 star reviews for new courses that they launch.

Here is an egregious example of course that was offered for free. This is a 4.3 star course that has been widely given away for free using instructor coupon codes. Put simply, it’s terrible, and it does not deserve its rating.

I can’t say for sure how many reviews were from freebie users, but listen to this 12 second audio clip (taken from this 4.3 star rated course) and you can decide for yourself if this sounds like the voiceover quality of a 4+ star course.

Can you imagine enduring that robotic voice for 2 hours? 

Now, on a cursory inspection, we can see that the course is very short (2 hours), the voiceover is… less than pleasing to the ear. There are dozens of java tutorials on YouTube that are more comprehensive and won’t put you to sleep in the first 15 minutes. And to top it all off, the starting price of the course (when not on sale) is $94.99.

This course is worth exactly what most 99% of Udemy students have paid for it: nothing.

The fact that courses like this have a high rating highlight this issue with Udemy’s current review system. Reviews from free users should be marked as such, and they should  have little to no impact on a course’s overall rating.

Sadly, there are thousands of courses on Udemy just like this one. The instructor gave away the course and generated lots of 5 star reviews due to low expectations from students and because Udemy prompts students for reviews at just minutes into a course.

SUMMARY

Instructors give away their course for free as a way to get lots of 5 star ratings, then they raise the price afterwards. 

How to Expose Fake Reviews

There’s a simple technique you can use to figure out which courses generated most of their reviews by giving the course away for free. These are courses you want to avoid, because they are almost always worse than what you can find on YouTube.

The formula:

Take the number of course ratings and divide it by the number of students enrolled. Take the resulting decimal and convert it to a percentage. You now have the percent of course enrollees that left a review.

If that percentage is lower than 5%, that course almost certainly was given away for free in massive amounts to drum up unearned 5-star reviews.

This formula works so well because a tiny % of people who take a course for free will bother writing a review. When people pay for a course, that percentage shoots up by a factor of 5x-10x.

Example – Low Quality Course

Let’s use this course as an example.

At the time of this writing, it has 5,146 ratings and 182,750 students enrolled.

Now we divide 5,146 by 182,750 and we get 0.028 (I rounded it.) In a percent, that becomes 2.8%.

At a review rate of 2.8%, it almost certainly has generated most of its reviews from freebie users. Save your money and find a course that earned its rating from paying customers.

Example – High Quality Course

Now, let’s try this method on a mega popular course that has sold tens of thousands of copies, like Colt Steele’s web developer bootcamp.

173,582 / 571,563 = 30.4%. That means the review rate is over 30%. A high review rate is a great sign, because the reviews are left by people that actually purchased the course.

Take any top instructor like Colt Steele or Rob Percival. Check any of their courses, and very rarely will you see their review rate drop below 10%.

SUMMARY

If less than 5% of enrolled students have left a rating, you should avoid that course because most of the students were likely given the course for free.

FAQ

Does Udemy Remove 1-star Reviews to artificially inflate course ratings?

The short answer is no, there is no credible evidence that Udemy is systematically removing poor reviews.

If you look around on the web, you will find dozens of instructors complaining about negative reviews on their courses (1, 2, 3) and very few students claiming that Udemy did not post their negative review.

Udemy’s official position is that they will only remove reviews if the review violates Udemy’s review policy which includes common-sense things like:

  • A student left a bad review because the student knows the instructor personally and is holding a grudge against the instructor.
  • Using the negative review as a thinly veiled attempt to promote another course. E.g “This course sucks, but you should check out the course I just made on the same topic, it’s way better”
  • The review focuses on criticisms that have nothing to do with the course. E.g ‘This instructor is ugly, I can’t spend another minute watching this.’

I have been using the Udemy platform since 2016 and when it comes to removing reviews, they tend to err on the side of protecting the student. They are more likely to remove a positive review that looks fake/suspicious than they are to remove a negative review that is reported for violating Udemy’s review policy.

SUMMARY

Udemy does not remove poor reviews to sell more courses.

How common are fake reviews on Udemy?

Take a look at this:

One of several facebook groups where instructors swap fake 5 star reviews of each others courses.

Like every other feedback driven online marketplace in existence, sellers are working to game the system and collect fake reviews to boost sales.

Thankfully, Udemy leverages an algorithm that automatically filters reviews that have a high probability of being fake.

Additionally, Udemy is very aggressive about banning instructors that buy and trade reviews. They consider this a ‘Major violation’ of their review policy and the instructor is almost always permanently banned if they are caught doing this.

Review manipulation on Udemy is a highly risky endeavor for instructors due to how devastating it is to have an account banned from the platform.

It takes weeks or months of hard work to create a decent course. Getting your course content banned is not easy to recover from.

On other marketplaces like Amazon or eBay, sellers can just create a new account to sell their stuff if they get banned for fake reviews.

On Udemy, they would need to create an entirely new course to do this, which is incredibly time consuming. Because of the threat of a permanent ban, very few established instructors risk buying fake reviews.

SUMMARY

Due to Udemy’s zero tolerance policy and aggressive enforcement, fake reviews are rare on the Udemy platform. 

Does Udemy have a reputation for honesty?

Udemy HQ is based in San Francisco, California, USA. They’ve been around since 2010 and they boast over 24 million students. They are the largest course marketplace in the world.

This is not some fly-by-night operation run out of someone’s home office. They’re a huge established company with millions of satisfied customers.

Although I definitely am not a fan of Udemy’s practice of soliciting reviews so early in the course taking process, they offer tons of excellent courses at affordable prices. They are aggressive about removing fake reviews and they are very good about honoring their 30 day money back guarantee.

SUMMARY

Udemy is a reputable company with millions of satisfied customers.

Does Udemy Sort Positive Reviews First?

Udemy’s official position on review sorting is that they use a combination of factors including review length, most recent, and how many people have marked it as helpful.

I also believe that Udemy sorts reviews higher when the student has completed a higher % of the course as Udemy has stated this is a factor for consideration when determining which review will be featured. If they are considering this for featured reviews, they are probably considering this for review sorting as well.

Positive reviews tend to show up first before negative reviews. I don’t know if Udemy is explicitly weighing positive reviews higher as a way to sell more courses. This could be because more people are marking positive reviews as helpful, or Udemy could be intentionally weighing positive courses more highly.

Regardless of why it’s happening,  I highly recommend filtering by 1-3 star reviews so you can get a balanced perspective when evaluating a course. See if there are any recurring criticisms, and consider whether that will be a problem for you.

The Bottom Line

So, can you trust Udemy’s reviews or not?

Inidividual reviews that are detailed and specific are usually trustworthy. These are generally reliable because:

  1. Udemy is aggressive about removing paid/fake reviews, so the review is unlikely to be paid for by the instructor.
  2. The specific info they are citing is evidence they have actually completed the course.
  3. On any review platform, most fake/spam/paid reviews are short and generic. Specific and longer reviews take a lot more effort and stand a much better chance of being real.

However, don’t put too much stock into the overall course rating. It’s not a dependable way to gauge course quality on Udemy because:

  1. Since Udemy hits the learner up for a review when they are 5-10 minutes into the course, many of Udemy’s user submitted ratings have no value. This makes the overall rating less reliable.
  2. Many Udemy instructors (especially lesser known instructors without many sales) give their course away for free to generate lots of 5 star ratings from people with low expectations.

Have you ever purchased a bad Udemy course? Let me know in the comments below. I personally respond to each and every comment, and I would love to hear about the experiences that you’ve had with Udemy’s courses.

How useful was this post?

Click on a star to rate it!

Average rating 4.8 / 5. Vote count: 5

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email