September 19, 2018

How to Prove the ROI of Computer Vision Moderation

Table of Contents:

While other forms of AI are still in their infancy, one field of AI is already in practical use by businesses: computer vision. Computer vision (CV) has already shown itself to be worth the investment for a variety of companies from online marketplaces to social media platforms that looking to reduce costs, increase efficiencies, and improve customer satisfaction. Where these businesses rely on user-generated content (UGC), computer vision is especially valuable as it enables them to better filter out content that is irrelevant, inappropriate, or not of the desired quality.

Recently, we published a free kit, How To Measure the ROI of Computer Vision Moderation,” which includes both an ebook and ROI calculator. In the ebook, we outline the different ways businesses have benefited from automating moderation while the calculator enables you to input your own budgets to understand the potential financial impact CV moderation can have for your business.

 

In this post, I’ve built a simple business scenario showing you just how beneficial investing in computer vision for moderation can be and to provide an example of how the ROI calculator can be used for your own needs. Let’s dive in!

 

The Scenario

Company X is an online marketplace, offering its customers a platform on which they can sell their clothes. 

kris-atomic-73939-unsplash

To get started, users first upload one or multiple pictures of what they intend to sell. Then, they can write a description of the clothing item and either create or select a pre-existing tag for it. Finally, the user decides on a price and can now post their listing.

 

Company X’s policy dictates that users are only allowed to sell slightly used clothes and accessories on the site. However, they have found users attempting to sell books, electronics, and even used makeup. Further, to facilitate organization, the company knows that high-quality images can make a sale up to three times more likely. Still, users have been found to post pictures that are too blurry, dark, or otherwise low-quality. To alleviate these issues, the company has now hired a team of 4 moderators to moderate the user-generated content and ensure only the best and relevant images get posted to the sites.

 

The Numbers

  • Company X receives approximately 150,000 unique images a day.
  • 15% are irrelevant or low-quality content (i.e. 22,500).
  • It takes moderators 10 seconds to look at each image to determine whether an item is appropriate or of the desired quality.
  • Moderators are paid $12/hr for their work.

For simplicity, we’ll assume the following:

  1. Moderators work for 8 hours a day.  
  2. In that time, the moderators take no breaks, working non-stop for the entire 8 hours.
  3. The moderators make no mistakes, correctly identifying inappropriate content and/or labeling the uploaded content.

Let's do some calculations: 

ebook-roi-notebook-charts

If it takes each moderator 10 seconds to view an image and there are 28,800 seconds in their workday, in a perfect world, each moderator would view 2,880 pictures per day.

That means, even if we assume the moderators work non-stop for the entire day and make no mistakes, only 11,520 (or less than 8%) of the 150,000 images uploaded to the site every day are going to be moderated.

 

Now, Company X has three options:

  1. Increase their human workforce by hiring more moderators: Let’s assume all moderators will be able to view 2,880 images a day. To fulfill their daily moderation needs (i.e. moderating 150,000 images), they would need to hire approximately 48 more moderators. At $12/hr, each moderator will earn $1920 per month. Company X’s currently spends $7,680 a month for 4 moderators. Increasing their moderation team by 48 people will require a 1200% increase for their salaries alone.
  2. Develop a method for having to moderate fewer items: This might mean only moderating listings flagged by users as being inappropriate, irrelevant or low-quality. However, this could mean a lot of listings that should be flagged falling through the cracks. Having more irrelevant listings will clutter Company X’s marketplace, making things even harder for their users to both sell and buy on the site. Similarly, low-quality images will also result in a poor customer experience that may see them losing users to a competitor.
  3. Invest in technology automate the moderation process: By implementing a technology such as computer vision into their workflow, they would be able to moderate over 90% of the content uploaded to the site every day without having to increase their human moderation team or reduce their moderation quality. This would allow them to have a marketplace that shows only relevant listings, making it easier for worthwhile posts to make it front of buyers.

 

Plus, by filtering out low-grade content, buyers will be able to view images from which they can decipher all the appropriate product information (such as patterns or colors). Sellers, meanwhile, will be better able to make sales on the site since customers are more likely to buy from them when they post the high-quality visual content.

 

But what about the costs?

Using our ROI Calculator, let’s see how many returns Company X would get if they invested in a great computer vision platform for moderation:

Screen Shot 2018-09-19 at 12.44.50 PM

 

The Results

After investing in computer vision, Company X’s moderators have now regained almost 6 hours of their workday back. The improved user experience can then lead to an increase in users, so some of their moderators have been transferred to Company X’s customer support team. This gives Company X the opportunity to further differentiate themselves from the competition by offering a more personal customer service experience to their users.

rawpixel-733989-unsplash

Company X is now able to have a small team of full-time moderators who focus on more specialized issues related to Company X’s line of business. These issues may include identifying counterfeit merchandise that users may try to sell as authentic or training the computer vision model to tag and moderate even more accurately. This kind of specificity has helped Company X to become the top marketplace for fashionistas devotees and trendsetters looking to buy and sell their merchandise.