Introducing Landscape Quality, Portrait Quality, and Textures & Patterns Visual Recognition Models

Being in the business of computer vision, we deal a lot with photos – good and bad. But what makes a photo “good” vs. “bad” – the composition? The lighting? The way it makes you feel? We decided to try and distill those elements into an algorithm to help photographers and media managers sort through large volumes of content to find the highest quality images and video.  

Being in the business of computer vision, we deal a lot with photos. These photos can range from selfies using a cell phone camera, to computer-generated images created by designers, to professional photographs using high-end DSLR cameras. Our broad range of models helps computers understand “what’s in an image” and “where the object is located in an image”. For the first time, we’re releasing models that help computers understand image quality, or “is this image good or bad?” We are happy to release Landscape Quality Model and Portrait Quality Model into Beta which understands the quality of an image, and responds back with the confidence level of whether an image is “high quality” or “low quality”.

Good quality photo attributes:

  • Good lighting
  • Sharp and in focus
  • If retouching is present, it is not obvious (no completely airbrushed skin)
  • Not too much grain/noise (**unless it’s the artist’s intention)

Poor quality photo attributes:

  • Severe chromatic aberration
  • Red eyes
  • Extremely backlit
  • Unnatural vignetting, often digitally added

“With our computer vision capabilities, we want photographers to focus on what they do best: capture amazing moments.”Week

Professional photographers and even photography enthusiasts can take thousands (if not tens of thousands) of photos on a daily basis. They would then go through each photo and decide on whether or not that picture is worth post-processing or not. Assuming a photographer takes 5,000 photos in a day, and spends 10 seconds to figure out whether or not the photo should be post processed or not, this filtration process could take over 13 hours for one day’s worth of photos. With our computer vision capabilities, we want photographers to focus on what they do best: capture amazing moments.

Speaking from personal experience, I was overwhelmed by the number of photos I had on my camera after my wildlife photography trip to Nairobi a few years ago. To this date, I still haven’t had the chance to go through every single image to filter high quality vs. low quality photo. Speaking to some professional fashion photographers, they spend hours manually going through each image that they’ve captured during a ramp show or a photoshoot. “Having computers make the initial pass at filtering would save me tens of hours on a weekly basis”, said Kimal Lloyd-Phillip, a photographer for Toronto Women’s and Men’s Fashion Week.

“Having computers make the initial pass at filtering would save me tens of hours on a weekly basis.” – Kimal Lloyd-Phillip, photographer for Toronto Women’s and Men’s Fashion Week

Our Developer Evangelism team hacked away at these models and created a tool that would group your photos within a folder into two separate folders: good and bad.

In addition to the Landscape and Portrait Quality Models, we are also introducing a Textures & Patterns Model that helps photographers, and designers identify common textures (feathers, woodgrain), unique/fresh texture concepts (petrified wood, glacial ice), and overarching descriptive texture concepts (veined, metallic).

We have partnered with a global consumer apparel manufacturer to integrate the Textures & Patterns Model into their design workflow. They are using Textures & Patterns model to inspire creativity amongst their designers, and further develop their design ideas. They indexed their design database (internal and external images) using our model; they then inputted any new, raw design ideas into our platform, ran our Visual Search tool to explore and discover various ways the design could evolve.

We’re excited to apply artificial intelligence to the arts and provide tools that would empower creators to be more effective at their work. We hope our broader customers enjoy using the new set of models as much as our initial testers did. If there’s any feedback and/or additional requests, feel free to shoot us a message at feedback@clarifai.com.


Workflows: Clarifai's improved Custom Training gives you more accurate predictions with fewer labeled images required

We’ve made improvements to our knowledge transfer algorithms and neural network architecture, so all your Custom Models have been automatically upgraded with greater accuracy. And, we’re now allowing you to build Custom Models using knowledge from more of our base models (not just our General model) with a new feature called ‘Base Workflows.’

Last year, we launched our Custom Training product which empowered our customers to create their own visual recognition models without having any background in machine learning. Since then, customers have built thousands of models for their businesses and passion projects, and we’ve been learning how to make Custom Training better from each and every one of you. That means we’ve been working hard on finding ways to improve Custom Training such that users are able to get more accurate prediction results with fewer labeled images.

So, without further ado, we’re excited to announce two new features that will make you love Custom Training even more.

Receive more accurate results for your Custom Model

Clarifai uses convolutional neural networks (CNN) for visual recognition. Using CNN, we have built a large collection of base models to classify objects in various domains. Building a base model requires millions of labeled inputs, hundreds of training hours on high-performance machines, and many experiments conducted by machine learning researchers. In the end, a base model is produced that is accurate enough to be trusted by our users.

Building custom models using the methodology defined above is not pragmatic for a non-AI company due to the data, time, and resource requirements. This is why we decided to use knowledge transfer from our base models to build the Custom Training product. Our knowledge transfer methodology uses the knowledge that was developed during the base model creation process and builds additional knowledge from all the feedback received from usage. This knowledge base is what allows our users to create an accurate custom model with just ten labeled images.

In our initial launch of the Custom Training product, we were using knowledge built from our initial base models. By combining the cumulative knowledge from all of our other sources, we have improved our Custom Training product to deliver higher accuracy models. The changes have been applied automatically to the platform, so starting right now, users should see a 19% increase in accuracy, or an average of 0.2 increase in both Precision and Recall for their custom models.  Go ahead, check out your prediction results in your custom models!

Increase accuracy of your Custom Model by choosing a Base Workflow

Since we released Custom Training, we’ve been using our knowledge base from our most comprehensive public model – General. Our General model is still the most comprehensive general recognition model in the market, covering 11,000+ concepts trained over millions of labeled images. Therefore, knowledge transfer from this model generally works well for most custom models that are built on top of it.

However, we’ve found that sometimes users want to build a specific Custom Model that has a closer match to the knowledge base from another base model. For example, a user might want to build a Calorie Recognition Model from our Food model. We are now allowing our users to modify the Base Workflow* field to select a model, which is a closer representation of the model that they would like to build. By selecting a Base Workflow that is visually more similar to a user’s desired model, the accuracy of the custom model should increase, and the required number of labeled inputs should also go down!

For more information, you can refer to our Documentation Guide. Or, you can let us know your feedback at product@clarifai.com!

*A Workflow is a new set of features for Clarifai that empower users to be more flexible with our API and feature set. Read more about Workflows here!


Workflows: Use multiple visual recognition models at the same time, with a single API call

Clarifai has quite the collection of visual recognition models you can choose from – general, apparel, travel, demographics, moderation, etc. But what if you want to use more than just one at a time? Introducing Workflow Predict, our latest feature that allows you to save time and reduce latency by using multiple visual recognition models at the same time in a single API call.

Every day, we learn more from our customers and build features to address their needs and extend their capabilities. Most Clarifai users like to take advantage of more than one of our visual recognition models, so we’ve made it easier than ever for you to do so with our new Workflow Predict feature.

Workflows encompass one or more Public or Custom models. You can make a Predict operation call on a Workflow endpoint, which will provide results from the models that are contained inside of it.

Workflow Predict can significantly reduce the number of API calls that you have to make, since the same results will be available by making a single API call. This enables your apps to be simpler in design and have lower latency.

Please note: You will still be charged by normal pricing based on number of inputs being processed through number of models (e.g. if you send 5 images through to a Workflow which has 3 models inside of it, you will be charged for 15 operations).

Users can create a Workflow within an application on the Manage Your Applications page. On this page, you can select from a list of Public Models and Custom Models (within this app) to be added to the Workflow.

image1

Once the Workflow has been created, you can start making API calls to the Workflow endpoint! All the details for the documentation can be found in our Docs section.

How do you like our new Workflow Predict feature? Shoot us a message at feedback@clarifai.com!


Announcing Clarifai's new and improved v2 API video recognition feature

We are happy to announce that the video recognition feature in Clarifai’s v1 API has now been upgraded to our v2 API from beta to general availability! Now, all Clarifai users will be able to understand what’s happening in their videos with the latest and greatest features and improvements to our API. 

As one of the first companies to hit the market with an AI-powered video recognition solution, we’re excited to announce that our v2 API now supports all the video recognition functionality from v1 … and more! We received lots of positive feedback for the video product in the beta phase, and after making some enhancements to the product, we’re making it generally available for all developers and businesses to leverage in moderating and analyzing video content to improve workflows and user experiences.

image1

Users will be able to make Predict operation calls on the following Public Models:

  • General
  • Apparel
  • Food
  • Moderation
  • NSFW
  • Travel
  • Wedding

We are offering more models with greater accuracy than our previous v1 API video product, and we have also expanded our infrastructure capacity to respond back to your requests faster. We decided to move the video functionality into the v2 API because we have lots of other exciting features planned for video, which you’re going to love (stay tuned!).

Video in our v2 API analyzes and processes inputs at a rate of 1 frame per second, which means you will receive a list of predicted results for every second of your video. Similarly to our pricing for image recognition, 1 frame of video will be equal to 1 operation. This means that a 2 minute video would be 120 frames, and you would be charged for 120 operations.

Our Documentation Guide has details on how you can integrate the video product into your app, and you can try out the new API in our web demo!


Introducing API Keys - a safer way to authenticate your applications

As our customers build more sophisticated applications using our API, their requirements and needs are evolving. To provide greater flexibility and control to our customers, we are introducing API Keys, an authentication method that gives more power to developers to build applications for all of their needs.

We are introducing API Keys, an authentication method that gives more power to developers to build applications for all of their needs. Read on to learn more about API keys and why they’re better for you, or skip directly to the usage instructions at the bottom of this post! 

Authentication through API Keys will now contain scopes, which allows the key to only make selected API calls for an app. For example, a developer could generate an API Key that has permission to only make a ‘Predict’ or a ‘Search’ API call. This key could be considered as a ‘read-only’ key, as you wouldn’t be able to make any changes to the application using this key. Alternatively, if a developer is looking to create and train a custom model, then they would add “Models:Add” and “Models:Patch” scopes to their key.

There are many benefits to using API Keys, but they prove to be especially beneficial when developing an application that authorizes on the client side (e.g. the browser), or a mobile application. In each of those situations, there is a risk of an end user reverse engineering the code to retrieve the key that is being used. When building those applications, developers should protect their app by using a ‘read-only’ key in the production environment, reducing vulnerability of their app.

In the past, we have used OAuth2 client credentials (Client ID, Client Secret, Access Token) to authenticate access to our API. While this authentication method is also secure, it didn’t provide the flexibility of finer level scopes to be developed, nor did it allow our customers to delete a token, in case it was compromised. API Keys will empower our customers to do more.

We realize that this is a significant change in how developers interact with our API. However, we believe that this change is worth making, as it provides more power, security, and control for our customers, which in turn makes applications more secure, and makes the Clarifai platform friendlier to use.

With this change, existing developers that already have applications using Access Tokens for authentication will not experience a breaking change. We will allow both auth methods (Access Tokens and API Keys) until late 2017 to give everyone enough time to transition over to the new authentication method. Following that, we will deprecate Access Tokens, and developers will only be able to authenticate using API Keys.


Start using API Keys

  1. Log in to https://developer.clarifai.com, and click on “API Keys” on the left navigation barimage1
  2. Each of your existing application will have an auto generated key, with ‘All’ scopes.
  3. Click on “Create New API Key” to create a key with finer scopes.
  4. Select your application, and the scopes you want to apply to the API Key. Enter a description that will help you identify your key.image2
  5. Update your API Client, so it uses the new API Key method for authentication.
  6. Initialize your API Client with the API Key that was generated.

You can continue building your apps using the new and safer authentication method!

Further details for API Keys can be found in the Clarifai Documentation. Please contact Support if you have any additional questions.