As you may have noticed a couple of weeks ago, we’ve opened up a whole new suite of products for developers to start building on for free. Here’s a walkthrough of our new API and how to get the most out of it. Enjoy!
Good evening, my fellow Americans. And those who aren’t American. Hello, all.
Guess what. We’re releasing a new version of the Clarifai API! It’s available now in developer preview mode, meaning you can start using it if you want to or you can ignore it and go back to your regular routine … for now. The new version of our API is packed with more new features than ever, so you might want to take a look or skip directly down to the tutorial.
What’s new in our API? I’ll tell you.
/predict - Similar to Version 1's /tag, this allows users to get the tags from a given image based on a model. What's exciting about this change is that as users are creating their own models and concepts (which I'm about to mention in just a bit), they can easily get information from their media from various models through the same endpoint.
/searches - A feature that's a VERY common case for using Clarifai is to get the concepts predicted in an image, and then use those concepts to power ~search~. The new Search API allows you to send images to the service, and have them indexed by their concepts and their visual representations. Once they’re indexed, you can search for images by concept or using *gasp* reverse image search. Or, if you want to be fancy, you can even combine search by image with search by tag!
/inputs and /models - Now this is the really exciting stuff. Clarifai provides a few different models that can “see” the world differently ("food" for example, only sees things as they are related to food, "weddings" only sees things related to weddings, etc). A model contains a group of concepts. A model will only see the concepts it contains. So, if you only use our models, though we have thousands of concepts available, there might be times when you'll want a model that sees the world the way ~you~ see it.
The new Custom Training API allows you to do this! You can create your own model and train it with your own images and concepts. Once you train it to see how you would like it to see, you can then use that model to make predictions (with the /predict endpoint mentioned earlier).
That being said, in terms of pricing, we are changing it up for V2, but not too much. You'll still get your 5,000 free API calls per month. But, what's new is that you have these custom models you're building! For free, you get 10 custom models each month. Now, instead of tiered pricing on top of this, we've decided to make it per model and per API call on top of your free ones per month. It's only $0.0020 per API call, and $0.46 each per custom model, so you're able to get creative and build exciting things for a very low price, and you only have to pay for what you use. But wait, there’s more. It’s FREE until November 1st! Jump on it!
BUT HOW, you might ask. Well, let’s walk through it. Let’s say you’re really into something and you save photos of it all the time. Could be candy. Or your dog. Or your neighbor’s dog. Or cookie jars. I don’t know you. But you’re into something. For me, I’m really into mechanical keyboards. I want to make an app that can go through the photos I have of them, tell me what type they are (heh, get it? Type? Pun? I’m so lonely) and when I want to search for that one awesome orthographic blank keyboard, I can find it.
So, let’s get started. I’m first going to go to https://preview.clarifai.com and log in. Easy enough. Hit that “Create Application” button:
... and that redirects you to developer.clarifai.com where you can create your application like you usually do. I’m just going to call mine keyboard-store for now.
Okay, so now we’re gonna go back to preview.clarifai.com and click on our application.
Nothing. How thrilling. Let’s upload some pics. Conveniently for me, I have a bunch saved on my desktop already. You can click that up arrow on the right to upload your pics by copying and pasting, dragging from the desktop, plugging in an image URL, the works.
That’s more like it.
Now, I’m going to start making concepts here in the UI, because I don’t feel like coding right now. That’s Clarifai: keeping developers lazy one pic at a time. Keep in mind, you can do everything I’m about to show you with code.
If you click on that arrow thing in the top left, you get a fun pop out menu. I’m going to make a few concepts (by clicking “Create concept”, wow) really quickly:
Now, I have to train said concepts so that I can organize my photos that way! The more photos I add, the more accurate the concept’s predictions.
By clicking on the floating checkmark on each image, I can train all of them at once:
So, if I hit that “Add Concepts” button, I get this popup and can add the concepts I’ve created:
Now I can click on one of my concepts here, and that trains my model to learn that these images (AKA inputs) are artisan keycaps.
I repeated this a few times for different chunks of images. If I click on one of them now, I can see the probabilities of each tag, similar images, and other general tags for the photo:
Notice that little square icon on the left. If I click it and drag over part of the image, like so:
...then I can get more similar images to that part of this picture. Neat!
Imagine if I had a TON of images tagged with really detailed concepts about mechanical keyboards, like SA versus DSA keycap shapes, Topre versus Cherry versus Matias switches, 40% versus 60% versus Ergodox layouts, different types of keycap images … you could theoretically crop part of a keyboard and know exactly what type of switch that is, and you could build an app to help you find where you can buy it, or find where it was made and by whom, or you could just get details about a given switch style. THAT is the power of Custom Training and Visual Search.