Tools we love Vol. 6: Segments.ai

About the series

As you know, we at Humans in the Loop have a great love and appreciation of a well-designed annotation tool. After the great feedback on the reviews we published of our the best platforms on the market here and here, we decided that it’s time for a deep dive in some of our all-time favorites!

This article is the sixth in a series of 10 reviews which will be published each week. Our previous reviews on Supervise.ly, TrainingData.io, Annotate.Online, Hasty.ai and Picterra can be found here. Soon we will be uploading the links to the other articles as they are released.

The whole series is based on the premise of transparency and honesty and none of these reviews are sponsored. They are just our way to give props to the best teams out there working on making annotation easier for AI teams, and to share some of the know-how that we have been accumulating over the past few years as a professional annotation company.

As in previous reviews, our parameters are:

• price
• functions
• project management
• automation

If you have additional questions or want to get in touch with us to beta test or feature your tool in an upcoming article, feel free to email us at hello@humansintheloop.org!.

Segments.ai

Segments.ai was founded in January 2020 in Brussels, Belgium. Its 3 founders met at KU Leuven where they were doing their PhDs and altogether they bring 12 years of experience in Machine Learning to the endeavor, which has developed significantly in just six months.

Professor Luc Van Gool who co-created DEXTR (i.e. Deep Extreme Cut, a model for automatic segmentation which uses the extreme points in an object) has also agreed to become a part of the team as a scientific advisor.

The main goal of the platform is not only to provide an interface for labeling but to drastically speed up segmentation by creating ultra-precise polygons with just a few clicks. The platform is currently offering a limited free plan (with just 100 images in private datasets) as well as a paid (payment per image) and enterprise plan (unlimited images).

Features

The interface of the platform is very clean and easy to use. The only type of annotation available currently is segmentation using superpixels (as well as video annotation in the paid versions). Essentially, users can easily control how many superpixels the image is broken down into and select the regions of their choice with a click.

The superpixel segmentation is extremely accurate but if users need to adjust it, they can use the brush and eraser tools. In addition, a super useful ‘line’ sub-feature has been enabled to account for polygonal annotation of objects with straight edges which is quite challenging otherwise with a brush. Existing objects can be ‘locked’ so that the brush paints only under them, which is a good solution to a common challenge with mask layers.

For visualization, the tool has an easy toggle between instance and semantic segmentation. The output for all images is generated as a bitmap mask stored in the cloud. In terms of export, users are able to create ‘Releases’ of the data which save the version of the dataset at the specific point in time. All functions are available through an API or Python SDK access, even in the free version.

Project management

Segments.ai supports all of the main project management features needed to successfully execute a project, as well as some interesting public/private options that are not available elsewhere. Projects are structured on the dataset level and they can be set to either public or private (as mentioned, there is a limit in the free version). In addition, ‘Public labeling’ might be enabled, which could potentially be a great recipe for crowdsourcing dataset labeling.

In terms of supervision and QC functions, additional users can be added to the project but the user management system is not rolled out yet as of this moment, so collaborators have to be registered on the website already and there isn’t a classification of users yet (admins, supervisors, annotators, etc). For QC, collaborators can use the ‘Review’ feature in order to to ‘accept’ and ‘reject’ the labelling. Analytics are not currently available but a birdie told us they might be soon!

Automation

The secret sauce behind the tool is the combination of different technologies to accelerate precise segmentation at a maximum. Active learning, few-shot learning, data distillation, generative modeling, self-supervised learning, and uncertainty estimation are all part of the offering of Segments.ai.

Most of these features are available for paying customers only, such as the ability to have the backend models be retrained and customized to each dataset as the data is being labeled. Another option is for paid users to leverage their own models to prelabel the dataset, after which a human-in-the-loop pipeline can be built using active learning.

However, a great feature available in the free tier as well is the option to select a backend model which is pre-trained for a specific industry. Currently available are ‘Automotive’, ‘People’, ‘Vegetation’, ‘Satellite’, ‘Medical’, and ‘Other’. Since the types of images in all of these use cases are quite different, this allows for a great quality.

In conclusion, we are amazed by how much Segments.ai has been able to achieve in just 6 months and we really appreciate the fact that they’ve decided to become the best at one thing and are focusing on it: providing the fastest segmentation there is

So we are looking forward to see what else is on the roadmap for them!