eva-labeling

What is EVA Labeling?

The EVA Labeling is a wrapper to enable you to run EVA alongside Label-Studio It lets you connect to Label Studio server to do the following:

• Annotate your dataset more quickly and easily by allowing you to label them in bulk.
• Dynamically pre-annotate data based on model inference results.
• Retrain or fine-tune a model based on recently annotated data.

Demo video

bulk-prop-demo.mp4

How it works

1. This application helps fetch data from Label Studio server and add it to EVA.
2. Then you use Label Studio ML Backend to run EVA Query Statements on you data. (e.g. Clustering, Object detection, Object Tracking)
3. Returns the desired output to Label Studio

Quickstart

⚠️ The Label Studio server should be started with flag EXPERIMENTAL_FEATURES=1 Otherwise Bulk Propagation will not work!

1. Clone this repository

   git clone https://github.com/georgia-tech-db/eva-labeling.git

2. Setup Environment
   It is highly recommended to use venv python environments. You can use the same environment as Label Studio. Read more about creating virtual environments via venv.

   cd eva-labeling
   
   # Install eva-labeling
   pip install -e .

3. Start eva-labeling server

   Note: this will also start the evadb server.

   label-studio-ml start ./evaml -eu <eva-server-url> -ep  <eva-server-port> -k <apikey> -ls <label-studio-url>

4. Add Image Clustering Interface

   <View>
   <Image name="image" value="$image"/>
   <RectangleLabels name="label" toName="image">
     
   </RectangleLabels>
   <TextArea name="cluster" toName="image"
             rows="1" editable="true" />
   </View>

5. Start and Register EVA ML Backend to Label Studio.

   Following Label Studio ML documentation

   

Adding Custom Models to EVA

1. Register your Feature Extractor model into EVA.

   Refer EVA docs for more information. Here is an example of adding a custom model.

2. inside ./evaml/cluster_image.py, you may add your queries.

   Refer Label Studio ML Backend for more information.