It has always been challenging to launch an annotation effort. A team of researchers introduce The Portable Text Annotation tool (Potato), a web-based application approved for use in the EMNLP 2022 DEMO track. Potato’s project hub is designed to make it easier to replicate existing annotation efforts.
Potato facilitates the rapid prototyping and deployment of several text annotation tasks. This work aims to make it possible for individuals or small groups to annotate text data with minimal effort, starting from scratch and finishing annotation with just a few configuration lines. Annotators utilize a web-based front-end to work with data, while the back-end of Potato functions as a web server that may be launched locally.
A single configuration file determines the task and data types used by Potato. To start with Potato, users don’t need to know how to code. Potato is adaptable, allowing users to modify the user interface and the items their annotators interact with without needing additional web design. Users can quickly retrieve a project with potato and then open the annotation site.
The variety of annotation aids supported by Potato is impressive.
- Simple to set up and adaptable to various requirements: Changing Potato’s settings is as simple as modifying a file. Creating an annotation website doesn’t involve coding. Like other features, Potato offers a wide range of customization options.
- Predefined structures and defaults: Annotation schemas like radio, likert, checkbox, textbox, span, pairwise comparison, best-worst-scaling, image/video-as-label, etc., are all supported by Potato.
- Several data formats: Potato can show anything from brief documents to long ones, including conversations, comparisons, and more.
- Researchers in natural language processing (NLP) may need to perform a battery of related but distinct tasks (e.g., multilingual annotation). Potato has supported the Multilingual Twitter Intimacy Analysis task, making it possible to build configuration files for all tasks with minimal effort.
- Increasing Efficiency in Annotation: To improve the annotators’ experience and provide annotations more quickly, Potato was thoughtfully designed with several features.
- Keyboard shortcuts are simple to configure: Keyboards allow annotators to quickly and easily enter their responses.
- Smartly emphasizing the probable relationship between labels and keywords in the document is possible with dynamic highlighting, which can be set up for tasks with many labels or extremely long documents.
- With many labels, it can be difficult for annotators to keep track of their definitions without the help of tooltips. Thanks to Potato’s customizable label tooltips, annotators can learn more about labels by hovering their mice over them.
- Improving knowledge of annotators: Potato provides tools that may be used to learn more about the annotators who worked on user data and spot any potential biases. Potato’s user-friendly interface makes creating both pre- and post-screening questionnaires simple, which might provide light on users’ annotators’ professional histories. Potato includes a set of question templates that make it simple to set up standard qualifying inquiries like demographics.
- Enhancing Quality Assurance: Potato includes tools for identifying spammers and gathering more trustworthy remarks.
- Potato’s attention test feature makes it simple to create questions designed to detect spammers and randomly insert them into the annotation queue.
- Before proceeding with full data labeling, users can quickly and easily identify unqualified annotators using Potato’s built-in qualification test.
- With Potato’s built-in time check, it can easily monitor how long annotators spend on each instance and gain insight into their work habits.
Since Potato is hosted on pypi, users can simply run “pip install potato-annotation” to get it up and to run. Potato may be easily deployed online to gather annotations from popular crowdsourcing platforms like Prolifc.com. Users will need a server with accessible ports to use Potato in a crowdsourcing environment. Potato works flawlessly with Prolific, a platform for finding and recruiting task participants.
Check out the GitHub and Documentation. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 16k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Bhubaneswar. She is a Data Science enthusiast and has a keen interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring the new advancements in technologies and their real-life application.