Doccano Tutorial

1. Legal Document Classification
Legal document classification is the process of categorizing legal texts like contracts, court rulings, or compliance forms into predefined classes. Labels can include confidentiality level, contract type, or jurisdiction. Doccano allows annotators to mark segments and tag them with specific categories, helping legal professionals sort and retrieve documents easily. This accelerates legal research and enables automation of repetitive document analysis tasks in law firms and regulatory bodies.

document = "This is a non-disclosure agreement (NDA) under US law."

label = "Confidential - NDA"

print("Classified as:", label)

      

2. Customer Feedback Analysis
Analyzing customer feedback involves reviewing survey comments, support tickets, or reviews to understand user sentiment, product issues, or feature requests. In Doccano, such texts are annotated by sentiment, topic, or urgency. This classification aids companies in making data-driven decisions, improving product features, and identifying negative experiences early. It also enables automated triaging of user complaints.

feedback = "The app keeps crashing on startup."

sentiment = "Negative"

category = "Bug Report"

print("Feedback Type:", sentiment, "| Category:", category)

      

3. Sentiment Labeling
Sentiment labeling assigns emotional tone to texts—positive, negative, or neutral. It's common in social media monitoring and customer service. Annotators mark sentences in Doccano with emotion tags to train models that detect opinions and public sentiment trends. Sentiment analysis is critical in brand monitoring, elections, and market research to assess people's views.

sentence = "I love this product!"

sentiment = "Positive"

print(f"Sentence: '{sentence}' labeled as {sentiment}")

      

4. Medical Named Entity Recognition
This involves tagging medical entities like diseases, drugs, or procedures in clinical notes. Annotators in Doccano highlight terms like "diabetes" or "aspirin" and label them with appropriate categories. It’s crucial for building systems that extract structured data from unstructured medical records to aid diagnostics or research. NER models trained on this data improve clinical decision support.

text = "Patient was prescribed ibuprofen for headache."

entities = [{"entity": "ibuprofen", "label": "Drug"}, {"entity": "headache", "label": "Symptom"}]

print("Extracted entities:", entities)

      

5. Social Media Classification
Posts from Twitter, Facebook, or Reddit can be classified by topic, sentiment, or user intent. This is especially useful for brands or researchers trying to monitor public discourse. In Doccano, annotators assign labels such as "complaint", "praise", or "product inquiry" to help train models that automate this task at scale.

tweet = "Why is my order delayed again?"

label = "Complaint"

print("Tweet Label:", label)

      

6. Chatbot Intent Tagging
Intent tagging defines what a user wants when they input a query to a chatbot (e.g., "Book flight", "Cancel reservation"). Annotators use Doccano to tag these sentences with intents so AI chatbots can learn to map phrases to backend actions. It forms the foundation of NLP-based conversational agents.

user_input = "I want to cancel my booking."

intent = "CancelBooking"

print("Detected intent:", intent)

      

7. Toxic Comment Detection
Detecting toxic content involves labeling offensive, abusive, or threatening language online. Annotators tag such instances in Doccano with categories like "hate speech", "harassment", or "spam". This annotated data is critical to developing content moderation tools used on platforms like YouTube, Twitter, or Reddit.

comment = "You are so dumb and useless!"

label = "Toxic - Personal Insult"

print("Flagged comment:", label)

      

8. Product Review Analysis
Annotators tag product reviews with aspects such as "battery life", "camera", or "shipping" to extract fine-grained sentiment. This helps e-commerce platforms understand detailed feedback, like customers liking the camera but disliking battery performance. The annotations can power aspect-based sentiment analysis systems.

review = "Camera quality is good but battery drains quickly."

tags = [{"aspect": "Camera", "sentiment": "Positive"}, {"aspect": "Battery", "sentiment": "Negative"}]

print("Review Analysis:", tags)

      

9. Contract Term Extraction
Legal agreements often require extracting terms like expiration date, governing law, or liabilities. Using Doccano, these key clauses can be annotated and later extracted automatically by models. It's valuable for automating compliance checks, contract analysis, and due diligence.

clause = "This agreement terminates on Dec 31, 2025."

term = {"label": "End Date", "value": "Dec 31, 2025"}

print("Extracted term:", term)

      

10. OCR Error Correction
OCR (Optical Character Recognition) often introduces text errors. Annotators can use Doccano to correct these by labeling incorrect segments and writing corrections. This helps improve OCR engines and build training datasets for error correction in scanned document pipelines.

ocr_text = "Th1s 1s a t3st."

corrections = {"Th1s": "This", "t3st": "test"}

print("Corrected Output:", corrections)

      

1. QA Workflow Setup
A QA workflow in annotation ensures high-quality labeled data by defining structured steps: annotation, review, feedback, and correction. In Doccano, this could involve roles like annotators, reviewers, and project managers. Setting up this workflow includes configuring permissions, defining review checkpoints, and establishing versioning. Without QA, annotations may suffer from inconsistency, which can degrade model performance.

workflow = ["Annotate", "Review", "Feedback", "Finalize"]

print("QA Workflow steps:", workflow)

      

2. Inter-Annotator Agreement
Inter-Annotator Agreement (IAA) measures how consistently multiple annotators label the same text. High agreement indicates reliability, while low agreement suggests ambiguity or poor guidelines. Metrics like Cohen’s Kappa or F1-score are commonly used. Doccano exports can be used to calculate IAA across overlapping tasks.

annotator1 = ["Positive", "Negative", "Neutral"]

annotator2 = ["Positive", "Negative", "Negative"]

matches = sum([a1 == a2 for a1, a2 in zip(annotator1, annotator2)])

print("Agreement Ratio:", matches/len(annotator1))

      

3. Precision/Recall Metrics
Precision and recall are standard evaluation metrics. Precision measures how many selected labels were correct, while recall measures how many correct labels were selected. These metrics guide annotator performance and help in QA reports.

true_positives = 8

false_positives = 2

false_negatives = 3

precision = true_positives / (true_positives + false_positives)

recall = true_positives / (true_positives + false_negatives)

print("Precision:", precision, "Recall:", recall)

      

4. Manual Spot Checks
Spot checking involves manually reviewing random samples to detect annotation errors. It’s cost-effective and especially useful in large datasets. Doccano allows exporting annotations for sampling externally. Annotators are then given targeted feedback for improvement.

import random

annotations = ["A", "B", "C", "D", "E"]

spot_check = random.choice(annotations)

print("Sample chosen for spot check:", spot_check)

      

5. Reviewer Comments
Reviewers can leave comments during or after checking annotations. These are essential for continuous learning and correcting annotation errors. Some tools support threaded comments or in-line suggestions that annotators must review and acknowledge.

reviewer_comment = "Ensure correct sentiment tag for sarcasm cases."

print("Reviewer Note:", reviewer_comment)

      

6. Annotator Training
Before annotation starts, annotators must undergo training sessions with sample texts and guidelines. Doccano allows using practice projects to prepare them. This ensures consistency and prevents early-stage errors that may cascade.

training_dataset = ["Text A", "Text B", "Text C"]

print("Training examples provided:", training_dataset)

      

7. Audit Trails
Audit trails track who did what and when. Doccano logs annotations, edits, and reviewer actions. This information is critical when debugging large projects or handling compliance audits.

audit = [{"user": "Ann", "action": "label", "time": "12:00"}, {"user": "Rev", "action": "approve", "time": "12:30"}]

print("Audit Log:", audit)

      

8. Automated QA Scripts
Scripts can flag missing labels, inconsistent formats, or schema violations. They are crucial in large datasets where manual review isn’t scalable. These can be run before model training to clean data.

annotations = ["Positive", "", "Negative"]

flagged = [i for i, a in enumerate(annotations) if a == ""]

print("Flagged entries:", flagged)

      

9. Comparing Annotation Versions
Comparing old and new annotation versions helps track changes or model-driven improvements. It’s especially useful when refining guidelines or retraining teams.

version1 = ["A", "B", "C"]

version2 = ["A", "B", "D"]

changes = [(i, v1, v2) for i, (v1, v2) in enumerate(zip(version1, version2)) if v1 != v2]

print("Differences:", changes)

      

10. Common QA Issues
Common problems include missing annotations, inconsistent labeling, bias, and ambiguous definitions. These can often be solved by clearer guidelines, better training, and using automated tools to validate consistency before exporting datasets for training models.

issues = ["Inconsistent tag", "Missed label", "Overlapping span"]

print("Frequent QA Issues:", issues)