AI | NINE Lab

Annotation Garden Initiative: Collaborative Annotation for Neuroscience

Tue, 02 Dec 2025 00:00:00 +0000

Overview

The Annotation Garden Initiative (AGI) establishes an open infrastructure for collaborative, multi-layered annotation of stimuli used in neuroscience research. Building on emerging standards including Stim-BIDS (BEP044) and HED specifications, AGI addresses the critical gap between increasingly complex naturalistic stimuli and the need for standardized, reusable annotations.

Through GitHub-based version control and modern web interfaces, AGI enables researchers to share, refine, and build upon stimulus annotations across studies, transforming how we document and analyze brain responses to complex sensory experiences.

The Problem We’re Solving

Modern neuroscience increasingly employs naturalistic stimuli, from the 73,000 COCO images prepared for the Natural Scenes Dataset to the 2-hour audiovisual experience of Forrest Gump and the diverse movie-watching paradigms in the Child Mind Institute’s Healthy Brain Network. Each dataset represents thousands of hours of wasted researcher time as labs independently annotate the same stimuli: scene boundaries, emotional valence, semantic content, acoustic features, narrative structure.

Yet these annotations remain siloed within individual studies, limiting reproducibility and preventing the field from building cumulative knowledge about brain-behavior relationships.

Current approaches fragment across:

Tools: ELAN, Praat, custom scripts
Formats: CSV, JSON, proprietary
Storage: Lab servers, supplementary materials, forgotten hard drives

Architecture

Each stimulus becomes a Git repository with branches serving as annotation layers. Different perspectives (scene boundaries, emotional arcs, object detection) can be developed independently and merged through standard Git workflows.

Repository Structure

Each stimulus repository follows a consistent structure aligned with stimuli-BIDS:

stimulus-name/
├── stimuli/
│ └── [stimulus files or pointers]
├── annotations/
│ ├── visual-saliency/
│ │ ├── events.tsv
│ │ └── events.json
│ ├── emotional-ratings/
│ └── scene-boundaries/
├── README.md
└── LICENSE

Flagship Datasets

Natural Scenes Dataset (NSD)

73,000 COCO images prepared for this dataset, with each of 8 subjects viewing approximately 10,000 distinct images during 7T fMRI scanning. AGI hosts annotation layers including: object detection, scene categorization, emotional ratings, and visual saliency maps.

Forrest Gump (studyforrest.org)

The 2-hour movie stimulus has published annotations: 870 shot boundaries, 2,500+ spoken sentences with speaker identification, emotion characterizations, and frame-wise perceptual features. AGI unifies these layers into HED-compatible formats.

HBN Movies

The diverse movie collection from the Child Mind Institute (Despicable Me, The Present, Pixar shorts) serves as the foundation for the NeurIPS 2025 EEG Foundation Challenge. AGI handles copyright complexities through pointer-based architecture.

Current Status

The platform at annotation.garden hosts:

VLM-generated annotations for NSD Shared 1000 images (see Image Annotation Tool)
Infrastructure for hosting and versioning stimulus annotations
Plans for HED integration to standardize semantic annotations

Tools and Resources

Image Annotation Tool

The Image Annotation tool provides a web-based interface for VLM-powered image annotation, starting with the Natural Scenes Dataset.

Standards Integration

BIDS: Brain Imaging Data Structure; Stim-BIDS (BEP044) for standardized annotation format
HED: Hierarchical Event Descriptors for semantic annotation framework

Access Information

License

License: CC-BY-NC-SA 4.0

References

Allen, E. J., et al. (2021). A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25(1), 116-126.
Hanke, M., et al. (2014). A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie. Scientific Data, 1, 140003.
Shirazi, S. Y., et al. (2024). HBN-EEG: The FAIR implementation of the Healthy Brain Network electroencephalography dataset. bioRxiv.

Image Annotation Tool: VLM-Based Annotation for Neuroscience

Tue, 02 Dec 2025 00:00:00 +0000

Overview

The Image Annotation Tool generates annotations for static image datasets using open-source Vision Language Models (VLMs). We have used this tool to create an annotation bank for the NSD Shared 1000 images, the subset of images viewed by all 8 subjects in the Natural Scenes Dataset study.

The tool runs VLMs locally via OLLAMA. The current annotation bank includes outputs from six models: Qwen2.5-VL (7B, 32B), Gemma3 (4B, 12B, 27B), and Mistral-Small3.2 (24B). Quality assessment across models is in progress.

Key Features

Models Used

All annotations generated locally via OLLAMA:

Qwen2.5-VL: 7B and 32B parameter versions
Gemma3: 4B, 12B, and 27B parameter versions
Mistral-Small3.2: 24B parameters

Multi-Prompt Annotation

Each image is annotated using multiple prompts (general description, foreground/background, entities and interactions, mood and emotions) across all models to capture different aspects of the scene.

HED Integration (Planned)

Integration with Hierarchical Event Descriptors (HED) is the next development priority:

Mapping VLM annotations to HED tags
Validation against HED schema
Export in BIDS-compliant events.tsv format

BIDS Compliance

Annotations follow stimuli-BIDS specifications:

Standardized events.tsv format
JSON sidecars with annotation schema
Compatible with neuroimaging datasets

Web Dashboard

Interactive visualization with AGI branding, real-time annotation preview, and easy navigation through large datasets.

Annotation Types

The tool supports multiple annotation types for comprehensive image description:

General Description

Detailed natural language descriptions of image content, setting, main elements, colors, lighting, and overall composition.

Object Detection

Identification and localization of objects within images, compatible with COCO categories.

Scene Categorization

Classification of scenes into semantic categories for cross-dataset analysis.

Emotional Ratings

Valence and arousal ratings for affective neuroscience applications.

Natural Scenes Dataset (NSD)

The tool is optimized for the NSD Shared 1000 Dataset, featuring 1,000 images viewed by all 8 subjects in the Natural Scenes Dataset study. This shared subset enables:

Cross-subject analysis of neural representations
Benchmark annotations for model comparison
Foundation for the broader 73,000 image collection

Architecture

Backend

FastAPI: Python API for VLM orchestration
OLLAMA: Local model serving on GPU

Frontend

Next.js: Modern React framework
AGI Branding: Consistent design with Annotation Garden
Real-time Updates: Live annotation preview

Storage

JSON files with comprehensive metrics
Database support for large datasets
Version control through Git

Performance Benchmarks

All performance metrics were generated using an NVIDIA GeForce RTX 4090 GPU with OLLAMA for local model inference.

Annotation Tools

Powerful CLI tools for post-processing annotations:

from image_annotation.utils import reorder_annotations, remove_model, export_to_csv

# Reorder model annotations by quality
reorder_annotations("annotations/", ["best_model", "second_best"])

# Remove underperforming models
remove_model("annotations/", "poor_model")

# Export for analysis
export_to_csv("annotations/", "results.csv", include_metrics=True)

Quick Start

Prerequisites

Python 3.11+
Node.js 18+
OLLAMA
GPU with sufficient VRAM for target models

Installation

# Clone from Annotation Garden
git clone https://github.com/Annotation-Garden/image-annotation.git
cd image-annotation

# Python environment
conda create -n torch-312 python=3.12
conda activate torch-312
pip install -e .

# Frontend
cd frontend && npm install

Usage

# Start OLLAMA (for local models)
ollama serve

# Run frontend dashboard
cd frontend && npm run dev
# Visit http://localhost:3000

Access Information

License

License: CC-BY-NC-SA 4.0

Annotation Garden Initiative: Parent organization for collaborative annotation
HED-MCP: Model Context Protocol for HED integration
Natural Scenes Dataset: Source of stimulus images

NEMAR Dataset Citations Analysis Dashboard

Fri, 01 Aug 2025 00:00:00 +0000

NEMAR Dataset Citations Analysis

The NeuroElectroMagnetic Data Archive and Tools Resource (NEMAR) is an integrated data, tools, and compute resource that serves as a gateway to human neuroelectromagnetic data (EEG, MEG, iEEG) shared through OpenNeuro. By providing seamless access to high-quality datasets alongside high-performance computing resources, NEMAR enables researchers to discover, assess, and process neurophysiological data without the burden of large downloads.

This dashboard analyzes citation patterns within the NEMAR ecosystem to reveal how open data sharing drives scientific collaboration and research impact in human neuroscience.

What You’ll Find

Citation Network Analysis - Discover how datasets are co-cited and identify research clusters within the neuroscience community.

Temporal Trends - Track the evolution of research themes and citation growth over time.

Research Impact Assessment - Identify which datasets generate the most influential research outcomes.

Collaboration Patterns - Understand how NEMAR facilitates interdisciplinary research and cross-institutional partnerships.

About NEMAR’s Impact

NEMAR addresses a critical challenge in neuroscience: ensuring that valuable neuroelectromagnetic data, collected at considerable expense and effort, remains accessible and useful to the broader research community. By combining data discovery tools, quality assessment features, and direct integration with high-performance computing resources through the Neuroscience Gateway, NEMAR exemplifies how integrated data-tools-compute resources (DATCORs) can accelerate scientific discovery.

The citation analysis below demonstrates the real-world impact of this open science approach on human neuroscience research.

Technical Note: This analysis uses network analysis and natural language processing to examine citation patterns across research publications utilizing NEMAR datasets. All visualizations are interactive—explore the data to uncover insights about open neuroscience collaboration.

HBN Dataset Insights Dashboard

Mon, 21 Jul 2025 00:00:00 +0000

Overview

The Healthy Brain Network (HBN) EEG dataset represents one of the largest open-access developmental neuroimaging initiatives, with 3,602 participants aged 5-22 years across 11 dataset releases. This dashboard provides comprehensive insights into participant demographics, mental health patterns, and data availability to help researchers understand and utilize this valuable resource.

This dashboard complements our other HBN content:

HBN-EEG paper and dataset downloads - Read the FAIR implementation and BIDS formatting of the HBN dataset, and access all 11 releases with technical documentation

Each resource addresses a different aspect of the HBN-EEG dataset: the paper details the scientific methodology, the data portal provides access to downloads and technical specs, and this dashboard enables data exploration.

3,602

Total Participants

Dataset Releases

5-22

Age Range (years)

EEG Task Paradigms

Age Distribution

Sex Distribution

EEG Task Data Availability

Mental Health Metrics Correlation

Mental Health Patterns by Age Group

Release-Specific Detailed Analysis

Select Release:

Total Participants

w/ Mental Health Score

Sex Distribution (M/F)%

Age Range (yrs)

Age Distribution

Mental Health Correlations with Age

Task Status Distribution

Data Status Categories

Available: Data and event markers available, passes quality checks

Caution: Data exists but may have quality issues, look for quality reports.

Unavailable: Task is not available.

No Event: Task is available but no event markers were found.

Key Insights

Demographics & Development

Age Distribution: The dataset shows a robust developmental span with peak representation in middle childhood (6-10 years), providing excellent coverage for studying developmental trajectories
Sex Balance: 64.1% male, 35.9% female participants, reflecting typical recruitment patterns in neurodevelopmental research
Developmental Coverage: Strong representation across critical developmental periods from early childhood through early adulthood

Mental Health Patterns

Correlation Structure: Mental health metrics show expected patterns - p-factor correlates positively with both internalizing (r=0.16) and externalizing (r=0.22) behaviors
Developmental Trends: Internalizing symptoms increase markedly with age, from -0.23 in early childhood to 0.52 in late adolescence/early adulthood
Age-Related Changes: Externalizing behaviors peak in early childhood (0.26) and decrease with age, while attention difficulties remain relatively stable

Data Quality & Availability

High-Quality Core Tasks: RestingState (86.3%) and movie-watching tasks (70-82%) show excellent data availability
Challenging Paradigms: Sequence learning tasks show lower completion rates (22-40%), likely due to age-appropriateness and task difficulty
Task Categorization: Passive viewing tasks generally show higher completion rates than active cognitive tasks

Research Implications

Large-Scale Analyses: With 3,547 participants having complete mental health data, the dataset supports robust statistical analyses and machine learning approaches
Cross-Task Studies: High availability of core tasks enables comprehensive cross-paradigm analyses
Developmental Studies: Excellent age coverage supports both cross-sectional and cohort-based developmental research

Technical Notes

This dashboard visualizes comprehensive analysis of the HBN master participants dataset. Data processing included:

Age group categorization for developmental analysis
Task availability rate calculations across all paradigms
Mental health correlation analysis with 3,547 complete cases
Quality control metrics based on data availability flags

For detailed dataset information and download links, visit the HBN-EEG data page.

Data Source: HBN Master Participants List (Releases 1-11)
Analysis Date: January 2024
Sample Size: 3,602 participants
Mental Health Sample: 3,547 participants with complete data

AI | NINE Lab

Annotation Garden Initiative: Collaborative Annotation for Neuroscience

Overview

The Problem We’re Solving

Architecture

Repository Structure

Flagship Datasets

Natural Scenes Dataset (NSD)

Forrest Gump (studyforrest.org)

HBN Movies

Current Status

Tools and Resources

Image Annotation Tool

Standards Integration

Access Information

Links

License

References

Image Annotation Tool: VLM-Based Annotation for Neuroscience

Overview

Key Features

Models Used

Multi-Prompt Annotation

HED Integration (Planned)

BIDS Compliance

Web Dashboard

Annotation Types

General Description

Object Detection

Scene Categorization

Emotional Ratings

Natural Scenes Dataset (NSD)

Architecture

Backend

Frontend

Storage

Performance Benchmarks

Annotation Tools

Quick Start

Prerequisites

Installation

Usage

Access Information

Links

License

Related Projects

NEMAR Dataset Citations Analysis Dashboard

NEMAR Dataset Citations Analysis

What You’ll Find

About NEMAR’s Impact

HBN Dataset Insights Dashboard

Overview

Related HBN Resources

Release-Specific Detailed Analysis

Task Status Distribution

Data Status Categories

Key Insights

Demographics & Development

Mental Health Patterns

Data Quality & Availability

Research Implications

Technical Notes