SensorMCP Server combines the power of foundation models (like GroundedSAM) with custom model training (YOLOv8) to create a seamless workflow for object detection. Using the Model Context Protocol, it enables Large Language Models to automatically label images using foundation models, create custom object detection datasets, train specialized detection models, and download images from Unsplash for training data.
The system supports a complete end-to-end workflow: from defining object ontologies through natural language, to automatically labeling training data with foundation models, to fine-tuning custom YOLOv8 detection models. All functionality is exposed through the Model Context Protocol, making it seamlessly accessible to LLM-powered applications and chat interfaces.
Uses GroundedSAM for automatic image labeling and object detection.
Fine-tune YOLOv8 models on your specific objects and datasets.
Native integration with LLM workflows and chat interfaces.
Download images from Unsplash and automatically create labeled datasets.
Define object classes to detect through natural language (e.g., "tiger, elephant, zebra")
Initialize GroundedSAM as the base model for automatic image labeling
Download images from Unsplash or import local image collections
Use foundation models to automatically generate labels and bounding boxes
Fine-tune YOLOv8 models on the automatically labeled dataset
All functionality exposed through Model Context Protocol for LLM integration
Defining object classes through natural language commands.
Foundation models automatically generating labels and bounding boxes.
Training custom YOLOv8 models on automatically labeled datasets.
list_available_models()define_ontology(objects_list)set_base_model(model_name)set_target_model(model_name)fetch_unsplash_images(query, max_images)import_images_from_folder(folder_path)label_images()train_model(epochs, device)git clone <repository-url>
cd sensor-mcp
uv sync{
"mcpServers": {
"autodistill-server": {
"type": "stdio",
"command": "uv",
"args": [
"--directory",
"/path/to/sensor-mcp",
"run",
"src/zoo_mcp.py"
]
}
}
}UNSPLASH_API_KEY=your_unsplash_api_key_hereIf you use this code or data in your research, please cite our paper:
@inproceedings{Guo2025,
author = {Guo, Yunqi and Zhu, Guanyu and Liu, Kaiwei and Xing, Guoliang},
title = {A Model Context Protocol Server for Custom Sensor Tool Creation},
booktitle = {3rd International Workshop on Networked AI Systems (NetAISys '25)},
year = {2025},
month = {jun},
address = {Anaheim, CA, USA},
publisher = {ACM},
doi = {10.1145/3711875.3736687},
isbn = {979-8-4007-1453-5/25/06}
}For questions about the zoo dataset or technical inquiries:
Email: yq@anysign.net
For MCP protocol documentation: Model Context Protocol