Huskylens2 MCP bridge to Claude Desktop
dstrout 2025-11-13 05:11:51 156 Views0 Replies Hi all,
I ran into an issue trying to use the MCP server on the Huskylens 2 with Claude desktop. It turns out that the MCP server returns both image data and text data in its get_recognition_result tool responses. However, Claude Desktop's MCP interface cannot handle the image data format returned by the camera, resulting in "The tool returned content in an unsupported format" errors.
I just wrote a little proxy that sits between them and filters out resource_link items with mimeType: "image/png" from tool responses. It's at https://github.com/dstrout/huskylens2_MCP_filter and hopefully will save someone the trouble of sorting this out. Once it's working, you can do things like:
use the huskylens tool to tell me what objects the camera can currently identify in the frame
I'll check what the HuskyLens camera can currently see and identify.
Get recognition result
The HuskyLens camera currently identifies 4 objects in the frame:
TV - Large object taking up much of the left side of the frame (confidence: 29.7%)Cup - Located in the center-right area (confidence: 26.4%)Two people - Both detected on the right side of the frame at different positions (confidence: 16.9% and 15.0%)The confidence scores are relatively low, which might indicate partial visibility of objects or challenging lighting conditions. Would you like me to take a photo to see the actual scene, or check what algorithm/application the HuskyLens is currently using?
Right now the toolset is a bit limited, but still pretty cool. What is exposed as of firmware 1.6 is:
give me a full list of the tools and data available from the huskylens tool
Looking at the HuskyLens tools available, here's a complete breakdown of what can be accessed:
Huskylens2:get_recognition_result
Operation: get_result - Retrieves real-time recognition results including image data and recognized labels (object types, person names, etc.)Returns both visual data and detection coordinates with confidence scoresHuskylens2:manage_applications
Controls the internal algorithms/applications:
Operations:current_application - Shows which algorithm is currently activeswitch_application - Changes to a different algorithm (requires listing first)application_list - Lists all available algorithmsParameter: algorithm - English name of the application when switchingImportant note: Must call application_list before switching to get valid algorithm namesHuskylens2:multimedia_control
Operation: take_photo - Captures a photo from the cameraHuskylens2:task_scheduler
Creates automated tasks based on triggers:
Operations:create - Sets up new scheduled taskslist - Shows current scheduled tasksTask parameters:trigger (optional) - Recognition trigger name (e.g., 'keyboard', 'person', 'tiger')handler (required) - Currently only supports take_phototime (optional) - HH:MM:SS for scheduled executionCan create conditional actions like "take a photo when you see X" or time-based actionsThe data available from recognition results includes:
Object/person name (label)Bounding box coordinates (x, y, width, height)Confidence score (0-1 range)Multiple detections can be returned simultaneously
Hope this is helpful to someone!
