🔥 ShutterMuse: Capture-Time Photography Guidance with MLLMs

💡 The paper introduces a new benchmark and dataset for photography assistance, focusing on capture-time guidance for both camera framing and subject pose. Existing models primarily evaluate post-hoc crop prediction and overlook subject-side recommendations, leaving a gap in the capabilities of multimodal large language models. To address this, the researchers developed CaptureGuide-Bench, a benchmark with two tasks: photographer-side composition decision and refinement, and subject-side scene-conditioned pose recommendation. They also constructed CaptureGuide-Dataset, comprising 130K samples with textual rationales and visual annotations.

The researchers then developed ShutterMuse, a unified multimodal large language model trained with supervised and reinforcement fine-tuning. ShutterMuse provides both composition guidance and pose recommendations during image capture. The evaluation reveals that general-purpose models can make composition decisions but lack precise refinement localization, while specialized aesthetic cropping models localize crops effectively but are limited to refinement and do not provide pose guidance.

The experiments on CaptureGuide-Bench show that ShutterMuse achieves the best overall photographer-side performance among evaluated baselines and competitive subject-side pose recommendation with lower inference cost. This demonstrates the potential of multimodal large language models as interactive assistants for photography during image capture, addressing the need for capture-time guidance in real-world photography. The paper contributes to the development of models that can provide effective guidance for both camera framing and subject pose, making it a significant step forward in the field of photography assistance.

📅 Published on Jun 24

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.25763
• PDF: https://arxiv.org/pdf/2606.25763
• Project Page: https://lijayutnt.github.io/ShutterMuse/

🤖 Models citing this paper:
• https://huggingface.co/ShutterMuse/ShutterMuse

📊 Datasets citing this paper:
• https://huggingface.co/datasets/ShutterMuse/CaptureGuide-Bench

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/ShutterMuse/ShutterMuse-Video-Demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLargeLanguageModels #CaptureTimeGuidance #PhotographyAssistance #CameraFramingTechniques #MultimodalLearningModels