Spring AI Can someone help When I upload an MP4 video, there will be an error that recognizes the video as an image

LLM：Qwen2.5-Omni-7B-AWQ spring-ai: 1.1.0-M2

My core code：

`String mime; String suffix = requestPojo.getFileName().split("\.")[1]; switch (suffix) { case "png": mime = "image/png"; break; case "mp4": mime = "video/mp4"; break; default: mime = "image/jpeg"; }

        String finalMime = mime;
        promptUserSpec = u -> u
                .text(prompt1.getContents())
                .media(
                        Media.builder()
                                .mimeType(MimeType.valueOf(finalMime))
                                .data(new FileSystemResource(uploadPath + File.separator +requestPojo.getFileName()))
                                .build()
        );`

After uploading to the large model, the error message is as follows：

(APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] Error in preprocessing prompt inputs (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] Traceback (most recent call last): (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/utils.py", line 244, in fetch_image_async (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] return await self.load_from_url_async( (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/utils.py", line 165, in load_from_url_async (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] return await future (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] result = self.fn(self.args, self.kwargs) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/utils.py", line 100, in _load_data_url (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] return media_io.load_base64(media_type, data) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/image.py", line 91, in load_base64 (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] return self.load_bytes(pybase64.b64decode(data, validate=True)) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/image.py", line 86, in load_bytes (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] image = Image.open(BytesIO(data)) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/PIL/Image.py", line 3572, in open (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] raise UnidentifiedImageError(msg) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x78fdf9a06610> (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] The above exception was the direct cause of the following exception: (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] Traceback (most recent call last): (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_chat.py", line 220, in create_chat_completion (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ) = await self._preprocess_chat( (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_engine.py", line 903, in _preprocess_chat (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] mm_data = await mm_data_future (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 612, in all_mm_data (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] modality: await asyncio.gather(items) (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] File "/home/venv_vllm_new/lib/python3.12/site-packages/vllm/multimodal/utils.py", line 251, in fetch_image_async (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] raise ValueError(str(e)) from e (APIServer pid=196656) ERROR 10-09 16:17:15 [serving_chat.py:245] ValueError: cannot identify image file <_io.BytesIO object at 0x78fdf9a06610>

Comment From: checkHup

When I upload images directly, they can be recognized

Comment From: checkHup

@YunKuiLu May I ask if you are familiar with this issue

Comment From: YunKuiLu

According to the error message, it seems that your vLLM tried to recognize the MP4 as an image and open it, which may be a configuration issue of vLLM.

ValueError: cannot identify image file <_io.BytesIO object at 0x78fdf9a06610>

I'm using Mac M4, and vLLM's support for Mac M4 is not yet well-established, so I cannot reproduce it locally. Perhaps you can ask the LLM.

Comment From: checkHup

According to the error message, it seems that your vLLM tried to recognize the MP4 as an image and open it, which may be a configuration issue of vLLM.

ValueError: cannot identify image file <_io.BytesIO object at 0x78fdf9a06610>

I'm using Mac M4, and vLLM's support for Mac M4 is not yet well-established, so I cannot reproduce it locally. Perhaps you can ask the LLM.

Thank you for your answer. I will consult VLLM and Qwen according to your answer. Thank you again