Write mJev-Qwen3-VL-4B-RLCD model card

README.md

@@ -1,204 +1,111 @@
 ---
 license: apache-2.0
-pipeline_tag: image-text-to-text
 library_name: transformers
+pipeline_tag: image-text-to-text
+base_model: Qwen/Qwen3-VL-4B-Instruct
+base_model_relation: finetune
+model_name: mJev-Qwen3-VL-4B-RLCD
+tags:
+  - mjev
+  - multimodal
+  - decision-making
+  - reinforcement-learning
+  - grpo
+  - rlcd
 ---
-<a href="https://chat.qwenlm.ai/" target="_blank" style="margin: 2px;">
-    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
-</a>
-
-
-# mJev
-
-mJev is a Jev-style multimodal decision model based on Qwen3-VL-4B-Instruct.
-It turns multimodal context into explicit choices with candidate probabilities.
-
-## Base model
-
-- Architecture: Qwen3-VL-4B-Instruct
-- Task: Multimodal probabilistic decision making
-
-The architecture, loading method, and usage examples below follow the base model.
-
-## About Qwen3-VL
-
-
-Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.
-
-This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities.
-
-Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment.
-
-
-#### Key Enhancements:
-
-* **Visual Agent**: Operates PC/mobile GUIs—recognizes elements, understands functions, invokes tools, completes tasks.
-
-* **Visual Coding Boost**: Generates Draw.io/HTML/CSS/JS from images/videos.
-
-* **Advanced Spatial Perception**: Judges object positions, viewpoints, and occlusions; provides stronger 2D grounding and enables 3D grounding for spatial reasoning and embodied AI.
-
-* **Long Context & Video Understanding**: Native 256K context, expandable to 1M; handles books and hours-long video with full recall and second-level indexing.
-
-* **Enhanced Multimodal Reasoning**: Excels in STEM/Math—causal analysis and logical, evidence-based answers.
-
-* **Upgraded Visual Recognition**: Broader, higher-quality pretraining is able to “recognize everything”—celebrities, anime, products, landmarks, flora/fauna, etc.
-
-* **Expanded OCR**: Supports 32 languages (up from 19); robust in low light, blur, and tilt; better with rare/ancient characters and jargon; improved long-document structure parsing.
 
-* **Text Understanding on par with pure LLMs**: Seamless text–vision fusion for lossless, unified comprehension.
+# mJev-Qwen3-VL-4B-RLCD
 
+**Jev, with senses. Multimodal context in, explicit decisions out.**
 
-#### Model Architecture Updates:
+[GitHub](https://github.com/SoMarkAI/mJev) · [Evaluation dataset](https://huggingface.co/datasets/Immortal-Zhang/mJev-Compositional-VQA) · [Deployment guide](https://github.com/SoMarkAI/mJev/blob/main/docs/models.md)
 
-<p align="center">
-    <img src="https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_arc.jpg" width="80%"/>
-<p>
+mJev-Qwen3-VL-4B-RLCD is SoMark's 4B multimodal decision model, fine-tuned with GRPO from [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct). It is designed to select among explicit candidates given visual context and a question.
 
+With the mJev runtime, each question returns a selected answer, candidate probabilities and raw logits. Multiple questions can share the same context, with optional prefix-cache reuse. This repository contains the full fine-tuned weights and processor/tokenizer files; the inference runtime lives in the [mJev code repository](https://github.com/SoMarkAI/mJev).
 
-1. **Interleaved-MRoPE**: Full‑frequency allocation over time, width, and height via robust positional embeddings, enhancing long‑horizon video reasoning.
+## Model at a glance
 
-2. **DeepStack**: Fuses multi‑level ViT features to capture fine‑grained details and sharpen image–text alignment.
+| Property | Details |
+| --- | --- |
+| Developer | SoMark |
+| Model family | Qwen3-VL, 4B |
+| Task | Multimodal probabilistic decision making |
+| Inputs | Images or video with text questions and candidate choices |
+| Decision interface | Candidate-label logits, normalized candidate probabilities and an argmax decision |
+| Post-training | GRPO with target-probability-weighted correctness rewards |
+| Weight format | Full model, BF16 Safetensors |
+| License | Apache 2.0 |
 
-3. **Text–Timestamp Alignment:** Moves beyond T‑RoPE to precise, timestamp‑grounded event localization for stronger video temporal modeling.
+## Evaluation
 
-This is the weight repository for Qwen3-VL-4B-Instruct.
+On a **195-question evaluation subset** of [mJev-Compositional-VQA](https://huggingface.co/datasets/Immortal-Zhang/mJev-Compositional-VQA), GRPO fine-tuning improves accuracy from **77.95% to 80.00% (+2.05 percentage points)**, answering four additional questions correctly.
 
+| Model | Correct / total | Accuracy |
+| --- | ---: | ---: |
+| Qwen3-VL-4B-Instruct — before GRPO | 152 / 195 | 77.95% |
+| **mJev-Qwen3-VL-4B-RLCD — after GRPO** | **156 / 195** | **80.00%** |
 
----
-
-## Model Performance
-
-**Multimodal performance**
+Both models were evaluated on the same questions, held out from RL training. Accuracy is the number of correct answers divided by the total number of questions. These results describe this evaluation subset; broader benchmark performance has not been established by this comparison.
 
-![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_4b_8b_vl_instruct.jpg)
+## Quick start
 
-**Pure text performance**
-![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_4b_8b_text_instruct.jpg)
+Use the mJev runtime to obtain candidate probabilities and decisions. The HF path scores candidate labels with a model forward pass; it does not require generating a free-form answer.
 
-## Quickstart
+Prepare Linux, Python 3.11+, a compatible NVIDIA GPU environment and system FFmpeg. Install the runtime and download this model:
 
-Below, we provide simple examples to show how to use Qwen3-VL with 🤖 ModelScope and 🤗 Transformers.
-
-The code of Qwen3-VL has been in the latest Hugging Face transformers and we advise you to build from source with command:
-```
-pip install git+https://github.com/huggingface/transformers
-# pip install transformers==4.57.0 # currently, V4.57.0 is not released
+```bash
+git clone https://github.com/SoMarkAI/mJev.git
+cd mJev
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install torchcodec==0.11.0+cpu --index-url https://download.pytorch.org/whl/cpu
+python -m pip install -e '.[hf-vl]' huggingface_hub
+
+export MODEL_DIR="$HOME/models/mJev-Qwen3-VL-4B-RLCD"
+hf download SoMarkAI/mJev-Qwen3-VL-4B-RLCD --local-dir "$MODEL_DIR"
+
+CUDA_VISIBLE_DEVICES=0 python demo_hf.py \
+  --model "$MODEL_DIR" \
+  --input examples/motion-demo/input.json \
+  --mode causal --numerics stable --projection full \
+  --prefix-cache --question-batch-size 3 \
+  --output outputs/mjev-decisions.json
 ```
 
-### Using 🤗 Transformers to Chat
-
-Here we show a code snippet to show how to use the chat model with `transformers`:
-
-```python
-from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
-
-# default: Load the model on the available device(s)
-model = Qwen3VLForConditionalGeneration.from_pretrained(
-    "Qwen/Qwen3-VL-4B-Instruct", dtype="auto", device_map="auto"
-)
-
-# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.
-# model = Qwen3VLForConditionalGeneration.from_pretrained(
-#     "Qwen/Qwen3-VL-4B-Instruct",
-#     dtype=torch.bfloat16,
-#     attn_implementation="flash_attention_2",
-#     device_map="auto",
-# )
-
-processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-4B-Instruct")
+The example asks three questions about a project-created video. Inspect `outputs/mjev-decisions.json` for decisions and candidate scores. For your own image, create an input JSON file with this structure and pass it to `--input`:
 
-messages = [
+```json
+{
+  "image": "your-image.png",
+  "context": "Inspect the supplied picture.",
+  "questions": [
     {
-        "role": "user",
-        "content": [
-            {
-                "type": "image",
-                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
-            },
-            {"type": "text", "text": "Describe this image."},
-        ],
+      "question": "Which object is visible?",
+      "candidates": ["A bicycle", "A car", "A bus"]
     }
-]
-
-# Preparation for inference
-inputs = processor.apply_chat_template(
-    messages,
-    tokenize=True,
-    add_generation_prompt=True,
-    return_dict=True,
-    return_tensors="pt"
-)
-inputs = inputs.to(model.device)
-
-# Inference: Generation of the output
-generated_ids = model.generate(**inputs, max_new_tokens=128)
-generated_ids_trimmed = [
-    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
-]
-output_text = processor.batch_decode(
-    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
-)
-print(output_text)
+  ]
+}
 ```
 
-### Generation Hyperparameters
-#### VL
-```bash
-export greedy='false'
-export top_p=0.8
-export top_k=20
-export temperature=0.7
-export repetition_penalty=1.0
-export presence_penalty=1.5
-export out_seq_length=16384
-```
+Media paths are relative to the input JSON file. See the [HF guide](https://github.com/SoMarkAI/mJev/blob/main/docs/hf.md) for the Python API, video inputs and memory settings. The weights retain the standard Qwen3-VL architecture and can also be loaded with Transformers; use the mJev runtime for the candidate-scoring interface described here.
 
-#### Text
-```bash
-export greedy='false'
-export top_p=1.0
-export top_k=40
-export repetition_penalty=1.0
-export presence_penalty=2.0
-export temperature=1.0
-export out_seq_length=32768
-```
+## Training and reward
 
+Before RL, a frozen candidate scorer records the probability `p_target` assigned to the ground-truth option. Each sampled label receives **+p_target when correct** and **-p_target when incorrect or invalid**. Rewards are bounded in [-1, 1], and examples with higher target probabilities produce stronger training signals.
 
-## Citation
+GRPO rollouts are constrained to a single candidate label, with no separate format reward. Reward scaling is disabled to preserve the target-probability weighting. A separate KL penalty limits drift from the reference model. Training updates the language model while keeping the vision tower and aligner frozen.
 
-If you find our work helpful, feel free to give us a cite.
+## Use and limitations
 
-```
-@misc{qwen3technicalreport,
-      title={Qwen3 Technical Report}, 
-      author={Qwen Team},
-      year={2025},
-      eprint={2505.09388},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2505.09388}, 
-}
+- Intended for visual question answering and decision workflows with explicitly defined candidate sets.
+- Candidate probabilities are relative to the supplied choices, not calibrated confidence estimates. Candidate wording, order and coverage can affect decisions.
+- This release supports visual inputs; it does not add audio support. The reported RL evaluation covers image questions, not video or audio benchmarks.
+- The 195-question comparison is an initial evaluation. It does not establish broad reasoning gains or statistical significance.
+- Input length, image resolution, video sampling and question batching affect memory use and latency. See the runtime documentation for supported settings.
 
-@article{Qwen2.5-VL,
-  title={Qwen2.5-VL Technical Report},
-  author={Bai, Shuai and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Song, Sibo and Dang, Kai and Wang, Peng and Wang, Shijie and Tang, Jun and Zhong, Humen and Zhu, Yuanzhi and Yang, Mingkun and Li, Zhaohai and Wan, Jianqiang and Wang, Pengfei and Ding, Wei and Fu, Zheren and Xu, Yiheng and Ye, Jiabo and Zhang, Xi and Xie, Tianbao and Cheng, Zesen and Zhang, Hang and Yang, Zhibo and Xu, Haiyang and Lin, Junyang},
-  journal={arXiv preprint arXiv:2502.13923},
-  year={2025}
-}
+## License and acknowledgments
 
-@article{Qwen2VL,
-  title={Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution},
-  author={Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jingren and Lin, Junyang},
-  journal={arXiv preprint arXiv:2409.12191},
-  year={2024}
-}
+Released under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). mJev builds on Qwen3-VL-4B-Instruct; we thank the Qwen team for the base model and the Jev project for the decision-oriented interface inspiration. Evaluation data retains its own license and terms.
 
-@article{Qwen-VL,
-  title={Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond},
-  author={Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren},
-  journal={arXiv preprint arXiv:2308.12966},
-  year={2023}
-}
-```
+For issues, implementation details and contributions, visit [SoMarkAI/mJev](https://github.com/SoMarkAI/mJev).