UI-Venus-1.5 Technical Report

Changlong Gao, Zhangxuan Gu, Yulin Liu, …, Yichen Gong, …, Weiqiang Wang

GitHub

• UI-Venus-1.5 is the next-generation GUI agent built at Ant Group, advancing the UI-Venus series with improved multi-turn interaction and cross-platform generalization capabilities.

UI-Venus Technical Report: Building High-Performance UI Agents with RFT

Zhangxuan Gu, Zhengwen Zeng, …, Yichen Gong, …, Weiqiang Wang

GitHub

• UI-Venus is a screenshot-based UI agent built on Qwen2.5-VL and fine-tuned via reinforcement fine-tuning (RFT), achieving state-of-the-art performance on UI grounding and navigation benchmarks.

Figstep: Jailbreaking large vision-language models via typographic visual prompts

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, Xiaoyun Wang

Project

• This work exposes safety vulnerabilities in Large Vision-Language Models (LVLMs) and proposes FigStep, a black-box jailbreak algorithm that converts harmful text into visual modality to bypass safe guardrail, achieving an 82.50\% attack success rate. Our analysis highlights deficiencies in visual embedding safety alignment and underscores the need for robust cross-modality safety solutions. Our findings underscore the urgent need for robust cross-modality safety alignment techniques to secure LVLMs against such attacks.

Safety Misalignment against Large Language Models

Yichen Gong, Delong Ran, Xinlei He, Tianshuo Cong, Anyu Wang, and Xiaoyun Wang

• This work introduces a framework to evaluate various safety misalignment attacks against LLMs, revealing vulnerabilities to attacks like supervised fine-tuning and our novel SSRA. We propose SSRD to effectively re-align models, preserving safety after fine-tuning.

JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models

Delong Ran, Jinyuan Liu, Yichen Gong, Jingyi Zheng, Xinlei He, Tianshuo Cong, Anyu Wang

Project

• This work analyzes jailbreak evaluation methods for LLMs. We propose a taxonomy of jailbreak evaluators and introduce JailbreakEval, a toolkit to streamline and standardize jailbreak evaluation, advancing the efficiency and fairness of evaluation.

Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging

Tianshuo Cong, Delong Ran, Zesen Liu, Xinlei He, Jinyuan Liu, Yichen Gong, Qi Li, Anyu Wang, Xiaoyun Wang

Project

• This work studies the robustness of IP protection techniques for Large Language Models (LLMs) under model merging. While watermark fails in merged models, fingerprint remains effective, highlighting the need to address IP protection in model merging scenarios.