The University of Hong Kong (HKU) Business School has released its latest comprehensive evaluation report on AI model image generation capabilities, assessing 15 text-to-image models and 7 multimodal large language models. The study highlights that ByteDance's Dreamina and Doubao, along with Baidu's ERNIE Bot, delivered outstanding results in both new image generation and image revision tasks.
The evaluation focused on two primary tasks: generation of new images and revision of existing images.
1. Generation of new images
The task included assessments on two aspects: image content quality and adherence to safety and responsibility standards.
- Content Quality: This was evaluated across three dimensions: alignment with instructions, image integrity, and image aesthetics
ByteDance's Dreamina excelled in content quality, achieving the top score of 1,123, followed by Baidu's ERNIE Bot V3.2.0, Midjourney v6.1, and Doubao.
- Adherence to Safety and Responsibility Standards: This covered topics including discrimination and bias, illegal activities, harmful or dangerous content, ethical concerns, copyright infringement, privacy violations, and portrait rights violations.
In this category, OpenAI's GPT-4o ranked first with an average score of 6.04, followed by Alibaba's Qwen V2.5.0 (5.49) and Google's Gemini 1.5 Pro (5.23). The report also noted that some models with strong content quality performed less well in safety and responsibility, reflecting uneven capabilities and highlighting potential societal risks if ethical safeguards are insufficient.
2. Revision of existing images
In this task, models were required to modify reference images uploaded by the user based on text prompts specifying the desired changes, either in terms of the style or content of the reference image.
Among the 13 models supporting image modification, Doubao, Dreamina, and ERNIE Bot V3.2.0 demonstrated strong performance, followed closely by GPT-4o and Gemini 1.5 Pro.
Related News:
DeepSeek achieves 20 mn daily active users, establishes two companies in HK
Comment