By Tencent AI Platform Dept. (AIPD)

HuggingFace Space: https://huggingface.co/Tinytron

Github Space: https://github.com/orgs/Tinytron/repositories

—— See Also: Tencent AI Arena、Tencent GiiNEX

Table of Contents

<aside> ⚠️

The models, datasets, training methods, and code implementations released by Tinytron are designed to ensure the reproducibility of technical research and demonstrations. The models themselves have not been optimized for any potential use cases beyond the EdgeLLM challenge, nor have they been evaluated for tasks outside the public EdgeLLM Benchmark. Therefore, it is not recommended to use or deploy these models in real-world production environments.

</aside>

1 - Methods for Track1: Compressing LLMs

Challenge Objective:

Track 1 focuses on adapting three open-source models—Llama-3.1-8B-Instruct, Qwen2-7B-Instruct, and Phi-2—for smartphone deployment using designated datasets and teacher models. The goal is to optimize these models for efficient mobile operation while maintaining their performance on downstream tasks.

Effectiveness Evaluation:

The compressed models are evaluated on these datasets:

• Commonsense QA (Multiple-choice common-sense reasoning)
• FewCLUE-chid (Chinese idiom cloze test, multiple-choice)
• HumanEval (Programming tasks, generative)
• GSM8K (Grade-school math, generative)