Hon Hai Research Institute Launches Traditional Chinese LLM With Reasoning Capabilities

InfiniBand networking, and finished in just about four weeks. Compared with inference models recently launched in the market, the more efficient and lower-cost model training method sets a new milestone for the development of Taiwan's AI technology.

FoxBrain is based on the Meta Llama 3.1 architecture with 70B parameters. In most categories among TMMLU+ test dataset, it outperforms Llama-3-Taiwan-70B of the same scale, particularly exceling in mathematics and logical reasoning (For TMMLU+ benchmark of FoxBrain, please refer to Fig.1). The following are the technical specifications and training strategies for FoxBrain:

Established data augmentation methods and quality assessment for 24 topic categories through proprietary technology, generating 98B tokens of high-quality pre-training data for Traditional ChineseContext window length: 128 K tokensUtilized 120 NVIDIA H100 GPUs for training, with total computational cost of 2,688 GPU daysEmployed multi-node parallel training architecture to ensure high performance and stabilityUsed a unique Adaptive Reasoning Reflection technique to train the model in autonomous reasoning

In test results, FoxBrain showed comprehensive improvements in mathematics compared to the base Meta Llama 3.1 model. It achieved significant progress in mathematical tests compared to Taiwan Llama, currently the best Traditional Chinese large model, and surpassed Meta's current models of the same class in mathematical reasoning ability. While there is still a slight gap with DeepSeek's distillation model, its performance is already very close to world-leading standards.

FoxBrain's development – from data collection, cleaning and augmentation, to Continual Pre-Training, Supervised Finetuning, RLAIF, and Adaptive Reasoning Reflection – was accomplished step by step through independent research, ultimately achieving benefits approaching world-class AI models despite limited computational resources. This large language model research demonstrates that Taiwan's technology talent can compete with international counterparts in the AI model field.

Although FoxBrain was originally designed for internal group applications, in the future, the Group will continue to collaborate with technology partners to expand FoxBrain's applications, share its open-source information, and promote AI in manufacturing, supply chain management, and intelligent decision-making.

During model training, NVIDIA provided support through the Taipei-1 Supercomputer and technical consultation, enabling Hon Hai Research Institute to successfully complete the model pre-training with NVIDIA NeMo. FoxBrain will also become an important engine to drive the upgrade of Foxconn's three major platforms: Smart Manufacturing. Smart EV. Smart City.

The results of FoxBrain is scheduled to be shared for the first time at a major conference during NVIDIA GTC 2025 Session Talk "From Open Source to Frontier AI: Build, Customize, and Extend Foundation Models" on March 20.

About Hon Hai Research Institute

The institute has five research centers. Each center has an average of 40 high technology R&D professionals, all of whom are focused on the research and development of new technologies, the strengthening of Foxconn's technology and product innovation pipeline, efforts to support the Group's transformation from "brawn" to "brains", and the enhancement of the competitiveness of Foxconn's "3+3" strategy.

About Foxconn here.

Photo - https://mma.prnewswire.com/media/2637576/Fig_1_TMMLU__benchmark_results_FoxBrain_Meta_Llama_3_1_70B_Taiwan_Llama_70B.jpg

View original content:https://www.prnewswire.co.uk/news-releases/hon-hai-research-institute-launches-traditional-chinese-llm-with-reasoning-capabilities-302396908.html