SoftBank Corp.'s Large Telecom Model
Achieves Top-Tier Ranking
in GSMA Open-Telco LLM Benchmarks

Enhancing LTM through a telecom-specialized learning framework

In developing generative AI foundation models, simply applying a general-purpose LLM is insufficient to achieve performance at a level suitable for real-world deployment. In the telecommunications domain, models must not only understand complex technical specifications but also perform domain-specific question answering and interpret operational logs, which requires a deep understanding of sophisticated technical systems and telecom-specific contexts. Accordingly, a structured learning framework specialized for the telecommunications industry is essential.

To meet these requirements, SoftBank established a telecom-specialized learning framework (the "Framework") for LTM, systematizing both data design and training processes while taking into account the complex structures and interdependencies unique to telecommunications network data.

The Framework utilizes telecommunications-related public datasets as well as proprietary datasets held by SoftBank, including network data and expertise in network design, management, and operations. LTM is enhanced through staged additional training that integrates continual pre-training^*2, fine-tuning, and reinforcement learning^*3.

In addition, telecom domain data includes not only conventional document-based materials but also tabular data and code-based descriptions in diverse formats. These datasets are reorganized and transformed into synthetic data optimized for each training stage and objective. Furthermore, SoftBank improves training data quality through LLM-based data filtering and enhances learning efficiency and model performance through hyperparameter optimization (HPO)^*4 using small language models (SLMs).

Through this Framework, SoftBank has established a foundation that enables reproducible improvement of telecom-domain performance while flexibly adapting to data expansion, updates to base models, and evolving business requirements.

[Notes]

1. *2
   Continual pre-training refers to a method in which a general-purpose model is further trained on large volumes of domain-specific data to incorporate specialized knowledge.
2. *3
   Reinforcement learning is a method in which AI autonomously learns optimal actions by receiving rewards based on outcomes and maximizing cumulative rewards.
3. *4
   Hyperparameter optimization (HPO) refers to the process of optimizing model training parameters, such as learning rates and epoch numbers, using search algorithms prior to training.

Achieving a top-tier ranking in the GSMA Open-Telco LLM Benchmarks

Through the advancement enabled by this Framework, LTM achieved a top-tier overall ranking in the GSMA Open-Telco LLM Benchmarks, led by the GSMA, the global telecommunications industry organization.

The GSMA Open-Telco LLM Benchmarks evaluate LLM performance across multiple datasets reflecting real-world telecommunications operations, including telecom specification comprehension, domain-specific question answering, operational log interpretation, mathematical reasoning in telecom contexts, and configuration description. Rankings are determined based on the average scores across all evaluation dimensions.

This achievement demonstrates the effectiveness of LTM as a telecom-specialized model leveraging the data assets and operational expertise accumulated by SoftBank as a telecommunications operator. It also highlights the international recognition of SoftBank's capabilities and technological expertise in developing generative AI foundation models.

SoftBank will continue to promote the full-scale internal adoption of LTM to reduce reliance on individual expertise in network operations, alleviate operational workloads, and improve operational efficiency, thereby contributing to the delivery of higher-quality mobile network services.

Ryuji Wakikawa, Vice President, Head of the Research Institute of Advanced Technology at SoftBank said:
"SoftBank has been developing a telecom industry–specific LTM, trained on telecom expertise and real operational data, and achieved top-class results in the GSMA Open-Telco LLM Benchmarks. This demonstrates that our training foundation is also at a high international standard. By leveraging LTM, SoftBank will thoroughly enhance its operations and lead the advancement of the telecommunications industry."

Louis Powell, Director of AI initiatives, GSMA, said:
"Telecom networks demand precision and context that general-purpose AI often struggles to deliver. By testing models against telecom-relevant datasets and tasks, the GSMA Open-Telco LLM Benchmarks spotlight genuine capability improvements. SoftBank's top-tier ranking is a strong example of that progress, and exactly the kind of momentum the industry needs as it scales AI responsibly into operations."

For more details on the Large Telecom Model (LTM), please refer to the press release dated March 19, 2025, titled "SoftBank Develops Telecom-Industry-Focused Generative AI Foundation Model 'Large Telecom Model (LTM)'."

About GSMA

The GSMA is a global organisation unifying the mobile ecosystem to discover, develop and deliver innovation foundational to positive business environments and societal change. Our vision is to unlock the full power of connectivity so that people, industry, and society thrive. Representing mobile operators and organisations across the mobile ecosystem and adjacent industries, the GSMA delivers for its members across three broad pillars: Connectivity for Good, Industry Services and Solutions, and Outreach. This activity includes advancing policy, tackling today's biggest societal challenges, underpinning the technology and interoperability that make mobile work, and providing the world's largest platform to convene the mobile ecosystem at the MWC and M360 series of events.
We invite you to find out more at gsma.com.