The Importance of Open Models in the Development of the Large Telecom Model (LTM)

1. Why Telecom Models Need Open Foundations

The capabilities of generative AI models are improving rapidly alongside advances in general-purpose foundation models. When building domain-specific models for telecom such as its Large Telecom Model (LTM), combining Sarashina, an in-house model with open models makes it possible for SoftBank to inherit the latest capabilities from global foundation models without having to replicate that investment.

A key strength of industry-specific models is the ability to continuously improve them by using open models as their base and applying telecommunications-specific training to improve accuracy for telecom-specific tasks such as network planning, configuration, and optimization. As open models become more capable, new telecom use-cases become feasible without rebuilding base models from scratch, reducing both development time and cost.

The role of open models is not limited to serving as base models. Large Language Models (LLMs) are also widely used in data preprocessing and synthetic data generation, such as creating privacy-preserving, representative datasets from network data, standards documents, and operational knowledge accumulated by telecommunications operators. Using open models for these pipelines allows operators to generate and curate telecom-specific training data entirely within their own environments, improving privacy, compliance, and cost efficiency.

However, many models described today as “open-source models” do not fully disclose their provenance, including their training data and training recipes. In practice, they are often “open-weight models” for which only the trained model weights are made available. Although such models can be deployed in on-premises environments and further trained, it is difficult to adequately verify how they were developed if the training data, data-processing methods, and other elements of the training recipe are not disclosed. With these models, there are limits to how far users can trace biases and risks in the training data back through the development process, assess them, and manage model behavior.

As introduced in our previous article*, SoftBank is pursuing a full-stack sovereign AI approach in which it manages and governs the entire stack, from computing infrastructure, data, models, and agents to actual autonomous networks. At the model layer, this requires management and governance not only of the inference environment, but also of the training data and development process. The availability of open models that provide transparency into their training data and development processes is important not only for SoftBank, but also for Japan’s ability to build and operate its own sovereign AI infrastructure.

* For more information on SoftBank’s full-stack sovereign AI approach, please refer to this article.

2. Continuous Model Development Using NVIDIA Nemotron

Based on this approach, SoftBank makes extensive use of NVIDIA Nemotron models in the development of LTM. The Nemotron model family delivers the best-in-class performance among Western open models and is also one of the few model families that broadly publishes model weights, training data, and training recipes across multiple stages of development, including pretraining and post-training. Users do not merely use trained model weights, as with other open-weight models; they can also understand how the model was developed and continue their own training based on the published assets. Making high-performance, transparent open models broadly available gives more people opportunities to benefit from advances in AI and build on those advances, creating significant value for society as a whole.

For its LTM, SoftBank works across every stage of model development: continual pretraining (CPT) to add telecommunications-domain knowledge, supervised fine-tuning (SFT) to strengthen capabilities for specific use cases, and reinforcement learning (RL) to improve reasoning capabilities or learn more appropriate actions from simulation environments.
Because Nemotron publishes data and recipes for each of these stages, SoftBank can seamlessly integrate LTM-specific training data and recipes into the development process and independently continue the downstream development of specialized models.

It is also important that Nemotron is available in different model sizes: Ultra, Super, and Nano. This enables an approach in which telecommunications-specific training is first applied to the latest high-performance large models, from which models of an appropriate size can then be derived for each use case. Specifically, Ultra-class large models can be used for tasks requiring advanced reasoning, while Nano-class small models can be used for use cases with well-defined tasks where low-latency responses are more important, allowing model size to be selected according to the capabilities required. This model deployment approach is also highly compatible with the AI-RAN concept, especially AI Grid, in which computing resources are deployed hierarchically across central data centers and the network edge.

When using LTM as an AI agent, an agent harness that controls the tools available to the agent, its access permissions, and approval conditions is also essential. In addition to collaborating on AI infrastructure and model development, SoftBank and NVIDIA are working together across the layers that will constitute future autonomous networks, including agent harnesses and validation using digital twins, to build a safe and verifiable agent execution platform.

SoftBank will continue to use open models such as Nemotron that combine performance and transparency and are compatible with its full-stack sovereign AI approach. By combining in-house technologies with open models, SoftBank will continue to advance full-stack sovereign AI for the telecommunications industry centered on LTM. By making the resulting capabilities available in a form that telecommunications operators can manage themselves, SoftBank aims to enable Level 4 or higher autonomous networks across the telecommunications industry.