Laying the foundation for
the next leap by building
a Japanese-based LLM

As a wholly owned subsidiary of SoftBank, our mission is to develop technology centered on generative AI. Currently, we are working to complete a multimodal^*1 large language model (LLM) with approximately 390 billion parameters^*2 developed using Japanese datasets^*3 by the end of FY2024. In the medium- to long-term, we aspire to develop models with approximately 1 trillion parameters.

When we look ahead to the future, if generative AI becomes a core part of digital services, I believe having a Japanese-based LLM domestically is crucial. Currently, we rely on overseas companies for the majority of digital services, such as operating systems and cloud services. The digital trade deficit is growing each year and reached ¥5.5 trillion in 2023. If all the commonly used generative AI also comes from overseas, I fear that this deficit will continue to grow as a result of paying license fees and other related expenses. Through the LLM we are developing, we hope to create a structure that keeps the national wealth within Japan while contributing to the growth of SoftBank.

We expect our LLM to have the competitive advantage of accurate responses taking into account Japanese business practices, cultural, and linguistic nuances. LLMs could be described as “collections of knowledge with linguistic structures,” but our LLM will be trained on “information written in Japanese that exists in the Japanese-speaking world.” Therefore, the linguistic structure of Japanese is reflected in the model, resulting in high accuracy.
LLMs developed by overseas companies are trained on “information written in English that exists in the English-speaking world.” Although they can also handle requests in Japanese, the knowledge and linguistic structures acquired from English data are strongly reflected within these LLMs. As a result, Japanese output may appear translated and feel unnatural to native Japanese speakers.
In addition, there are major differences in culture and business practices between the English-speaking and Japanese-speaking worlds. To take the banking industry as an example, in Japanese, a bank account that has been dormant for a long time is written using the Chinese character for “sleep.” If this is translated as “sleeping account” instead of “dormant account,” it may undermine customer trust. To take an example from the travel industry, while traveling by private cars is the norm in the US, in Japan it is more common to use trains and tour buses. When providing a solution for creating travel plans using an LLM developed by an overseas company, if the only response customers get is something like “we have booked a hotel in that direction, here are the famous tourist spots, please use your own car to get there,” the satisfaction level of customers in the Japanese-speaking world will probably drop.
Of course, you can feed this kind of information one-by-one to train LLMs developed by overseas companies, but it is difficult to cover all of the business practices, cultural, and linguistic nuances. Recently, there have been some initiatives by Japanese companies to enhance foreign LLMs with additional Japanese learning, but I believe the same issues remain.
We would like to differentiate ourselves by developing a Japanese-based LLM that can respond more naturally and accurately, providing it to companies in Japan.

Once we create a large parameter model, we can take flexible approaches tailored to various purposes, so we would like to prioritize creating a large parameter LLM first, even if it takes a bit longer to launch.
The current trend in creating generative AI models is to take an approach called “model distillation,” which involves taking a large parameter LLM and reducing it to a smaller, optimized version tailored to the intended use case. Our policy is to take the same approach. By distilling an LLM according to the intended use case, we can create a small language model (SLM), such as one specializing in conversation or medical care. These SLMs can inherit some of the capabilities of the LLM, meaning they can maintain high Japanese accuracy while also providing quick responses and consuming less power. When creating a small parameter LLM from scratch, often it will be designed to be specialized for a particular field. This makes it difficult to apply it to other fields. For example, an SLM that can answer questions about Japanese statutory law while also being able to handle everyday conversation to a certain extent by inheriting the capabilities of the LLM would probably better capture customer needs, rather than an LLM that can answer questions about Japanese statutory law but has difficulty handling everyday conversation.

Rather than monetizing the Japanese-based LLM on its own, we aim to create added value for enterprise customers and monetize it either by “providing a platform for using the LLM” or “incorporating the LLM into solutions and providing them as a service.”
By “providing a platform for using the LLM,” I mean a platform-as-a-service (PaaS) business that enables customers to use our Japanese-based LLM as one of the functions on our cloud service platform. For example, let's say we want to build a service on SoftBank's cloud service that can show drivers of electric vehicles where they can recharge their batteries on the highway. With conventional digital services, the location of charging stations would be displayed on a screen and drivers would be advised to charge their electric vehicles when their batteries run low. However, if we can use the functions of the Japanese-based LLM to autonomously determine which charging station will be most cost-effective for the driver and guide them there in natural Japanese, this will be an added value for the customer.
By “incorporating the LLM into solutions and providing them as a service,” I am referring to developing a software-as-a-service (SaaS) solution in-house that incorporates the Japanese-based LLM and providing it to enterprise customers. SoftBank is developing an automated call center solution that uses generative AI. If the Japanese-based LLM can instantly provide responses to customer inquiries in natural Japanese, it would be immensely beneficial for call center staff, improving retention rates and reducing recruitment costs, thus adding value for clients. If we can create added value like this, it will translate into a commensurate share of SoftBank's earnings.
In addition, leasing the AI computing infrastructure an infrastructure-as-a-service (IaaS) solution to governments, municipalities, and private companies, as well as consulting on the implementation of generative AI, are expected to generate revenue.
In the short- and medium-term, I expect that our Japanese-based LLMs will help boost the value of PaaS and SaaS solutions. In the long-term, I expect that the generative AI system itself will “generate” value beyond human capabilities. Let's say that human being spends five years verifying the effectiveness of 10,000 potential drug ingredients in order to develop a medicine that treats a new disease. If generative AI could narrow down the number of potential ingredients to 100 and verify its effectiveness in a year, it would add immense value. Similarly, if generative AI could find potential materials for building a lighter aircraft or potential electrodes for a new type of battery, that would also yield tremendous value.
In this way, we hope to create added value by building platforms and services that leverage LLM in order to contribute to earnings for SoftBank as well as the entire Group.

As a basic approach, different LLMs will be used for different purposes, ultimately benefiting users. Amid such differentiation, we aim to provide the essential Japanese-centric LLMs.
We are seeing a variety of LLMs emerging with different features, and I expect that going forward, people will be able to understand the strengths and weaknesses, as well as what each LLM knows and does not know, and people will combine them to offer services. In the future, I believe generative AI could decide on which generative AI to use. Taking medical care as an example, we would first prepare a high-capacity LLM with strong Japanese language skills that can accurately understand the symptoms described by the patient. Next, we will have smaller specialized LLMs providing expertise in fields such as surgery, internal medicine, and image diagnostics. Then, the LLM with strong Japanese language skills will discuss the treatment plan according to the patient's symptoms with the specialized LLM in a particular field. Since the discussion will be between computers and is specialized in a particular field, it will probably be over very quickly. Finally, the LLM with strong Japanese language skills will explain the outcome to the patient in appropriate terms. When considering such a use case, I think the specialized LLMs do not necessarily need to be developed in-house, but the LLM with strong Japanese language skills that can accurately grasp the content of the patient's explanation will always be necessary. I believe our Japanese-based LLM will play a central role in such use cases and make a significant contribution.
We are working on this with a sense of urgency, fearing that if LLMs become the central focus of all digital services in the future, missing the current challenge may mean losing the final opportunity. For the time being, our R&D phase will continue, but we want to continue taking on challenges so that we can use the results as stepping stones for SoftBank's next leap in growth.