From April 3 to 4, 2024, a pioneering trial took place at Fukuoka PayPay Dome (now MIZUHO PayPay Dome FUKUOKA) to demonstrate a new way to experience baseball game viewing with augmented reality (AR). Get a first hand look at how this advanced technology developed by SoftBank Corp. (TOKYO: 9434) is being integrated with spectator sports by way of a Fukuoka SoftBank HAWKS exhibition match that took place in March.
Mascots Harry Hawk and Honey Hawk escort spectators to their seats
After donning AR glasses, which are linked to a smartphone app, the first thing spectators experience is a navigated guide to their seats.
Who’s that? It’s Harry Hawk, a Fukuoka SoftBank HAWKS mascot. After he appears in the AR display, a high-five with Harry goes off without a hitch!
The sensation of giving Harry a high-five is accompanied by realistic sound effects to heighten the sense of anticipation before the game.
Next, arrows appear in the AR display to show a path that resembles a tunnel of light. Even people without a good sense of direction should be able to find their way to their seats easily.
The illuminated pathway brings spectators to another mascot, Honey Hawk, who is waiting just around the corner for another high-five.
AR effects amplify the game viewing experience
What could possibly enhance the spectator experience in a VIP box seat?
At the end of the fifth inning, something suddenly appears in the display of the AR glasses.
A combination of real and virtual elements creates visuals to get the crowd fired up.
Other AR features make it possible to write and send personalized messages to players and show signs of support with virtual baseballs. These interactive features greatly enhance the spectator experience!
Participants can also watch a post-game video of what they experienced.
Advanced Technologies Bring a New Dimension to Sports Viewing
A leader of SoftBank’s technical team responsible for this demonstration spoke about the technology in depth.
Tetsuji Otsuka
Director, Product Development Department
Content Promotion Division, Service Planning Division
Consumer Business Promotion Unit, SoftBank Corp.
Otsuka oversees product development for content-related services, including content for sports, coaching, Web3, and XR.
Otsuka: To guide spectators to their seats, we utilized a technology called Visual Positioning System (VPS), which detects a person's location and the direction they are facing using camera footage taken from the AR glasses. One characteristic of VPS is that it struggles in areas where the scenery is similar, and this makes it difficult to determine a person's position or direction. When there is confusion about which direction to proceed, the guidance can't be displayed, so we used a 'multi-sensor' that uses Bluetooth Low Energy (BLE), a technical standard that enables low-power communications.
That was used with the guided pathway to the seats.
Otsuka: To determine the position of someone in such an area, we send signals to their smartphone from beacons installed every few tens of meters along the stadium's corridors. Once the approximate location of a person is determined based on their distance from a beacon, VPS then adjusts itself to pinpoint the exact location. When users arrive in front of the Harry and Honey mascot panels, the AR glasses are triggered to display a high-five and give directions.
What technologies were used for the in-game AR productions?
Otsuka: After the bottom of the fifth inning, for instance, when we want to trigger an event using the AR glasses, relying solely on cellular networks like 4G or 5G can lead to instances where the event may or may not be activated for some people. This is because connecting tens of thousands of smartphones to the server directing the AR productions at the same time can overload the network.
Since crowd unity is essential with spectator sports, we employed a 'multi-network' setup that combined 4G, 5G, and BLE to synchronize the event activation timing. By sending event activation notifications through two networks to the smartphones, we increased the event trigger reliability.
We’ll continue to work toward creating a smart stadium environment that makes people want to come back and cheer for their team in person.
- *
Images shown with the AR glasses are simulated.
Related News
(Posted on May 22, 2024, Original article posted on April 8, 2024)
by SoftBank News Editors
