The sports industry continues to evolve as it incorporates various new technologies, and the front line of sports broadcasting is no exception.
SoftBank Corp. is a top partner for B.LEAGUE, Japan’s professional baseball league. B.LEAGUE held its “B.LEAGUE U15 CHALLENGE CUP 2019” tournament from March 25 to 27 at the Arena Tachikawa Tachihi in Tokyo. SoftBank Corp. utilized AI-powered cameras to broadcast this tournament without using a camera crew. SoftBank News took a closer look to see how these automated cameras work and the possibilities they hold for the future.
AI-powered cameras used to broadcast matches
Have a look at the video below. It appears to be a typical basketball game broadcast, but part of the game was actually filmed by cameras powered by AI.
Can you tell which is filmed by AI?
An AI-powered automated video production solution called “Keemotion” was used to film the game. With more than ten years of OTT* professional sports broadcasting under its belt, SoftBank Corp. used this solution as a new way to stream games.
-
- *Over The Top: the provision of messaging, audio and video content and services via the Internet.
How the Keemotion system works
The automated filming is conducted by two cameras: the main camera and the score camera. The main camera uses three lenses to film the on-court action from three directions. The three video feeds are instantly stitched together using servers to produce a single panoramic image.
The main camera installed in the ceiling. Three lenses are visible in the center.
The panoramic image taken by the main camera is then analyzed using AI. The optimal camera angles are automatically determined, and the game is streamed in real-time with zooming in and out and switching between angles.
The score camera films the scoreboard to obtain information on points and other data.
The other camera, the score camera, constantly films the scoreboard to keep track of scores, time and other data. This information is superimposed on the main camera image to display scoreboard data along with the streamed video.
Actual image from streamed video
Since the two cameras and AI servers are connected by LAN cables, servers need to be installed on-site at the venue close to the cameras. That means that these AI-powered broadcasts can only be performed at indoor venues with the required equipment. However, when 5G communication services are commercialized, broadcasts will not be constrained by these types of wiring and server setups. This might make it possible to broadcast sports games from anywhere in the country by using AI-powered cameras.
AI-powered cameras free up humans to pursue more creative tasks
Based on the results of the B.LEAGUE U15 CHALLENGE CUP 2019 broadcast, SoftBank Corp. plans to conduct further research and development to fully deploy AI-powered cameras. SoftBank News spoke to Toshihito Sakaki and Tetsuji Otsuka, two people at SoftBank Corp. involved in these future plans.
Toshihito Sakaki is a manager in SoftBank Corp.’s Sports Business Division. Sakaki is exploring business opportunities using Keemotion from a business producer position.
Tetsuji Otsuka is a manager in SoftBank Corp.’s Video Contents Division. Otsuka manages and supports Keemotion from a technical director position.
What are the advantages of using AI-powered cameras?
The biggest advantage is the reduction in costs related to video production. For most sports broadcasts, you need a team of more than 10 people, including the camera crew, the director, the commentators and so on. By introducing AI-powered cameras, the production could be done completely from a remote location.
You also get to reduce the number of risks, such as the cameraman getting sick on the day of the game or running into transportation problems. Since distance is not an issue, the camera angle switching could be done from a home in Tokyo for a game being played in Kyushu. In that sense it could greatly affect how camera crews work.
Does that mean camera operators could lose their jobs?
While some may have that concern, I don’t think AI would replace all the work required of camera operators in the current circumstances.
The good thing about people is that they can judge what to film based on the situation. They can shoot miraculous scenes during the game, film the bench during a time out and skillfully capture other scenes as only humans can. There is still a lot for people to do.
By actively leaving certain camerawork to AI, operators can devote more time to aspects that leverage their experience and sensibilities. With the spread of AI-powered cameras, I don’t think people will have their jobs taken away. On the contrary, they’ll be able to use their creativity more to make better video productions.
Operating AI-powered cameras to build a stronger team
Are there ways to utilize AI-powered cameras outside of sports broadcasts?
Scoreboard data filmed using Keemotion records all visual data, including who scored at what time. Administrators can add tags to organize data to filter out three-point plays, for example.
You could also film regular practice sessions. That way, if you wanted to check a play during a meeting, you could quickly find it. You wouldn’t have to worry about rewinding. AI-powered cameras could therefore be used as a tool to check team formation, player form and enhance play accuracy to strengthen the team.
The administrator screen allows the filtering of various data sets (not viewable by general users)
Sports broadcasts will become more interactive with technology
What are the main things you verified with this broadcast as you look toward full-fledged commercialization?
One thing we look at is how realistic the experience feels from a viewer standpoint. When a video is produced by AI, it’s natural to ask if it can be filmed in a way that’s interesting to humans.
So, to check the level of completion, we looked for things such as where the cameras should be focused on given the development of the game and which scenes should be filmed as close-ups.
On a technical level, AI automated cameras still can’t differentiate between referees and players. Accordingly, we are trying to incorporate technology that recognizes people. If that’s possible, it might be possible to follow certain players during video broadcasts.
Using AI to automatically follow players of interest (simulated image)
In closing, what would you like to achieve with the spread of AI-powered cameras?
For budgetary reasons, many minor leagues can’t broadcast their games. If they could broadcast at low-cost using AI-powered cameras, minor league and amateur games could see more broadcasts.
With that, more people could enjoy minor league sports, and this could give more vitality to the sports industry overall. We would very much like to make this a reality.
Currently Keemotion is mainly used for indoor sports like basketball, volleyball, futsal, judo and kendo. In the future we’d like to expand to baseball, soccer and other outdoor sports. We’d like to contribute even more excitement to the world of sports.
Thank you.
(Original article posted on April. 9, 2019)
By exwrite
Photography by Yuji Yamazaki
