How Wimbledon and Watson are using AI to curate video highlights

14

The Wimbledon tennis championship is among the world’s most prestigious sporting events. First held in 1877, it’s also the oldest tennis tournament in the world. Much has changed in the past 142 years – from the players’ attire to the quality of their equipment – but behind the scenes an AI evolution is transforming how footage is captured and curated for public consumption.

IBM’s Watson helps Wimbledon collate and produce highlights.

For context, Wimbledon has 18 grass courts, which host nearly 700 matches over a two-week period, spanning men’s and women’s singles, doubles, mixed doubles, and more. The six main “show courts” alone – which host the most prominent matches – each average around four matches a day. For the first time, last year all 18 courts were televised, using a mixture of “manned” and “robotic” cameras. This can give viewers unprecedented access, but it also creates a heck of a lot of footage, too much for mere mortals to sift through in a timely manner. This is where IBM believes automation can help.

IBM has actually worked with Wimbledon organizers at the All England Lawn Tennis & Croquet Club (AELTC) for three decades, taking care of general IT operations, cybersecurity, and all customer-facing digital properties, including apps and websites where fans catch up on all the action.

For the past six years, IBM has lent its Watson AI smarts to all manner of applications, but more recently it has been ramping up Watson’s use at Wimbledon, where it helps capture all the best bits of each game and package them as highlights within two minutes of a match ending, updating what had been a labor-intensive manual effort.

We’re taking an activity that traditionally would take a lot of manpower, create a lot of costs for the club, [and] take a lot of time and allowing Wimbledon to get those highlights out as quick as it can,” said Simon Boyden, chief architect at IBM.”

AI adoption process at Wimbledon. Source: Wimbledon.

To do this, the AI analyzes such elements as players’ gestures and reactions (e.g. celebratory air punches and fist pumps) and listens to crowd noises, like gasps and cheers. For Wimbledon 2019 highlights, IBM says it trained Watson to leverage even more acoustics data, such as detecting each strike of the ball – which helps in terms of enabling tighter automated cropping of highlights around key action in games and could also help identify long rallies.

Wimbledon highlights, powered by Watson.

AI bias

Of course, there is the perennial issue of AI bias to contend with. The Wimbledon environment contains many scenarios that could mislead an automated system – Center Court will have more fans (and hence more noise), while a megastar such as Roger Federer will likely have more vocal support than the world’s 89th ranked player who’s on Court 18. To detect the best action regardless of these external factors, IBM has trained Watson to better recognize trends and patterns in the acoustics – creating a kind of fourth data point to reduce the chance of bias.

This result is enabled through AI OpenScale, which IBM debuted last year and which offers a suite of autonomous bias detection and mitigation tools – continuously monitoring for prejudicial decision-making across AI applications. Specific to Wimbledon, this technology helps Watson “learn” the overall noise of a crowd and use that as a kind of normalizer so that any subsequent high points will be measured relative to the baseline.

Ace stats

In producing video highlights, Watson also taps gameplay statistics for further context. These stats are generated by tennis players (often county-level players) who are employed to sit courtside to identify subjective data points like unforced errors, in addition to other statistical tidbits – such as aces or double faults – and milestone moments in a tennis match, such as breakpoint, setpoint, or match point.

This collaborative approach actually helps demonstrate the role people play in training AI systems – at least for now. How close we are to automating the data capture process is another question.

This is very much an active area,” Boyden said. “You can (already) do 70% or 80% of it in an automated way – that would be great in a club training situation or something like that, but the challenge we have for Wimbledon is that they are looking for 100% accuracy all of the time.

What will likely happen going forward is a series of incremental changes, with Watson gradually taking on more responsibilities. For now, it is aiding people, but in the next step, it could start making recommendations. “The point where we’re 100% convinced that Watson will get it right every single time? That’s probably going to be a few more years,” Boyden added.

Objectives from this cooperation between Wimbledon & IBM:

  • Boost fan experience on Wimbledon platforms by accelerating and improving the match highlight production process across a 13 day period.
  • Expand content on offer to fans by producing a solution that could easily scale to provide highlights from Wimbledon courts that weren’t previously covered. Support Wimbledon’s strategic objective to behave more like a media organization by reducing dependence on third parties, thus increasing Wimbledon’s control over content output
  • Produce output with a high production value for pickup by international broadcast partners
  • Reduce strain on editorial and content teams, enabling them to focus on other areas generating further value add content. Contribute to improved video performance across all Wimbledon platforms during The Championships

Results achieved:

  • In a competitive sporting and digital landscape, Cognitive Highlights helped drive 69.9m digital visits during The Championships (69.4m in 2016) and 436m page views (395m in 2016)
  • 4M net new highlights video views produced without human intervention
  • Total of 250 highlight packages created – +252% from 2016 Championships
  • Reduced turnaround time from an hour minimum to 15 minutes
  • Automation of the highlight process enabled the content team to focus on serving the digital community, ultimately generating a record total of 200M video views across all platforms.
  • Unused highlight footage (due to rights restrictions) continually utilized by year-round team to maintain audience engagement outside of The Championships.
  • The new Cognitive Highlight approach drove significant press interest around Wimbledon’s digital platforms with 247 earned articles across 31 countries up 21% Y2Y.
  • First of a kind technical solution that used existing IBM patents.

 You can watch a highlight video:

Ryan Hoang (VentureBeats, IBM, Wimbledon)

Related posts: