“If we have more challenging problems in the future, it’s relatively easy to just add 10, 20, 30, 40 different cameras based on the installed base and the technical background of the venue,” he says. “You might want to focus cameras on certain parts of the field or place cameras for specific purposes.”
This kind of scalability also leads to the concept of a “digital twin” in sports: a player can be virtually recreated by capturing video streams and positional data as he moves on the field, with his movements, likeness and hand gestures all digitally rendered in real time — something that’s typically only possible with expensive cameras and computer systems of the type used in Hollywood and video game productions.
Creating digital twins in sports could have uses beyond just refereeing: broadcasters could use them in digital overlays showing real-time statistics, or in virtual reality, allowing players to watch the game inside a VR headset.
Soccer is just the technology’s initial playground: Nearly every sport can derive value from the creation of a digital twin, and Genius hopes to expand into basketball and American football soon.
But while a football digital twin sounds interesting, can Dragon actually solve the game’s offside detection problem? After all, there have always been issues with previous VAR systems, which has not boosted trust in motion capture technology among football’s key stakeholders and fans.
Genius says it has been testing Dragon in various formats for several years, at the Premier League and at several other venues. The company employs several in-house analysts who project the tracking data onto a video format and compare it frame-by-frame with broadcast video to detect inconsistencies. This allows the team to continually retrain the model until, in theory, there are no more errors. Genius analysts consider this a foundational testing level, a baseline on top of which they layer other tests.
Dragon’s inputs are compared side-by-side with VAR and detection systems to verify basic accuracy, and are also manually verified. Engineers spend long hours with various sports stakeholders (coaches, players, management) trialing complex plays to ensure the system’s output makes sense. Every client considering using Dragon also has an in-house team that vets the system and validates the output.
“We’ve worked with organisations like FIFA and done extensive testing,” D’Auria says. “The Dragon system is approved by FIFA. They test it by putting the players in the Vicon (motion capture) system, and we follow the players and compare the data sets to look for errors. We’ve done this five or six times.”
Notably, despite evaluating the iPhone system side-by-side with VAR, both Genius and EPL representatives declined to provide WIRED with any specific testing information or results, stating that comparisons with previous motion capture systems are difficult because the quantity and quality of data generated is orders of magnitude different. Interestingly, both EPL and Genius refused to give any indication of how accurate their smartphone technology is compared to VAR.
Of course, the real assessment will be made by the fans and players, who will only believe it works once they see the Dragon in action. The absurdities of VAR over the last few years have understandably left a bad impression on many people when it comes to optical tracking.
But as semi-automated offside decisions are introduced for the first time in the UK this season, remember this isn’t just any old system with a new packaging. This is the next generation of motion capture, and it’s something that will be closely watched by those in the sports and AI communities. Fans will have little patience for more glitches with a motion capture-based system, and Genius and the EPL are confident they can rise to the challenge. Let’s see what happens. The match begins.
