A Wisconsin company's AI tracked the Chinese spy balloon. Here's what to know about Synthetaic

ASHWAUBENON - A Wisconsin-based company used artificial intelligence and satellite imagery to independently track the Chinese spy balloon's path across the United States before it was shot down.

The New York Times and Wired have published stories about how Delafield-based Synthetaic took founder and CEO Corey Jaskolski's "incredibly crude hand drawing" of what the balloon would look like from space, daily satellite images, and artificial intelligence to quickly find the balloon and trace its path halfway around the globe before it was shot down Feb. 4 off the coast of South Carolina.

Jaskolski said the March 20 New York Times report in particular provided independent validation of Synthetaic's findings and of what the company's Rapid Automatic Image Categorization software can do. He said the work shows the company's technology can be effectively used at a global level.

The balloon "was the ultimate needle in a haystack," Jaskolski said. "We've shown we've developed the magnet to find the needle."

Jaskolski hopes the company's ability to track the spy balloon will get companies, non-government organizations and other groups thinking about other needles Synthetaic could help find. It also provided a showcase for Green Bay-based TitletownTech, one of the company's first investors.

How does Synthetaic use artificial intelligence?

Jaskolski calls the company's program Rapid Automatic Image Categorization (RAIC, pronounced "rake"). It trains AI to comb through huge amounts of data, like satellite images or security camera footage, in search of an object or target item.

It could be a spy balloon making its way across North America or an endangered species in the jungle. Synthetaic asks the AI to look for a rough form or shape in a set of images or videos, which RAIC can quickly identify, saving human workers days and weeks of tedious work labeling each image with identifying information and details AI needed to find its target.

All of that means RAIC is more accessible than other AI tools that might require a person to have a computer science or engineering degree to work with.

Should we be concerned over this sort of AI?

AI programs like ChatGPT have increased public interest in, and engagement with, artificial intelligence. But Jaskolski pointed out AI has been put to use in daily life for years in ways that bear little resemblance to popular culture portrayals.

Rather than aiming for world domination, AI does things like identify and track credit card fraud, Jaskolski explained.

In Synthetaic's case, RAIC requires human input to do anything.

"People still think AI can go run amok and do all this stuff by itself; it really can't," Jaskolski said. "Right now, with a tool like RAIC, it's still very much a human-machine collaboration. Without a single human to go guide the AI, to help it understand what you're looking for, we would have never found the balloon."

How did it find the spy balloon?

Jaskolski was at home in Delafield self-isolating from a close COVID-19 contact in early February when he wondered if RAIC could find the balloon in satellite data. He drew three, crude circles of what the balloon would look like from overhead and asked RAIC to comb through Planet Labs' database of daily satellite images taken of every inch of Earth.

"I thought it was really a long shot, but we uploaded the data for all of South Carolina and within two minutes found the balloon," Jaskolski said.

Synthetaic staffers sorted through the images RAIC produced and refined its search. They directed it to ignore things like churches that had a similar shape and also fed it images of other balloons found in Planet Labs' images so it could better know what to look for.

The result was data that identified the balloon's launch point and different points when it changed altitude, for example to ride wind current, or how it changed direction in ways that suggested someone was controlling the balloon's flight path.

What else can RAIC do?

The applications of RAIC could be vast now that Jaskolski said it has proven it can take "a beautiful haystack of information that's hard to search through" like satellite images or traffic videos, and make it much easier to find what you're looking for.

"We can now literally search for anything whether it's balloons, whether its aircraft, whether its ships, whether its greenhouse gas emission sites like oil and gas wells or concentrated animal feed lots," Jaskolski said. "We're working on a number of those across commercial, (non-governmental organization) and government applications."

Jaskoski, an engineer, drew upon his own experiences creating high-tech cameras and 3-D imaging systems to help conservation and exploration groups when he founded Synthetaic. At the time, Jaskolski, a National Geographic Fellow, had just captured a 3-D image of an endangered Sumatran rhino with National Geographic camera crews watching.

Could it help Ukraine repel Russia's invasion?

Jaskolski said RAIC would be particularly effective in situations like Russia's invasion of Ukraine, and the fighting that continues there today.

He said the program can help in a variety of applications where time is of the essence because it eliminates so much of the current work needed to prepare data for AI to interpret.

"RAIC allows you to literally find one example or take a hand-drawn example of something you're looking for and start searching for that immediately over areas the size of an entire country," he said. "In a conflict or in a disaster, or anything that's really high time sensitivity, RAIC can play a role in a way we don't think anyone else can right now."

How did Green Bay's TitletownTech help Synthetaic?

Well before spy balloons brought it wider notoriety, Synthetaic caught TitletownTech's attention.

The Green Bay area venture capital firm and startup nursery invested in Synthetaic in October 2020 and again in a March 2022 funding round that raised $13 million, bringing the company's total raised to $17.5 million.

"We thought we'd be on the east coast and the west coast raising all the capital to fund this company," Jaskolski said. "However, when we found TitletownTech, a venture firm right here in Wisconsin, we were very pleased to be able to work with them."

Now, Synthetaic has about 45 employees, concentrated out of its Delafield offices, though some work remotely. Jaskolski said TitletownTech not only has invested, but helps the company work through business opportunities and helping it grow from a start-up.

Wisconsin startups 'can build here, get funding here'

Jaskolski said his story, and the company's development, show Wisconsin technology startups can find the financing and support they nee d.

For Green Bay-based TitletownTech, the venture studio founded in 2017 by the Green Bay Packers and Microsoft, Synthetaic was the sort of innovative, high-growth-potential companies it looks to invest in. TitletownTech specifically focuses on start-ups in media and entertainment, digital health, agriculture and environment, advanced manufacturing and supply chain technology industries.

Synthetaic's notoriety is a chance to remind startup and early-stage entrepreneurs that they do not need to head to Silicon Valley for funding and support, said Craig Dickman, TitletownTech's managing director.

"It shows they can build here, get funding here, grow here," Dickman said.

Contact Jeff Bollier at (920) 431-8387 or jbollier@gannett.com. Follow him on Twitter at@JeffBollier.

Thanks to our subscribers for making this coverage possible. Your support helps local journalism.

This article originally appeared on Green Bay Press-Gazette: Here's how a Wisconsin company used AI to track Chinese spy balloon