Machine learning research can detect fake news domains upon registration

Creating fake news and using social media to amplify it has become a ubiquitous feature of election campaigns, not only in the United States but around the world. And it continues to proliferate in spite of efforts by governments and social media companies to stem the tide and defend against it.

That’s what makes new research by Johnson faculty member Bill Schmidt and his colleagues exciting: Using machine-learning technology, they are able to identify purveyors of false information right when they register their website domain, before they even begin to post and broadcast fake news across multiple channels. (“Real-Time Prediction of Online False Information Purveyors and their Characteristics,” a working paper co-authored by Anil R. Doshi, Sharat Raghavan, and William Schmidt, October 2020.)

Early detection system predicts fake news

“False information, disinformation, fake news, misinformation—all of those things can have a dark side to them: to sow confusion, evoke frustration, increase polarization,” said Schmidt, assistant professor of operations, technology, and information management at Johnson, speaking in an Oct. 30 webinar, Decision 2020: What Elections Can Mean for Business.

“A key aspect of the producers of fake news is that they are very nimble,” he said. “So to counter the proliferation of fake news you have to move fast and move early.” That’s why Schmidt and his fellow researchers set out to identify an early detection system to highlight domains that would produce and promote fake news early on, which they achieved by applying a machine-learning model to domain registration data.

“Ultimately we found that a machine-learning model based solely on domain registration is hugely predictive of which domains will become providers of false information,” said Schmidt. “In our analysis, we correctly identify 92 percent of the false information domains and correctly identify 96.2 percent of the non-false information domains. And we do this before they start operations, at the time they register their domain.”

Opaque contact details in registration information—such as a holding company, corporate entity, or LLC instead of an individual’s name—is one tip-off that helps to pinpoint bad actors, Schmidt said.  “We think this actually will go in our favor,” he said. “We don’t believe that bad actors will be able to guard against this in the future, because those who really want to sow confusion and strife with misinformation will always desire to not be identified after the fact.”

No drop-off in false information post-election

False information is not going to stop after the election; it is going to persist, Schmidt warns. “False information is a great vehicle for stirring doubt and uncertainty and elevating tensions,” he said. “So this isn’t just about the election outcome. It is a fight over the aftermath of it. I have reasonable confidence that our election officials and government agencies are preparing for these types of contingencies. I do think there are going to be hot spots. There are going to be issues that make us all feel nervous. But I’m hopeful that those issues and problems and flare ups are containable and localized.”

Companies are the next target for false information

“Leaders within companies need to recognize that these tools are going to be redeployed, and in many ways are already being redeployed, to target companies,” said Schmidt. False information in the political arena paved the way for further proliferation, he said. Because so many people watch big election cycles and because election outcomes are seen to indicate their effectiveness, they function as training grounds for developing highly visible processes, methods, and technology. “The 2016 election was like a big advertising campaign that this seems to work,” Schmidt said.

Facebook has already indicated that they anticipate these kinds of methods and tools to move from political objectives to targeting businesses, said Schmidt. He cited a 2018 false information campaign that targeted Nike and a 2020 campaign that Facebook alleged a major Southeast Asia telecom firm engaged that targeted its competitors.

“The time is fraught for corporations, as well,” Schmidt said. “And it’s incumbent on them to have a response plan in place and be able to quickly identify when they themselves are being targeted by a campaign.”

A fine balance: false information vs. free speech

Once identified, what is the best way to handle websites created to disseminate misinformation? On the one hand, “the critical thing is to move early, before false information gets seeded into our digital information environment,” Schmidt said, because “once it’s out there, it’s hard to put it back into the bottle.” On the other hand, “many societies are more willing to err on the side of free speech and avoid the possibility of engaging in censorship.”

Schmidt and his co-authors propose that their tool, and tools like theirs, be used to help regulators, platforms, and policy makers proceed with an escalated process. When you pinpoint false information sites, you can decide whether to “increase monitoring, send warnings, censor those domains, sanction them, and ultimately, whether you want to shut them down,” Schmidt said. “Our hope is if you know what domains you should be looking hard at very early on, you can much more readily monitor the initial bits of information that start coming out of those domains. I’m not an advocate for pulling the plug too early, but I am an advocate for making sure you know what you’re looking at.”

Social media companies need to address the issues created due to their platforms

“I don’t think there’s a bright line between what deserves to be censored and what doesn’t,” Schmidt said. “It’s gray, and where that grayness stops and starts is different for different people and different societies.

“I recognize that social media companies have an incredibly hard job to do,” he noted, “because they are operating in different communities and cultures, where that line differs, and they have the challenge of trying to balance across all of those. “That said,” Schmidt continued, “they’re incredibly profitable. And they have devoted a very small amount of that profit towards solving the problem that has been created due to their platform.

“It’s incumbent on social media companies to take a stronger hand in thinking about and addressing the types of issues that have been unleashed from their Pandora’s boxes.”

Schmidt hopes to see these companies called out and placed in uncomfortable positions so that they start thinking very hard about their role as companies, outside of generating shareholder value. He hopes they will seriously grapple with “what they should be doing to help stabilize the society that we all live in.”

Changing tactics require constant innovation and research

While Schmidt and his co-authors have developed a useful tool, he emphasizes that it is not a panacea. “This is a multiple-period game with changing tactics,” Schmidt said. “As soon as a tool like ours gets put out there, the bad guys will respond to it. That underscores the need for constant and ongoing innovation and research around this.”

View a recording of Decision 2020: What Elections Can Mean for Business to hear Schmidt discuss his research. The webinar also features Jawad Addoum, assistant professor and Robert R. Dyson Sesquicentennial Faculty Fellow in Finance at the Dyson School speaking about how the political climate can influence stock prices; and Simone Tang, assistant professor of organizational behavior in the School of Hotel Administration, who discusses her research on how difficult decisions affect concepts of fate. The webinar is moderated by Theomary Karamanis, senior lecturer of management communications at Johnson.