“It’s not possible for humans to learn everything,” says Diffbot founder Mike Tung. “So we have to design a system that can.”
Diffbot scours the Web and serves up its findings in customized bites for companies. For Cisco, it scrapes articles for mentions of conference call participants. For a sneaker company, it culls consumer reviews and discussion threads. For a business software firm, it finds prospective clients. Gathering this kind of data usually takes time or can be incomplete. Diffbot claims it scrapes nearly all of the public Web—and can produce search results in less than a second.