Niche search engines pick needles out of Web haystacks | Science| In-depth reporting on science and technology | DW | 22.09.2010
  1. Inhalt
  2. Navigation
  3. Weitere Inhalte
  4. Metanavigation
  5. Suche
  6. Choose from 30 Languages


Niche search engines pick needles out of Web haystacks

Several new search engines focus on segments of the Web, like social networking or microblogging sites. They are not looking to challenge Google, but they are letting users search places that Google can't reach.

Needle with a red thread running through it

Search engines are barred from indexing some sites

For Khash Sajadi, the search began with a bagel. While in New York on a business trip, he remembered that a friend had sent him a Facebook tip on where to buy the round boiled-and-baked bread.

Sajadi took out his mobile, logged into Facebook and scrolled through pages and pages of old messages. Later, when he googled "bagel shop in New York" and got 166,000 results.

"It was returned in 0.29 seconds, but it's not the [result] that I want." Sajadi said. "I would rather it takes five seconds but gives me what I want."

Upon returning to London, he and a partner began putting together Sentimnt, a search-engine prototype that he hoped would provide a result where both Facebook and Google had failed.

The stuff Google cannot see

Facebook and Twitter logos

Social networks sometimes see search engines as competitors

Sentimnt is one of several new niche search engines, which also includes Slense in Austria and Twingly in Sweden, that are not looking to beat Google. Rather, they focus on specific segments of the Web, searching social bookmarking sites like or social networking services.

Some of these popular sites are places that the dominant search engines cannot yet access - like Facebook or Twitter or Foursquare. Even as more people are using the social media services, there is not one comprehensive way to search through the information shared on all these networks.

"There is so much funny conversation going on in Facebook," said Jonathan Allen, an analyst at the industry newsletter, Search Engine Watch. "But unlike Gmail, where I can scroll back five years and reread e-mails I sent to friends recounting adventures - you just can't do that on Facebook."

A personal slice of the Web

Slense screenshot

Slense indexes the sites users say are important to them

Among the niche engines being built to plug these holes in the search space is Slense, a small startup based in Vienna.

Four doctoral students created the search engine, which indexes only the websites users' indicate they are interested in via bookmarks or RSS feeds. The search sources can also be shared so that users who like someone else's bookmarks can add them to their source list. In the founders' words, users can create their own personal "slice of the Web."

"We don't need to be all that simple to use as Google," said co-founder Christian Zeidler. "But we will be able to provide those expert searchers with more interesting results and an easier way to organize their stuff on the web."

Slense is still in its early stages - the public beta was launched last month - but its developers seem convinced that it will remain a search engine used by the tech savvy rather than the mainstream user.

That may be a sound strategy, according to Sven Achter, a technology investor at Holtzbrinck Ventures. Because the search giants like Google and Bing have large amounts of capital and thousands of engineers at their disposal, startups may better off catering to a niche clientele than going head-to-head with the incumbents, he said.

New sources, new uses

Sentimnt screenshot

Sentimnt lets users' know what their social circles are up to

The founders of the UK-based Sentimnt, however, hope their social search engine will find a place within the mainstream. About 10,000 people have signed up since the private beta was launched last month. Users choose from a list of sources - including their Gmail, Facebook and Twitter accounts - to include in their searchs.

At first, Sajadi had assumed people would use Sentimnt once every few weeks only to find those items - like his friend's bagel tip - that they remembered seeing somewhere. But he found that people were using the engine daily for reasons he had not imagined, like buzz tracking. These users would head to Sentimnt everyday to search for the same query word simply to see what their friends were saying across various social networks.

"I realized it's not about what's the trend at the moment on Twitter," said Sajadi. "It's about what's the trend within my own social circle."

When personalization becomes creepy

The need for a personalized search-engine may now seem obvious to developers like Sajadi. But why hasn't a search giant like Google improved and expanded its search capabilities to stamp out these upstarts?

Of course, many social-networking sites, such as Facebook, see Google as a competitor and do not want to give it free access to their data. But Joi Ito, a well-known IT investor and entrepreneur, has another theory: when a search engine knows too much about what you're looking for, it can start become creepy.

"The trick is going to be how do we create [a personalized search-engine] in a way that doesn't exploit the user and actually empowers the user," he said.

That is, personalized search-engine may have to make its results - and its algorithm - much more transparent to users.

The dinghy versus the oil tanker

But if Google did find a way to sidestep the privacy issues and expand its services, would users still reach out to add-on search engines like Slense and Sentimnt?

Right now, Sajadi is optimistic that users would. He calls Google's social graph “fragmented”, and says the different services it provides do not work together seamlessly. But when push comes to shove, he is not sure what strategy he would pursue with Sentimnt.

"When an oil tanker as big as Google is coming towards our dingy," Sajadi said, "I wouldn't consider staying in its way an option."

Author: Sruthi Pinnameneni

Editor: Cyrus Farivar

DW recommends

Audios and videos on the topic