Collecta Blog – Real-time Streaming Search

Did you notice something new on Collecta today? What you are seeing is our new real-time display of the latest hot trends from around the web – we call it Hot Now.

If you’ve used Collecta before you may have noticed the Hot Now topics displaying as an updating list in the Collecta client. But we decided to make things a little more fun and show sample results for each of the current trends in Hot Now.

We include samples from each of our content categories for images, stories, updates and comments from across the web. The sample items for each topic will change from time to time as new relevant information comes in – giving you the fastest, most up-to-date view of the topics in real time.

So now Collecta is not just the place for real time search, it’s the place to track the pulse of the web. Check back from time to time to see what’s Hot Now, and discover breaking news and events – in real time.

Last week we posted a very specific case where the residents of Charleston SC used Collecta to work their way around town during a big storm. This week we are going to take a step back and give you a broad view of ways real-time search can be used. Watch below as CTO Jack Moffitt explains the what and why of real-time search in five minutes.

CTO Jack Moffitt with “New Things Now: Intro to Real-time Search”

Want to share how you use Collecta? Email us at help at collecta dot com.

Read Write Web has invited people to vote for their favorite Web Products of 2009. We are excited to be on the list and are asking all you to help make us #1.

Here is the link to the voting.

Thanks!
Team Collecta

We’re never satisfied with just being the fastest. We always look for ways to make Collecta even faster. With that in mind we set out looking for ways to not only scale but to get faster as we did. Our quest led us to Basho’s Riak Search, a new kind of search indexing system.

CTO Jack Moffitt explains, “Indexing at Collecta is there to provide historical context; it is only part of our search technology. Our real-time technology is powered by streaming (using XMPP, not indices). As far as I’m aware, we are the only real-time company that is not based on indexing systems, and that fact probably explains why our search results appear in fractions of a second after, not minutes after, content is published.”

He continues, “Our users would not be impressed if the results page showed them nothing until the next time someone mentioned their query term, so we store a historical archive and index it for queries. Unfortunately, the traditional indexing systems fall well short of our needs.”

We are excited to see what this new opportunity brings not only us but all developers and administrators.

Read Jack’s entire in-depth blog post or Basho’s press release.

One of the questions we get here at Collecta from those who are new to real-time search is “What would I use it for?” A fair question, for those used to static web search, the importance of real-time information may not always be clear – until you need it. Which is why we have decided to present some case studies from real Collecta users. We could easily tell you all the ways you could use Collecta – to track your favorite sports team during the game, follow breaking news or get the latest scoop on your favorite celebrities, stay ahead of the curve in business by tracking the market buzz – but we feel like real life examples speak most clearly to the importance of real time, and how the real-time web combined with social media provides information that you can’t always get from traditional media sources.

On Dec 2nd, 2009 the city of Charleston SC had some severe weather causing road closures, flooding and damages. If you’ve ever visited Charleston, you know that the main city is on a peninsula, very close to sea level, so flooding potentially shuts down large parts of the city. Quite the bummer if you have somewhere to go. Suddenly knowing which streets are flooded and which streets are passable – right now – takes on a whole new importance.


Photo found via Collecta from The Post and Courier – Tony Bartelme/Staff

The Digitel Charleston started asking people to use a specific hash tag to post their information to Twitter, and to follow it on Collecta.com and used this to feed important real time information about a sever storm to its citizens helping them keep track of the storm, and to help users re-route around the most flooded areas when travel was a must.

By aggregating information from Twitter, local weather blogs, and traditional media sources from the rest of the web all in one place, Collecta provided a one-stop place to get the information Charleston residents needed about the storm, in real time. As a further example, below Christopher Zorn, a Collecta developer who lives in Charleston, used this real time feed from Collecta to guide his family through the storm.

Have you used Collecta to make your life easier, help you have more fun, or follow a breaking event that matters to you? Let us know by email or @collectadotcom on Twitter.

Six weeks ago we began our second API challenge. This time we challenged developers to build a creative, user-facing application on our XMPP API. There were some great entries into the contest and in the end it came down to two finalists:

David Bello’s CollectaAlerts lets you follow and receive realtime updates directly to your desktop and mobile phone.
Alvaro Videla’s Dashboard Widget for Mac OS X that allows you to search for terms using the Collecta XMPP API.
**Please follow the links and try them for your self.

But just like in the Highlander, there can be only one. Collecta is proud to announce the winner of its XMPP Real-Time Challenge…

Alvaro Videla! Your $2,000 Amazon gift certificate will be on it’s way shortly.

We thank each and every developer that took the time to use the XMPP API and submit to our contest, and we hope to see continuing use of the XMPP API in the future. Stay tuned for more fun announcements and
challenges from Collecta.

Today Google announced their real-time search feature, which loads new results in real time rather than the usual static page familiar to so many Google users. It’s nice to see Google validating what we have known all along – the idea that is at the heart of Collecta – real time search matters, and is here to stay.

Opinions are coming fast and furious on Google’s entry into real time search, but one thing is clear, the space is more exciting today than it was yesterday. Marshall Kirkpatrick of RWW says of Google’s new entry into real time that, “this looks a lot like what Collecta offers” and we agree, but Collecta is different.

Collecta’s design, infrastructure and spirit are entirely focused on real time. We are not just taking an old model and adapting it. That’s why we are confident that Collecta will continue to lead the way in real time search in the months ahead. We are happy to welcome Google to the real time party.