web2express.org

November 4, 2009

New feature: monitoring social media in real time

Filed under: news, social media, web intelligence — aj @ 1:41 am

I have just added a new feature to web2express digest application: monitoring twitter conversations and blogoshpere in real time. Real time monitoring is the second application of the real time data platform I have established since the beginning of this year. The first application is topics following for the public.

Who will benefit from social media monitoring?

Marketers can use the new “My Probes” feature to conveniently monitor new twitter conversions and new blogs as they come online. You can create any probe to monitoring what people are saying about your products, brands, or competitions right now.

How to monitor in real time?

To set up the monitoring, check “Enable realtime monitoring” on the account settings page, and then create a probe for each monitoring task on the “My Probes Settings” page. A probe is a set of keywords and/or phrases. After you create a probe, you can add or change keywords anytime. Monitoring can be started or stopped at any time. After monitoring starts, the real time data stream is probed by the probe’s keywords and phrases. You can view the found target conversations and blogs on the view probe page. Currently, only the latest targets are shown online. I’ll figure out how to effectively expose all of your monitoring results online. In the meantime, let me know if you want to see the archived results in your account.

Any comment or feature request? please drop me an email (ajchen @ web2express.org).

-aj

September 20, 2009

Web2express Digest Now Analyzing Millions of Tweets/Blogs in Realtime

Filed under: news, social media, twitter, web intelligence — aj @ 5:06 pm

A huge data stream of blogs has been added to the web2express digest server for a couple of weeks now, but the front end http://web2express.org was finally connected to this important addition of real time data yesterday.

For more than half a year the system has consumed only twitter streaming data, which is about 1000 tweets per minute.  New topics are auto-discovered from these twitter conversations every minute as popular topics emerge.

The blogsphere is another rich source of fresh conversations. It’s much bigger than twitter stream in bytes. Thanks to Kevin Burton, CEO of Spinn3r, I am permitted to pull in Spinn3r’s blog data stream to my real time analysis system.  We met on the event that I organized for twitter applications 2 months ago. Kevin was so generous to suggest me to try their data service. Spinn3r crawls the whole web for blogs and then make the data available through API. It’s not exactly in real time, but very close. Roughly one million blogs per 24-h period (after filtering) is now flowing from spinn3r to web2xpress.

Combining both twitter conversation data and spinn3r blog data, web2express digest server now analyzes 2-3 million fresh conversations and blogs on a continuous 24-hour basis. The primary analysis is still auto-discovery of new topics in real time conversations. Immediately after new topics are auto-discovered, they are provided to the frond-end web page, allowing users to follow realtime twitter conversations and blogs by topic.

It’s exciting to see such a huge amount of fresh web contents being analyzed in real time! I’m pretty sure this real time system can have many different applications or usages. For example, advertisers and marketers can use it to monitor products and brands on social media, do sentiment analysis, and identify potential targets. I’m interested in hearing feedback from marketers and finding ways to make this real time tool useful for social media marketing.

August 5, 2009

New Features: Follow and Tweet Topics

Filed under: news, open data feed, twitter, web intelligence — aj @ 4:32 am

After playing around with auto-discovery of topics in twitter conversations for a while, it seems to me that following topics is another effective way to communicate on twitter. So, I’ve added a new set of features on http://web2express.org website to make it easy for people to follow and tweet about topics. The auto-discovered topics give you a fairly good starting point to read about current new hot topics. In addition, one can create any topic to follow. By adding of a set of keywords or phrases to the topic, you will find that topic following pulls out more complete list of conversations from twitter with much less work. When you tweet or retweet about a topic, the topic’s hashtag is automatically appended to your message so that you don’t need to remember what hashtag to use.

I notice several websites including twitter.com homepage have started to provide similar functions recently. Anyone has any early experience or comment to share?

-aj

June 22, 2009

New feature: RSS feeds for twitter digest

Filed under: news, web intelligence — aj @ 9:50 am

Just added RSS feeds on the web2express digest website. There different feeds for the different views of hot topics:

  • Feed for today’s new topics.
  • Feed for today’s top topics.
  • Feed for daily new topics in the last 3 days.
  • Feed for daily new topics in the last 7 days (a week).

They are available for anyone to subscribe and use on their websites.

June 19, 2009

case study: web2x digest picked up news event early

Filed under: news, web intelligence — aj @ 12:14 am

I notice a perfect case study from today’s news:  “Continental airlines incident” was emerging at about 8:29am (PST) on the twitter daily new topic list on this http://web2express.org website. it was probably at the same time as other major news outlets broke the news, maybe even a litter bit earlier.  Surprisingly, this top news did not show up on Twitter.com’s trending topics list, nor on google trends at all for the whole day.

This case study shows the difference between various trending applications. I think the underlying technology is the key. Calais is the semantic analysis core of my real time trending system. It seems Calais does pretty good job as it promises.

more digging:

I just found the flight schedule from NYT.com report:  The flight, Flight 61, took off at 9:54 a.m. in Brussels (3:54 a.m. Eastern time), according to Continental’s Web site. It touched down at 11:47 a.m., earlier than its scheduled noon landing, at Gate C123 at Newark Liberty International Airport.

Something is amazing if you compare the time carefully. The incident showed up as new hot twitter topics at 8:29am PST while the flight was still in the air.  where did the tweets come from so early? did someone tweet in the airplane? or people on the ground in Europe got the information early and tweet? Anyway, news travels on twitter fast, very fast!

Older Posts »

Powered by WordPress