In the past several days, web2express digest site did not receive data from twitter api. This was because twitter was still trying to fix the problem with its timeline feed for data mining. Earlier today, twitter api team announced the retirement of the data mining timeline feed. But, good news is that twiiter has something better for developers: the streaming api.
I implemented streaming api for web2express digest web site yesterday. It has been running smoothly. The latest version of Twitter4j library released a couple of days supports streaming api. I just plug it in and it works beautifully.
New feature: a new feature is also added to allow visitors to add twitter user as friend while browsing the hot topics on the digest web site.Â I think this feature will make the free web site more useful to twitter users.
I got a comment earlier today on twitter digest topics: Though it hurts me in the opinion-of-humanity part of my brain to learn how heavily represented American Idol is on that list.
Yes, the daily hot topics may surprise many people. I could not believe what I saw when the system went online for the first time a few months ago. If you are used to reading tech news or WSJ, you may get a shock. The daily conversations on twitter, and probably other social networking sites, are mostly about TV shows, movies, games, and other entertainment stuff. But, on the other hand, this also makes sense. People are talking about their lives on social networking sites, and life is not all about technology and stock market, at least for most ordinary people.
Web2express Digest does not cut or selection of topics. It just shows whatever comes out of the ongoing conversations from millions of people. I think we can learn a lot from this information in addition to becoming more effective in navigating through the twitter sphere.
After several months of development, I’m happy to announce twitter digest web site is now available for the public. Web2express.org web site’s default home page is switched to this “Digest” web app.
Thanks to twitter’s api and the api team, the data feed for data mining is just wonderful. Web2express Digest is a real time system that takes in the feeds as they become available, and does some NLP analysis on the tweets using open tools like Open Calais and openNLP.Â The results are freely available on http://web2express.org/. Â Using this twiiter web app, you can spot Â daily hot topics and for each hot topic, quickly find the top contributing twitter users. I hope this real time information will
help users to understand the popular topics at any given moment and easily identify who to follow.
I have run the system internally since the beginning of this year. Tweets from millions of users have been analyzed. Each day, large amounts of public tweets pass through the system, from which hot topics are identified by semantic analysis and ranked by popularity. By comparing hot topics in current 24-hour period to the previous 24-hours, new topics are selected.
Please let me know if you have any comment.