Adam Green
Twitter API Consultant
adam@140dev.com
781-879-2960
@140dev
Twitter API Programming Tips, Tutorials, Source Code Libraries and Consulting
The Twitter streaming API will deliver tweets for any keywords in real-time. Learn how to cache the results in a database and display them online.
Yesterday’s streaming API post described a multiple server model for handling high rate tweet collection. Today I’d like to cover a different architecture that addresses this problem with a single server running multiple databases. Let’s say you want to display tweets for the most active stocks each day. The streaming API lets you collect tweets […]
{ 0 comments }
Now that I’ve upgraded the streaming API framework to make it easier to manage keyword tweet collection, the next step is handling the increased data flow that results from more keywords. One simple solution is to upgrade your server. MySQL loves as much RAM as it can be given, and switching to a solid state […]
{ 0 comments }
My latest set of enhancements to the streaming API framework is moving along nicely towards my goal of making this code a true production level tweet collection system. While I’m waiting for feedback on the new code, I wanted to take a minute to think about where this system can go. I see several possible […]
{ 0 comments }
I wanted to fit one more enhancement into this new version of the streaming API framework. The major limit on performance of a tweet collection system is the number of rows in each table. This is most important during the parsing phase, when lots of insertions are done. MySQL slows down dramatically when it has […]
{ 1 comment }
This set of changes is a significant improvement to parse_tweets.php. To allow you to use this separately from the production version, I’m calling it parse_tweets_keyword.php. You can place it in the same /db directory as the rest of the streaming API framework code. The only changes you will have to make is adding the following […]
{ 0 comments }
Yesterday I introduced the database changes needed to control tweet collection with a table of keywords. Now we can modify get_tweets.php to collect tweets based on the contents of the collection_words table. To avoid confusion with the production version of get_tweets.php, I’m calling this new version get_tweets_keyword.php. This will let you place it in the […]
{ 1 comment }
The previous post had an overview of my planned enhancements to the streaming API framework. The first step is adding some new tables to the 140dev tweet collection database: collection_words The words in this table will be used to collect matching tweets. We’ll see how to add this to the get_tweets.php script so that the […]
{ 0 comments }
The next few posts will describe a set of enhancements to the streaming API framework that will greatly expand the capabilities of the code for collecting tweets based on keywords. I thought I’d start with an overview of what I want to accomplish: Add a collection_keywords table to the database to hold keywords to be […]
{ 0 comments }
I’d like to start working on a book on the streaming API, but I’ve learned that actually walking people through code and installs is best before writing it all down. I’m learning a lot about where the sticking points are by helping people install the new version of the framework. It seems like running processes […]
{ 0 comments }
I’m a typical programmer when it comes to reading documentation before I try to run someone else’s code. First I try to run the code, then I read the docs. Hey, if I needed clear documentation to get things working, would I have become a Twitter API programmer? Unfortunately, Twitter’s streaming API requires a unique […]
{ 0 comments }
The streaming API is pretty amazing when you start seeing all that free data pouring into a database, but sadly the connection is not completely stable. I typically see a failure at least every couple of months. It’s hard to be sure what causes the failure. It could be the Phirehose library that talks to […]
{ 2 comments }
I’ve been spending the last few days helping people install the latest version of the streaming API framework. This has reminded me of how many moving parts there are, and how this can get in the way of building a mental model of what is actually going on. One of the biggest confusions seems to […]
{ 0 comments }
Make a smooth transition to API 1.1
with Adam Green’s first book on
Twitter API programming
Adam Green has been building custom Twitter applications for 5 years. Thousands of people have used his open source libraries, ebooks, and tutorials to build their own websites and online tools based on the Twitter API.
Now Green has harnessed his years of coding experience to present the secret formula for successful Twitter Engagement programming.
Collect. Identify. Engage.
Collect a complete engagement profile for the most influential accounts for any area of interest. These tweets, mentions, friends, and followers will be stored in a MySQL database on your own server.
Identify the users you need to interact with, including the most mentioned tweets and tags. The reporting system included as source code can easily be enhanced to select exactly the type of users you need to help spread your message.
Engage with these key accounts using automated tweets and DMs. Follow new accounts that have been selected based on your specific needs, and track these accounts to know when they follow back.
All of the PHP source code for this book is available as open source under the MIT license.
Free Javascript Ebook
Free Twitter OAuth Ebook
“When I started investigating on how to build my Twitter based Site the Twitter API made my head spin. Sure it was powerful but where do I even begin? Then I found the 140dev Streaming API Framework and it made Twitter development a LOT easier! Without it I would probably spend hundreds of hours just on standard functions such as connecting to the Twitter API, managing data and displaying results. Thanks 140DEV for such a cool tool!"
- Paul Valkama, http://StreamClock.com
“The most important part of any technology effort is the nature of the people you work with. The people at 140dev are world class best. They have years of experience, a deep knowledge of Internet technologies, extensive familiarity of the Twitter API and a proven ability to deliver. They also work great with others. Frankly they have been so incredible I am amazed and really hope we can continue the relationship for a decade."
- Bob Gourley, http://twitchimp.com
“We worked with Adam and the 140dev team to build a Twitter business focusing on healthcare companies. 140dev brought unparalleled knowledge of Twitter mechanics, but also an excellent understanding of how Twitter fits into business strategy. They proved to be ideal development partners: committed to reaching our project’s goals, generous with sharing their knowledge, and dedicated to delivering beyond our expectations, despite working across continents and time zones. We now see them as an extended part of our business and hope to work with them in many future projects."