This is a question I frequently get asked by new clients. They know there is a Twitter API available to collect tweets, but they have no idea how the results differ from just asking for tweets with Search.Twitter.com. I’ve recently explained the fact that a tweet database lets you create a long-term store that cannot […]
Database Cache
Tweet aggregation lets you build a faster and more reliable app
A database cache is the foundation of any serious Twitter application. Learn how to use PHP and MySQL to collect tweets from the Twitter API, and serve them to the rest of your code.
Most of the Twitter consulting I do involves some form of tweet collection and storage in a database. Even when clients approach me with this in mind, they hardly ever realize just how valuable tweet data can be. In fact, it is priceless in the truest sense of the word, because there is no way […]
There seems to be a good amount of interest in the new set of tutorials I’ve started writing, and most of the code I produce interacts with a MySQL database, so I’m going to post the code for my standard database library here. This makes it easy for me to link to this post multiple […]
This week we got crushed by the State of the Union speech. We normally get about 30,000 to 50,000 tweets per day in the 2012twit.com database, and our largest server can handle that without any showing any appreciable load. During the SOTU tweet volume exploded. We got 500,000 tweets in about 4 hours. I was […]
I got a call the other day from a developer who was receiving various 500 series errors when trying to gather large amounts of Twitter user data. The API has a number of errors in the 500 range, all of which generally mean that the Twitter servers are overloaded. The API is built on the […]
The first day has gone well. I announced the code on the Twitter dev list, and got 16 visitors to the site. The good thing is that the average pages per visitor was 7, and they spent an average of 11 minutes on the site. So people who get to the code are giving it […]
I just opened up access to version 0.10 of my free Twitter source code library. At first it just has modules for a tweet aggregation database and a tweet display plugin. My next module will be a WordPress display plugin. I also have a lot of tutorials planned that can use this source code as […]
I rebuilt the code for the 104dev system using a WP-style architecture. As expected the messiest part was getting the directory paths right. I think the idea of putting most of the functionality into separate plugin directories will be a big win. There will be a very small core of tweet collection code and general […]
I’ve been thinking about how to prepare the tweet aggregation code used for the dev tweets page. I know that I want to add multiple modules to the system, and make it easy to customize the appearance of the tweet list. This sounds a lot like the WordPress model with plugins and themes, so I’m […]
I just finished a tutorial on the two methods of searching for tweets. Whenever this subject comes up on the Twitter developers mailing list, the usual response is that the streaming API is best, but that depends on your goals and programming ability. If you want to search for tweets in the past, or if […]
Along with providing the source code for the Twitter engineering tweet database, I also want to use it as a model for explaining the proper database and code architecture for aggregating tweets. I’ve got my first tutorial up now on the benefits of decoupling your website Twitter app from the Twitter API.
