Twitter has announced that they will be turning off basic authentication for the Streaming API on May 7th. I’m preparing a new version of the 140dev Framework to use OAuth now, and will have it posted tomorrow. That will be good for new users of this code, but if you already have the code in production, I assume you have made changes to it. Here is a detailed set of instructions for modifying an existing installation to use OAuth. The good thing is that you only have to use single-user OAuth, which is pretty easy to implement. These notes should give you everything you need.

Obvious, but very IMPORTANT note: Make a complete backup of any current version of this code you now have in production before making these changes.

There are 5 steps you have to follow to convert an existing installation of this code to use OAuth with the streaming API version 1.1.

1. Create a Twitter application at http://dev.twitter.com, and get a complete set of tokens. If you haven’t created an app before, you will find detailed instruction in my free ebook.

You need 4 tokens as defined on the details page of the application settings:
Consumer Key
Consumer Secret
Access Token
Access Token Secret

It is traditional to rename these tokens in every library that uses them. This has happened again with the Phirehose OAuth library. Here are the new names:
Consumer Key = TWITTER_CONSUMER_KEY
Consumer Secret = TWITTER_CONSUMER_SECRET
Access Token = OAUTH_TOKEN
Access Token Secret = OAUTH_SECRET

2. Modify the 140dev_config.php file in the 140dev Framework code. This is found in /db/140dev_config.php.

You need to replace these lines of code:
// Basic authorization settings for connecting to the Twitter streaming API
// Fill in the values for a valid Twitter account
define(‘STREAM_ACCOUNT’, ‘*****’);
define(‘STREAM_PASSWORD’, ‘*****’);

With these new lines. Insert the actual tokens you received from Twitter in step 1:
// OAuth settings for connecting to the Twitter streaming API
// Fill in the values for a valid Twitter app
define(‘TWITTER_CONSUMER_KEY’,'******’);
define(‘TWITTER_CONSUMER_SECRET’,'******’);
define(‘OAUTH_TOKEN’,'******’);
define(‘OAUTH_SECRET’,'******’);

3. Get new copies of the Phirehose library code. You need to delete Phirehose.php in your current installation. It will be found in the /libraries/phirehose/ directory. Then download these files and place them in the /libraries/phirehose/ directory:
Phirehose.php
OauthPhirehose.php

4. The latest version of Phirehose.php still points to the 1.0 version of the streaming API. To point to version 1.1, you need to change line 20 of Phirehose.php from this:
const URL_BASE = ‘https://stream.twitter.com/1/statuses/’;

To this:
const URL_BASE = ‘https://stream.twitter.com/1.1/statuses/’;

5. The final step is modifying your copy of get_tweets.php, which is found in /db/get_tweets.php. There are two code changes:

First replace these lines starting at line 13:
// Extend the Phirehose class to capture tweets in the json_cache MySQL table
require_once(CODE_DIR . ‘libraries/phirehose/phirehose.php’);
class Consumer extends Phirehose

With these new lines:
require_once(CODE_DIR . ‘libraries/phirehose/Phirehose.php’);
require_once(CODE_DIR . ‘libraries/phirehose/OauthPhirehose.php’);
class Consumer extends OauthPhirehose

Then replace this line:
$stream = new Consumer(STREAM_ACCOUNT, STREAM_PASSWORD, Phirehose::METHOD_FILTER);

With this one:
$stream = new Consumer(OAUTH_TOKEN, OAUTH_SECRET, Phirehose::METHOD_FILTER);

Once you have made all these changes, you need to test out your code. Running the new code in the background with nohup is not an effective test method, since you can’t see the errror messages. To test the code, I would log into the directory with get_tweets.php and run it directly with:
php get_tweets.php

If it works, you will not see any error codes and data will flow into json_cache.php. Relive the thrill of when that first happened for you.

If it doesn’t work, there are a couple of things to try:
- Make sure all your paths are correct.
- Make sure you copied your OAuth tokens correctly into the config file.
- Make sure you downloaded a clean set of files from GitHub for Phirehose. It is very common to get HTML in the downloaded files by mistake.

If all else fails, I find cursing helps at this point. When you calm down, post a message at our Google Group, and I’ll try to help.

Javascript programming for Twitter changes dramatically with Twitter API version 1.1. The requirement to use OAuth with every API request means that you can no longer call the API directly from Javascript. Instead you have to rebuild all your Javascript code to proxy your requests through your own server.

I know that a lot of Twitter programmers don’t know how to program this way, so I’ve written a new ebook to explain this coding method. It is available as a free PDF download on our members page. You can also get the source code for all the examples in the ebook there as well. Twitter says that version 1.0 will be turned off on March 5th, so we all have to begin the upgrade process or else a lot of web pages are going to break. (Update: The turn-off date for API 1.0 has been extended to May 7th.)

I do my Javascript coding with jQuery on the client side and PHP on the server, so the ebook covers everything you need to know to create Twitter apps using these two languages.

Here are the topics covered:

• jQuery tutorial
• Ajax tutorial for client-server programming
• Getting a user timeline with Javascript
• Getting the results of a Twitter search with Javascript
• Creating a complete Twitter search app with proper tweet display formatting

I’ve also created a new Google Group for questions and discussions of the issues raised by this ebook.

My Twitter OAuth ebook has been out for 2 weeks now, and I’ve had a chance to help a lot of people get over the hump of running their first OAuth code. I’ve collected a list of the most common problems they have:

No callback URL
When you create an app, Twitter has an input field on the application creation page for filling in a callback URL. The URL is used when you create an OAuth login interface that lets people sign in on your site with Twitter. So if you are doing single-user OAuth, you could reasonably think that you can leave this blank. Twitter encourages this thinking by not requiring you to fill in the field on this form. The notes under the field also imply that you don’t need it: “To restrict your application from using callbacks, leave this field blank.” I’m not sure what this note means, but I do know that you MUST include a callback URL. If you don’t, the tmhOAuth library will not be able to make an OAuth connection and none of your API code will work. What URL should you use? It doesn’t matter, as long as it is valid. You can even use http://twitter.com.

Failure to set read write access
The Settings tab in the app creation page has a set of radio buttons that let you set the access level to read write. For some reason, this option is not displayed when you first create an app. You have to create the app, which is set to read only access by default, and then go to the Settings tab and change the access to read write. If you leave it as read only, you will not be able to tweet, follow, or do anything else with the API that changes an account.

Incorrect server clock
The OAuth system is very sensitive to differences between Twitter server clocks and your server. If your server’s clock is off by more than 5 or 10 minutes, all your OAuth requests will fail. If you don’t know how to check or set your server clock, ask your webhost.

Duplicate tweet
Some people have tried running my example post_tweet.php script and got a 403 error. This generally means that you have sent a duplicate tweet. There is a time limit after which duplicate tweets are allowed, but I’ve never been able to get an answer from Twitter HQ on what it is. If you get a 403 error when posting a tweet with the API, check your timeline to make sure this is not a duplicate of what you have already sent recently.

tmhOAuth files not found
There are only 2 files from the tmhOAuth files that you must use: cacert.pem and tmhOAuth.php. They both MUST be in the same directory and you have to use a valid path when you require or include tmhOAuth.php.

Invalid tmhOAuth files
I include the latest copies of the tmhOAuth files in the zip for the ebook, but some people prefer to download them from their home site at https://github.com/themattharris/tmhOAuth. That is fine, but you have to make sure you download clean copies of these files. I worked with someone for quite a while until we figured out that he had downloaded the entire page from Github, including the HTML, when he downloaded them.

I’m still interested in hearing about any problems you have with the ebook code. I want to make sure this is as clean as possible. Email me if you can’t get it to work.

Now that the switch to the Twitter API version 1.1 is getting closer, I’ve been asked if it will break the 140dev Framework. The requirement to switch to OAuth for API requests won’t affect the Streaming API, so version 1.1 won’t have any immediate impact on the the 140dev framework. Twitter has assured us that the Streaming API will still support Basic Authentication. I plan on converting the framework code to OAuth anyway as part of a long planned upgrade. I’ve learned a lot about performance enhancements in tweet collection since the framework was released, and I want to start building those improvements into the open source code for the framework. Right now I’m focused on writing a book on OAuth programming, but in a couple of months you can expect a new version of the framework to be released.

My Twitter OAuth ebook closes with the source code for an API Console application. This app got such a favorable response that I decided to enhance it and put it out as a free tool. I have found this to be an invaluable debugging aid when testing an API request. It lets you enter just the description of the request and quickly see the complete response without having to write any test code.

The tool is pretty obvious to use, but I also added a users guide to cover some of the features you might not notice, such as the ability to share the URL for an API request. This lets you easily demonstrate a bug in the API to Twitter HQ, or show a fellow developer how to do a specific API task.

For years now there has been a steady stream of requests, demands, pleas, and threats over the issue of getting user email addresses from the Twitter API. This thread is just one of many on the subject. The simple fact is that it ain’t gonna happen. No way, not ever. It’s pretty funny to see how angry devs get when they discover this. The better part is the reasons they give for having to get these addresses. They just want to get to know their users better.

It’s hard to say never when it comes to Twitter’s decision making process, but on this issue, I think never is never. Just give it up guys.

The solution is easy. If you want to get a user’s email address, just ask for it. We do that for this site, and get a pretty good response.

I’ve been having some Twitter discussions with devs about the breakage version 1.1 will cause for client-side Twitter apps. The simplest way to say it is that every page that uses direct Javascript calls to the Twitter API will break when version 1.0 is turned off in March. The only way you will be able to call the API will be with OAuth tokens. That is true for *ALL* API calls, even search. I don’t think people realize this yet, but they will when the switch to 1.1 happens.

So does this mean that Twitter development is done, and devs will just move elsewhere? No way. Devs go where the customers are, and the customers are using Twitter in increasing numbers. I do development as a business and as a businessman, I go where the money is. That is still Twitter.

The reality is that adapting to version 1.1 is not a big deal. The real change for client-side developers is that they will have to call their server with the API request, and let it call Twitter and return the results. In the end, the JS code is still making an Ajax request to a server and getting back JSON. The only difference will be that the server they call is one they control instead of calling Twitter’s API servers. Not a huge difference. Not enough to make devs leave all these users behind or give up coding.

It’s time to start converting your code to the Twitter API version 1.1. I’ve been using it for a while now, and it seems solid. Twitter is still saying that version 1.0 will be turned off on March 5th, so we all have to begin the upgrade process. (Update: The turn-off date for API 1.0 has been extended to May 7th.)

For a long time now I’ve been pointing people to my Hello OAuth tutorial as the best way to get started with OAuth. That will be obsoleted soon, so I’ve written up this new ebook to give people an expanded starting point. It covers everything you need to use OAuth from a single Twitter account. It is available as a free PDF download on our members page. You can also get the source code for all the examples in the ebook there as well.

Here are the topics covered:

• Create your first Twitter application
• Set up OAuth tokens for single-user access
• Connect to the Twitter API with the tmhOAuth library
• Posting tweets through the API
• Looking up complete account details for any Twitter user
• Converting Twitter API docs into working PHP code
• Debug and test any API request with your own API console

I’ve also created a new Google Group for questions and discussions of the issues raised by this ebook.

For the last couple of years we’ve been helping clients with a wide range of Twitter engagement tools. We’ve helped non-profits, political campaigns, and businesses identify and engage with the best Twitter accounts to help spread their message. Now we’ve committed our efforts to a free site that helps progressives unite for political action. UniteBlue.com combines a great set of Twitter accounts on the left with the same type of engagement tools we’ve perfected in our client work. Our first press release on the site has the details.

My latest ProgrammableWeb.com column is up, and it discusses the difference between follower quality and quantity. It’s time that the race for high follower count change into a desire for more engagement.