Twitter posted a notice today about the need for new SSL certificates:

At the end of 2013, all Browsers and Certificate Authorities will no longer support 1024 bits RSA certificates to be compliant to National Institute of Standards and Technology (NIST) guidelines [1].

The SSL certificate currently used on api.twitter.com is signed with the older Verisign G2 root CA certificate.

Due to NIST guidelines, api.twitter.com will change to a new certificate on Dec 10th, 2013. The new certificate will be signed with VeriSign Class 3 Secure Server CA – G3, which has the 2048 bits key length needed to meet recommended security levels.

This means that all HTTP clients used by your application must trust the new root certificate, otherwise you won’t be able to connect in the API. To ensure proper SSL certificate verification across all of Twitter’s services, your software should include all Verisign Root Certificates in its CAFile or other respective keystore. The root certificates are available at the following link:
* Verisign (https://www.symantec.com/page.jsp?id=roots)

I have to admit here that I don’t really know what an SSL certificate is. I know that it is a file that allows SSL to work, but beyond that my perception of this announcement was like Bart Simpson’s dog:

Blah blah blah Dec 10th blah blah everything breaks blah blah.

What I could tell was that I might need to download something and warn everyone using my code to do the same. So I immediately sent queries to the authors of Phirehose and tmhOAuth, the two libraries upon which all my Twitter code depends. The responses were that we are all good, and I could just ignore this cryptic, yet frightening warning.

Fenn Bailey, author of Phirehose, replied:

Phirehose has no visibility of SSL itself but simply uses PHP (and your OS’s) SSL layer. Basically, if PHP will work with these new certificates, Phirehose will.

And Matt Harris, author of tmhOAuth, said:

If you use the included cacert.pem you’ll be using the same SSL CA roots that Mozilla uses as of Sat Dec 29 20:03:40 2012 (the latest version) — so things should be fine.

Hooray! Everything is good. Nothing to worry about. If you were as mystified as me by the SSL certificate announcement, you can relax.

On a personal level, this lack of understanding of the layers beneath the code that I produce is like the perception of other scientific disciplines when I was in college. I was an organic chemistry major, and we used to laugh at the molecular biology majors, because they didn’t have good enough memories to hack Organic Chem. The physical chemists used to laugh at us, because we couldn’t handle the math to understand Pchem. The physicists laughed at the physical chemists, because they couldn’t base all their work on Quantum Theory. Who knows what the mathematicians thought of the physicists.

Modern Web development has so many moving parts and layers in its software stack that you just have to trust that the guys building the parts your code rests on know what they are doing. I just wish that the Twitter folks understood that many of their 3rd party devs are not CS majors, and yet we are able to build lots of cool apps.

Twitter has announced that they will be turning off basic authentication for the Streaming API on May 7th. I’m preparing a new version of the 140dev Framework to use OAuth now, and will have it posted tomorrow. That will be good for new users of this code, but if you already have the code in production, I assume you have made changes to it. Here is a detailed set of instructions for modifying an existing installation to use OAuth. The good thing is that you only have to use single-user OAuth, which is pretty easy to implement. These notes should give you everything you need.

Obvious, but very IMPORTANT note: Make a complete backup of any current version of this code you now have in production before making these changes.

There are 5 steps you have to follow to convert an existing installation of this code to use OAuth with the streaming API version 1.1.

1. Create a Twitter application at http://dev.twitter.com, and get a complete set of tokens. If you haven’t created an app before, you will find detailed instruction in my free ebook.

You need 4 tokens as defined on the details page of the application settings:
Consumer Key
Consumer Secret
Access Token
Access Token Secret

It is traditional to rename these tokens in every library that uses them. This has happened again with the Phirehose OAuth library. Here are the new names:
Consumer Key = TWITTER_CONSUMER_KEY
Consumer Secret = TWITTER_CONSUMER_SECRET
Access Token = OAUTH_TOKEN
Access Token Secret = OAUTH_SECRET

2. Modify the 140dev_config.php file in the 140dev Framework code. This is found in /db/140dev_config.php.

You need to replace these lines of code:
// Basic authorization settings for connecting to the Twitter streaming API
// Fill in the values for a valid Twitter account
define(‘STREAM_ACCOUNT’, ‘*****’);
define(‘STREAM_PASSWORD’, ‘*****’);

With these new lines. Insert the actual tokens you received from Twitter in step 1:
// OAuth settings for connecting to the Twitter streaming API
// Fill in the values for a valid Twitter app
define(‘TWITTER_CONSUMER_KEY’,'******’);
define(‘TWITTER_CONSUMER_SECRET’,'******’);
define(‘OAUTH_TOKEN’,'******’);
define(‘OAUTH_SECRET’,'******’);

3. Get new copies of the Phirehose library code. You need to delete Phirehose.php in your current installation. It will be found in the /libraries/phirehose/ directory. Then download these files and place them in the /libraries/phirehose/ directory:
Phirehose.php
OauthPhirehose.php

4. The latest version of Phirehose.php still points to the 1.0 version of the streaming API. To point to version 1.1, you need to change line 20 of Phirehose.php from this:
const URL_BASE = ‘https://stream.twitter.com/1/statuses/’;

To this:
const URL_BASE = ‘https://stream.twitter.com/1.1/statuses/’;

5. The final step is modifying your copy of get_tweets.php, which is found in /db/get_tweets.php. There are two code changes:

First replace these lines starting at line 13:
// Extend the Phirehose class to capture tweets in the json_cache MySQL table
require_once(CODE_DIR . ‘libraries/phirehose/phirehose.php’);
class Consumer extends Phirehose

With these new lines:
require_once(CODE_DIR . ‘libraries/phirehose/Phirehose.php’);
require_once(CODE_DIR . ‘libraries/phirehose/OauthPhirehose.php’);
class Consumer extends OauthPhirehose

Then replace this line:
$stream = new Consumer(STREAM_ACCOUNT, STREAM_PASSWORD, Phirehose::METHOD_FILTER);

With this one:
$stream = new Consumer(OAUTH_TOKEN, OAUTH_SECRET, Phirehose::METHOD_FILTER);

Once you have made all these changes, you need to test out your code. Running the new code in the background with nohup is not an effective test method, since you can’t see the errror messages. To test the code, I would log into the directory with get_tweets.php and run it directly with:
php get_tweets.php

If it works, you will not see any error codes and data will flow into json_cache.php. Relive the thrill of when that first happened for you.

If it doesn’t work, there are a couple of things to try:
- Make sure all your paths are correct.
- Make sure you copied your OAuth tokens correctly into the config file.
- Make sure you downloaded a clean set of files from GitHub for Phirehose. It is very common to get HTML in the downloaded files by mistake.

If all else fails, I find cursing helps at this point. When you calm down, post a message at our Google Group, and I’ll try to help.

My Twitter OAuth ebook has been out for 2 weeks now, and I’ve had a chance to help a lot of people get over the hump of running their first OAuth code. I’ve collected a list of the most common problems they have:

No callback URL
When you create an app, Twitter has an input field on the application creation page for filling in a callback URL. The URL is used when you create an OAuth login interface that lets people sign in on your site with Twitter. So if you are doing single-user OAuth, you could reasonably think that you can leave this blank. Twitter encourages this thinking by not requiring you to fill in the field on this form. The notes under the field also imply that you don’t need it: “To restrict your application from using callbacks, leave this field blank.” I’m not sure what this note means, but I do know that you MUST include a callback URL. If you don’t, the tmhOAuth library will not be able to make an OAuth connection and none of your API code will work. What URL should you use? It doesn’t matter, as long as it is valid. You can even use http://twitter.com.

Failure to set read write access
The Settings tab in the app creation page has a set of radio buttons that let you set the access level to read write. For some reason, this option is not displayed when you first create an app. You have to create the app, which is set to read only access by default, and then go to the Settings tab and change the access to read write. If you leave it as read only, you will not be able to tweet, follow, or do anything else with the API that changes an account.

Incorrect server clock
The OAuth system is very sensitive to differences between Twitter server clocks and your server. If your server’s clock is off by more than 5 or 10 minutes, all your OAuth requests will fail. If you don’t know how to check or set your server clock, ask your webhost.

Duplicate tweet
Some people have tried running my example post_tweet.php script and got a 403 error. This generally means that you have sent a duplicate tweet. There is a time limit after which duplicate tweets are allowed, but I’ve never been able to get an answer from Twitter HQ on what it is. If you get a 403 error when posting a tweet with the API, check your timeline to make sure this is not a duplicate of what you have already sent recently.

tmhOAuth files not found
There are only 2 files from the tmhOAuth files that you must use: cacert.pem and tmhOAuth.php. They both MUST be in the same directory and you have to use a valid path when you require or include tmhOAuth.php.

Invalid tmhOAuth files
I include the latest copies of the tmhOAuth files in the zip for the ebook, but some people prefer to download them from their home site at https://github.com/themattharris/tmhOAuth. That is fine, but you have to make sure you download clean copies of these files. I worked with someone for quite a while until we figured out that he had downloaded the entire page from Github, including the HTML, when he downloaded them.

I’m still interested in hearing about any problems you have with the ebook code. I want to make sure this is as clean as possible. Email me if you can’t get it to work.

My Twitter OAuth ebook closes with the source code for an API Console application. This app got such a favorable response that I decided to enhance it and put it out as a free tool. I have found this to be an invaluable debugging aid when testing an API request. It lets you enter just the description of the request and quickly see the complete response without having to write any test code.

The tool is pretty obvious to use, but I also added a users guide to cover some of the features you might not notice, such as the ability to share the URL for an API request. This lets you easily demonstrate a bug in the API to Twitter HQ, or show a fellow developer how to do a specific API task.

For years now there has been a steady stream of requests, demands, pleas, and threats over the issue of getting user email addresses from the Twitter API. This thread is just one of many on the subject. The simple fact is that it ain’t gonna happen. No way, not ever. It’s pretty funny to see how angry devs get when they discover this. The better part is the reasons they give for having to get these addresses. They just want to get to know their users better.

It’s hard to say never when it comes to Twitter’s decision making process, but on this issue, I think never is never. Just give it up guys.

The solution is easy. If you want to get a user’s email address, just ask for it. We do that for this site, and get a pretty good response.

I’ve been having some Twitter discussions with devs about the breakage version 1.1 will cause for client-side Twitter apps. The simplest way to say it is that every page that uses direct Javascript calls to the Twitter API will break when version 1.0 is turned off in March. The only way you will be able to call the API will be with OAuth tokens. That is true for *ALL* API calls, even search. I don’t think people realize this yet, but they will when the switch to 1.1 happens.

So does this mean that Twitter development is done, and devs will just move elsewhere? No way. Devs go where the customers are, and the customers are using Twitter in increasing numbers. I do development as a business and as a businessman, I go where the money is. That is still Twitter.

The reality is that adapting to version 1.1 is not a big deal. The real change for client-side developers is that they will have to call their server with the API request, and let it call Twitter and return the results. In the end, the JS code is still making an Ajax request to a server and getting back JSON. The only difference will be that the server they call is one they control instead of calling Twitter’s API servers. Not a huge difference. Not enough to make devs leave all these users behind or give up coding.

This free ebook covers everything you need to use OAuth from a single Twitter account. It is available as a free PDF download on our members page. You can also get the source code for all the examples in the ebook there as well.

Here are the topics covered:

• Create your first Twitter application
• Set up OAuth tokens for single-user access
• Connect to the Twitter API with the tmhOAuth library
• Posting tweets through the API
• Looking up complete account details for any Twitter user
• Converting Twitter API docs into working PHP code
• Debug and test any API request with your own API console

I’ve also created a new Google Group for questions and discussions of the issues raised by this ebook.

One benefit of running an app within Twitter.com is that it will eliminate the need for the OAuth dance. That is the complex exchange that goes on between a website and the Twitter API when the user logs into Twitter through the website. This communication is necessary to deliver a set of authorization keys that can be used by the site to get the Twitter account information for the user.

You’ve seen this dance before, even if you didn’t realize exactly what was happening. You are on a website, and see a button that says . Clicking that button takes you to a Twitter controlled page where you can give permission for the site that sent you there to operate on your behalf. When you then give permission, you are sent back to the site to continue where you left off. This whole procedure confuses both the user and the developer who has to implement it. One way of measuring how big a hurdle this is for Twitter API developers is the fact that my tutorial on using OAuth is the number one visited page on this site. It gets about 30% of all of my site traffic, even though it was written 2 years ago. There is no doubt that the OAuth dance is a pain for everyone involved.

As part of making it possible for developers to run apps within tweets, Twitter will have the chance to streamline this whole process. The Twitter page will already know who the logged in user is. It is just a matter of making this available to the application code running within the tweet.

There are several possible to ways to implement an API that passes this info. I can’t predict exactly how this will be done, but I am sure it will have to be done. Think about what would happen otherwise. A user who is logged into Twitter sees an app in a tweet, and tries to interact with it. Will Twitter then send that user to another sign-in page to log in again? That makes no sense. There will have to be a transparent way to let the app discover who the current user is.

Explicit authorization by the user will still be necessary before the app can change the user’s Twitter account or tweet on their behalf, but the UI for that will also have to be handled by Twitter. Otherwise the clean and consistent experience Twitter is striving for will not be achieved. I handle UI issues like this all the time in the website apps I build for clients. I’m sure Twitter can figure out how to make it happen. They have no choice.

Putting these implementation details aside, the real take away is that apps within tweets will be easier to build and easier for users to interact with. That has to be good for everyone involved, and result in increased adoption and production of Twitter apps.

There is something about OAuth that brings out the worst in techies. You can see it when someone asks how to get started with OAuth on the Twitter development talk mailing list. The general response is “get a copy of library X, and you’re all set.” Well if downloading a library would solve the problem, I don’t think so many people would keep asking for help.

The disconnect is that all the Twitter docs on OAuth assume that you already know how it works. It’s like giving driving directions to someone who does not know how to operate a car. If someone has never driven before, saying “Just go north 5 miles, and you’re all set” doesn’t help much.

What OAuth beginners need is a step by step set of instructions that starts by showing where all the moving parts are, and how to assemble them for a first working program that controls the Twitter API. That is what I have tried to do with my latest tutorial. It is called Hello Twitter OAuth, and it shows every step necessary to post tweets with OAuth. These techniques can then be applied to any API command.