Twitter lists don’t get the respect they deserve, but here is a way to use a well curated list as the source of great leads. You could just use the members of a list as leads themselves, but that will only give you a few hundred accounts. If you use the following procedure, you can turn a list into tens of thousands of accounts that are all interested in a specific subject.

First you start with a good Twitter list whose members have been carefully selected. Ironically, Twitter doesn’t offer a method of searching for its own lists, but Google does a good job of this. I find that a search like “twitter list [subject]” works best. With the Olympics in the news, let’s try to find a great list of sports journalists as a starting point. This can be used to find thousands of Twitter users who are avid consumers of sports news, a great lead list for anyone promoting sports related products.

A Google search for twitter list sports journalists leads me to the @SportSJA account. Clicking on the account’s Lists link gave me this excellent list.

The assumption I will make when data mining this list is that the more members of this list someone follows, the more they are interested in sports. Someone who follows a few members may just be an accident, but if an account follows a couple of dozen list members, they are a sports fanatic. Those are the people I’m looking for.

The next step is to collect all the followers of every member of this list. I do this by writing code that first collects all the lists members with the /lists/members API call. This gives me the user ids of the list members, which are stored in a list_members table.

Then I have a script loop through the user ids in the list_members table, and collect all the followers of each one with the /followers/ids request. This is the slowest part of the process, because only 60 follower requests can be made per hour, retrieving up to 5,000 followers each time. Depending on the number of list members and average followers of each, this can take a few hours up to several days. Running a script that does 15 requests as a cron job once every 15 minutes will eventually chew through all the data.

All of the follower user ids I get in this stage are added to a list_member_followers table. The important trick is that I allow duplicates. If someone follows 10 members of the list, their user id is added to the list_member_followers table 10 times.

Since users are added multiple times to this table, you can get a count of the total unique users with the query:

SELECT distinct user_id
FROM list_member_followers

From my experience, a list with close to 500 members (this one has 485), will deliver on the order of 500,000 to 1M unique followers.

I don’t want all the followers, just the best ones. The top 10% is usually enough, so if there are 500,000 followers, the 50,000 who follow the most are my target. I can get these with:

SELECT count(*) as cnt, user_id
FROM list_member_followers
GROUP BY user_id
ORDER BY cnt DESC
LIMIT 50000

To make sure these are all good leads, I need to review their account stats, such as follower count and number of tweets. I can have a script read through these 50,000 accounts, use /users/lookup to collect the account info, and store it in a list_member_leads table. This works on 100 users at a time, and can be run 720 times an hour. This means that collecting all the account data will take (50000/100/720) or .7 hours. Not bad for 50,000 potential leads.

Finally, I can run a SQL request that deletes any of the lead accounts that have poor stats, such as less than 50 friends, 50 followers, or 50 tweets. I also tend to delete accounts with egg avatars and no descriptions. This usually eliminates about 20% of the total.

The result is about 40,000 leads, all of whom have shown that they really want info on sports. And best of all, this data is free. Pretty amazing. Remember, automated following of a list like this is forbidden by Twitter’s terms of service, but there are many ways to track and engage with these users. What you need is a solid reporting and engagement management system to approach these users and record your engagement. Sort of like a CRM for Twitter leads. Luckily, I happen to have a book on just this subject.

My last post on lead generation described the best techniques for identifying popular tags for any subject. You can use this approach to select accounts to follow, but that is just half of the solution. You also want these accounts to follow you back.

When you follow someone, the most likely thing that person will do is check out your account quickly to see if you are worth a follow back. If they see a tag in your description that shows you are passionate about that subject, they are probably going to follow you in return. It is a way of proving that you are a member of their community.

The other place that description tags are helpful is in a Twitter search. Most people think of search.twitter.com as a way of finding tweets, but when someone clicks the people link at the top of the search results, Twitter is going to rank your account high if it finds the right tag in your description. Think of this as Twitter SEO for gaining followers.

Searching for hashtags is a common method of finding new leads on Twitter, but tags in tweets can be misleading. Sarcasm and irony are so often used in tweets that finding a tag doesn’t tell you if that user is for or against the tag’s meaning. In fact, tags are often used in tweets to make sure the opposite side sees the tweet in search. It is a way of saying “in your face” in a tweet.

A much more accurate indicator is the use of a tag within an account description. After this last Super Bowl, I’m sure lots of Seahawks fans sent derisive tweets with the #broncos tag, but I doubt if any of them put #broncos in their account bio. From my experience, a tag used in an account bio is the most accurate way of qualifying a lead for any issue.

My Engagement Programming book has a couple of scripts that can help you find the most popular tags for any subject. You can use this script to extract all the tags found in user descriptions you have collected with the API, and this report to see which tags are most popular.

Once you have identified the best tags for any subject, you can qualify a list of users by finding those who have these tags in their description.

Getting thousands of potential leads for any topic is no problem with the Twitter API. Identifying the best leads is the real challenge. There are two dates that can be effective in this task: the date when the Twitter account was created, and the date of the last tweet sent by the account. I like to qualify leads by making sure that the account is at least 30 days old, and has tweeted within the last 30 days. The Twitter API makes it easy to get both of these values. Each API request, such as the search or streaming APIs, includes these dates for each user it returns.

A simple way to see this in action is with the /users/show API call. Here is an example for @LadyGaga using this site’s API Console. You’ll have to log in with your Twitter account to use this tool, but if you just want a quick look at the results, here is a screenshot:

You can apply this technique when you extract results from either the search or streaming API. For example, if you have the user’s data in a variable called $user, you can get the account’s creation date with $user->created_at, and the last tweet date with $user->status->created_at.

My last post on lead generation introduced the idea of a follow signature. This is a technique that allows you to rank a user’s interest in a subject by counting the number of experts they follow.

This method is especially accurate when dealing with a set of users who hold opposing views. For example, how would you separate a Red Sox fan from a Yankees fan? You could count the number of times someone tweets with “red sox,” but what if they are tweeting “The Red Sox suck” or “I hate the Red Sox!”? Does that make them a Red Sox fan?

This is where a follow signature is really useful. No matter what they tweet, if someone follows 20 to 30 rabid Red Sox fans, there is no way they are a Yankees fan. They could never tolerate all those positive Red Sox tweets in their timeline. The same trick works the other way around. A Yankees fan will follow many of the Yankees experts, and hardly any Red Sox experts.

This allows you to set up two expert counts: one for the number of experts on one side of the argument, and another for the opposite site. You then can use the absolute counts, or even better, the ratio of the two expert counts to qualify a lead list.

I’ve also seen this approach work really well in politics. A progressive may follow Ted Cruz or Rand Paul to keep track of the wacky things they tweet, but nobody on the left will want to follow dozens of Tea Party accounts, and have those tweets showing up all the time in their timeline. The same thing works in reverse to identify conservatives.

As I said in the previous post on lead generation, it’s easy to find thousands of leads for any subject on Twitter. Assuring the quality of these accounts, however, is a time consuming task. What is also needed is an automated test that will filter out the highest quality leads.

An effective signature is a user who follows many expert accounts on the desired subject. Let me offer a concrete example. If I’m looking for users who care a lot about sushi restaurants, I would start by identifying the most influential accounts who tweet about sushi. This will take some time, but once I have a list of 100 sushi experts, I can use that to screen anyone and measure their own interest in sushi.

If someone follows one or two sushi experts, it may be a coincidence, or they could just be a spam account that accidentally followed sushi accounts. A user who follows 10 to 20 sushi experts is probably interested in sushi themselves. If they follow 50 or more sushi experts, they are a sushi freak, and will click every link presented to them that will lead to that ultimate 10 in sushi restaurants. For the record, my ultimate 10 is the sushi bar in Harrods in London.

Here is a technical perspective on how to implement this model:
1. Create a table, we’ll call it experts, that contains the user_id and screen name of 50 to 100 experts on any subject.
2. Create a table, we’ll call this one leads, that contains the user_id of your unqualified leads. Along with any other data you need about the lead accounts, include a expert_count field in the leads table, which will end up containing the number of experts they follow.
3. Loop through each of the leads, and collect the accounts they follow with the /friends/ids API call.
4. Loop through each of the friends of a lead, and test to see if they exist in the experts table. If a match is found, increment the expert_count field for that lead.

Unfortunately, /friends/ids can only be called 15 times every 15 minutes, but if you set this up with a cron job, you can screen 1,440 leads per day. After a day or two, you can sort the leads list to find those with the highest expert_count. These will be your best leads. You can only follow a few hundred accounts per day, so this process will quickly start giving you more than enough leads for a daily follow campaign.

This type of system can be run continuously. You can have a data collection script based on the streaming API or REST API that is steadily adding new accounts to the leads table, a screening script that is qualifying them as described above, and then a script that allows a user to follow the best leads by hand.

This is pretty cool once you get it running, great leads just start bubbling up to the top of the leads table when sorted by expert_count descending. Whenever I create a system like this, I am amazed at the quality of accounts that emerge. Twitter is the best lead generation system ever created.

Remember, automated following is forbidden, so this final step of actually following someone must be done manually, although you are allowed to create a web page that makes each follow a simple click. Once someone is followed, they can be flagged or deleted, so they aren’t offered for following again.

When compiling a list of prospective Twitter leads for any subject area, an obvious approach is to find accounts that tweet about that subject. This can be done with the streaming API or REST API. This still leaves a lot of curating work to select the highest quality accounts from this group.

A valuable shortcut is to build on the selection work already done by experts on that subject. When you identify an account that speaks authoritatively and focuses largely on your topic, you can copy the accounts they follow into a database, and use that as the starting point for a good lead list.

The list of accounts an expert is following will almost always be smaller than those who are following them, but the quality will be much higher. Spam followers are a chronic problem, and any list of followers will have a high number of spam and bot accounts. Nobody will deliberately follow a spam account, so the people the expert has chosen to follow is going to be much better.

A following list is not going to be perfect, however. Twitter users are human and have other interests, so you still have to review the accounts they are following to filter out inappropriate accounts. There is also the long-term Twitter bug that has most accounts randomly follow others at a low rate. All the accounts I manage seem to follow at least 10-20 accounts a day all by themselves. They are pretty easy to spot, since they are often in languages I don’t read.

From a technical viewpoint, you need to use the /friends/ids API call to get the accounts a user is following. You can find a complete PHP script for using this API to collect friends in my engagement programming source code.

A lot of my engagement programming for clients has involved finding good leads for them to build relationships with and gain as followers. Obviously, the search API is a good place to start, but just searching for keywords that match your engagement interests is not the best approach. For example, if I wanted to find high quality accounts that talk about food, I could run a search for keywords like food, restaurant, or cooking. Try this search and see how poor the quality of the tweets are. All you can really tell is that these users are probably hungry when they tweeted.

You can get much better results by using the leading accounts for your subject as the search keyword. Compare the results of the previous search with this search based on @FoodNetwork. Now we get people tweeting recipes, discussing cooking shows, and a good mix of celebrity chef accounts. The difference in quality is obvious. The rule is that people who tweet with the screen name of a leading account have a serious and informed interest in the subject. This group should be your starting point for an engagement campaign.

You can do your engagement directly from the search results, but if you want to store the results in a database for later analysis, you can start coding with this source code from my engagement programming book.

Part of the sales process for Twitter consulting is convincing a new client that Twitter is more than just another way to broadcast their message. You have to show them that what appears to be a random stream of tweets is really a collection of highly qualified sales prospects. By aggregating Twitter users as well as their Tweets, you can extract a great set of sales leads along with their contact info. One way to quickly demonstrate the value of tweet aggregation is to deliver an Excel spreadsheet of sales prospects that meet the client’s needs.

When you aggregate tweets from the Twitter streaming API, it also returns the complete account profile for each user. You can data mine this collection of users to extract highly targeted lists of users, along with their geographical location and home page URL.

The free 140dev Twitter framework is an example of the code you will need to do the tweet aggregation. The schema for the MySQL database it creates shows you it has a table for all the aggregated tweets, which links to the list of tweeting users. Since all of this data is collected for a specific set of keywords, you can then extract personal details on the users who tweet these keywords the most with a simple SQL statement:

SELECT count(*) AS cnt, users.screen_name, users.name, users.location, users.url
FROM tweets, users
WHERE tweets.user_id = users.user_id
AND users.location != ''
AND users.url != ''
GROUP BY tweets.user_id
ORDER BY cnt DESC
LIMIT 1000

The 140dev framework’s example database collects tweets for the keyword “recipe”, so this query gives us the most active tweeters in the food world. Here are the results in phpMyAdmin:

You can then export the results from phpMyAdmin to an Excel spreadsheet, and email it to your client. This gives them solid data in a familiar form. Twitter doesn’t deliver email addresses, and doesn’t even collect phone numbers, but you do get each user’s home page URL. This can be used to gather other contact info, a task that is easily farmed out to people on freelance sites like Mechanical Turk.

So the next time you want to convince a client that Twitter is not just a bunch of kids talking to each other, you can just create a tweet aggregation database for the client’s industry keywords, let it collect data for a few days, and pull out a list of targeted users.

I went to the OpenCoffee meetup in Cambridge the other day. They all recognized the importance of Twitter, but don’t understand how to make money from it. We are exactly where we were in 1996 with the World Wide Web when I helped start Andover.net. Great point in the cycle.

So here is my quick 5 point pitch on how clients can benefit from integrating Twitter into business and marketing models. But first keep in mind that you don’t make money “from Twitter”, you make money “with Twitter”. Meaning that Twitter is a lever for improving your other efforts, but you don’t get cash handed to you directly by Twitter users on Twitter. Anyway, here’s my list:

1. Putting keyword targeted tweets on pages in the right way is great for SEO. Google loves tweets. This will increase the page’s search rank, getting a lot more first time visitors.

2. Datamining of tweets lets you find the right people to follow in Twitter for your market. This can be used very effectively to build a big follower list. This list becomes profitable when you tweet messages with URLs you want people to click on. Think of it as free Adwords.

3. Follower lists are also essential if you want to make people do something in the real world, like contribute money, or go to an event. Twitter will be huge in 2012 election.

4. If you have a database of tweets, you can datamine it for sales leads. You can give sales people the Twitter accounts and home page URLs of people who tweet a lot about the products the salesperson is selling. The best part is that the salesperson can see exactly what prospects say about their products and competitors before contacting them.

5. You can also datamine a tweet database for sentiment trends. This is valuable for PR and customer service. It gives you a real-time read on how effective the rest of your communication program is.