How to Build a Simple Twitter Client using Python

Building a Twitter client is nothing new to Switch On The Code. We’ve done it in WCF and Silverlight. To be honest though, using the Twitter API is a great way to exercise two very useful parts of a language and framework – making web requests and parsing XML. That’s what we’re going to do today. We’re going to build a very simple command line twitter client (read-only) using Python.

Today’s app will be very simple. It will request Switch On The Code’s Twitter feed and display them with their dates. Here’s what the first five tweets in the output will look like:

Date: Thu Jul 22 19:46:19 +0000 2010
Tweet: VS 2010 Productivity Power Tools Update (with some cool new features) http://sotc.me/46680 via @scottgu #vs2010 #programming

Date: Thu Jul 22 17:34:45 +0000 2010
Tweet: SQLite 3.7.0 released http://sotc.me/92864 #sqlite

Date: Wed Jul 21 19:08:01 +0000 2010
Tweet: 20+ Required #Windows Apps: Web DesignerGÇÖs Choice http://sotc.me/28662 via @nettuts – I personally love Notepad++ myself!

Date: Mon Jul 19 20:20:23 +0000 2010
Tweet: Windows Phone 7 in-depth preview http://sotc.me/58189 via @engadget #wp7 #microsoft

Date: Mon Jul 19 17:13:38 +0000 2010
Tweet: Would love to get a hold of Windows Phone 7 device to build a couple applications for it, anyone know how to go about that? #windowsphone7

Python has a very large and powerful framework behind it, do doing simple things like making web requests and parsing XML are fairly straight forward and don’t require a lot of code. Let’s start by requesting the XML from Twitter’s API.

import httplib

# Open a connection to twitter.com.
twitterConn = httplib.HTTPConnection("www.twitter.com")

# Requestion Switch On The Code’s tweets.
twitterConn.request("GET", "/statuses/user_timeline.xml?screen_name=SwitchOnTheCode")
twitterResponse = twitterConn.getresponse()

# Check that everything went ok.
if twitterResponse.status != 200:
  print("Failed to request tweets.")
 
# Read the response.
tweets = twitterResponse.read()

# Close the connection.
twitterConn.close()

The first thing we do is create an HTTPConnection object for twitter.com. We then tell it to request the timeline for the user SwitchOnTheCode. Next, we get the response as an HTTPResponse object. I do a quick check to make sure the request went ok, since we all know how often Twitter goes down. All that’s left to do is simply read the contents of the response, which will be our XML.

Now we need to parse the XML. There are a couple of available options, but I chose the DOM approach as I think it makes more logical sense.

import httplib
from xml.dom.minidom import parseString

# Open a connection to twitter.com.
twitterConn = httplib.HTTPConnection("www.twitter.com")

# Requestion Switch On The Code’s tweets.
twitterConn.request("GET", "/statuses/user_timeline.xml?screen_name=SwitchOnTheCode")
twitterResponse = twitterConn.getresponse()

# Check that everything went ok.
if twitterResponse.status != 200:
  print("Failed to request tweets.")
 
# Read the response.
tweets = twitterResponse.read()

# Close the connection.
twitterConn.close()

#Parse the XML.
twitterDom = parseString(tweets)

# Find all the status tweets.
for tweet in twitterDom.getElementsByTagName("status"):
  for tweetParts in tweet.childNodes:
    # Find the date tag.
    if tweetParts.nodeName == "created_at":
      for textNode in tweetParts.childNodes:
        # Find the contents of the date tag.
        if textNode.nodeType == textNode.TEXT_NODE:
          print("Date: " + textNode.nodeValue)
    # Find the tweet tag.
    elif tweetParts.nodeName == "text":
      for textNode in tweetParts.childNodes:
      # Find the contents of the tweet tag.
        if textNode.nodeType == textNode.TEXT_NODE:
          print("Tweet: " + textNode.nodeValue.encode(‘utf-8’) + "\n")

Using Python’s minidom, we first parse the XML returned from Twitter using parseString (don’t forget to import parseString at the top of the file). Now things get a litte tricky. We need to rip through the DOM looking for the items we want to display. First off, every tweet is stored beneath a “status” tag, so we need to loop through every one of those. Underneath each status tag there’s the date (created_at) and the actual tweet (text), so we need to loop through every child of the status tag looking for those. Once we find one, we then need to find the text contained within it. Text is contained in TEXT_NODE type nodes, so we need to search through the children of our created_at and text tags looking for that specific type. Once we find it, we then print the nodeValue and we’re done.

Since tweets are Unicode, I needed to encode the text as utf-8 before printing it to the console.

And that’s it. With just a few lines of code, and some pretty complicated DOM traversing, we’ve got a very basic Twitter reader built in Python. If you’ve got any questions or comments, feel free to leave them below or head on over to our forums.

How to Authenticate Users With Twitter OAuth


Beginning August 16th, Twitter will no longer support the basic authentication protocol for its platform. That means the only way to authenticate users will be through a Twitter application. In this tutorial, I’ll show you how to use Twitter as your one-click authentication system, just as we did with Facebook.


Step 1: Setting Up The Application

We’ll first need to set up a new Twitter application.

  • Register a new app at dev.twitter.com/apps/
  • Fill in the fields for your site accordingly, just be sure to select Browser in Application Type, and set the Callback URL to something like http://localhost.com/twitter_login.php (http://localhost/ won’t be accepted because it doesn’t have a domain name).
  • Finally, select Read & Write. Fill in the captcha, click “Register Application,” and accept the Terms of Service.

Now, you’ll see the screen as shown below.

We will be using the Consumer key and Consumer secret values shortly.

Now that this is done, let’s download a library. As we will be coding with PHP, it seems the best one is twitteroauth; but if you’re using another language, you’ll find other good libraries here.

Find the twitteroauth directory inside the zip file, and extract it to your application’s folder.

Finally, since we’re using Twitter to authenticate users, we’ll need a database table to store those users. Here’s a quick example of what we will be doing.

CREATE TABLE `users` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `oauth_provider` varchar(10),
    `oauth_uid` text,
    `oauth_token` text,
    `oauth_secret` text,
    `username` text,
    PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1;

Notice the oauth_token and oauth_secret fields. Twitter’s OAuth requires token and a token_secret values to authenticate the users, so that’s why we’re including those. With that, we are done with the setup!


Step 2: Registering Users

In this step we, will be doing three things:

  • Requesting authorization from Twitter.
  • Registering or, if the user is already registered, logging the user in.
  • Setting the data into a session.

Requesting authorization

The OAuth workflow starts by generating a URL for the request; the user is redirected to that URL and is asked for authorization. After granting it, the application redirects back to our server with two tokens in the URL parameters, which are required for the authentication.

Let’s begin by including the library and starting a session handler.

require("twitteroauth/twitteroauth.php");
session_start();

After that, let’s create a new TwitterOAuth instance, giving it the consumer key and consumer secret that Twitter gave us when we created the application. Then, we’ll request the authentication tokens, saving them to the session, and redirect the user to Twitter for authorization.

// The TwitterOAuth instance
$twitteroauth = new TwitterOAuth('YOUR_CONSUMER_KEY', 'YOUR_CONSUMER_SECRET');
// Requesting authentication tokens, the parameter is the URL we will be redirected to
$request_token = $twitteroauth->getRequestToken('http://localhost.com/twitter_oauth.php');

// Saving them into the session
$_SESSION['oauth_token'] = $request_token['oauth_token'];
$_SESSION['oauth_token_secret'] = $request_token['oauth_token_secret'];

// If everything goes well..
if($twitteroauth->http_code==200){
    // Let's generate the URL and redirect
    $url = $twitteroauth->getAuthorizeURL($request_token['oauth_token']);
    header('Location: '. $url);
} else {
    // It's a bad idea to kill the script, but we've got to know when there's an error.
    die('Something wrong happened.');
}

Save it as twitter_login.php, go to http://localhost.com/twitter_login.php or whatever your local host name is. If everything went correctly, you should be redirected to twitter.com, and you should see something like this.

Click allow, and you will be redirected to http://localhost.com/twitter_oauth.php — since we set this URL as a parameter in the getRequestToken statement. We haven’t created that file, so it should throw an error. Create that file, and then include the library and start a session, just like we did in the first file.

After that, we will need three things:

  • Auth verifier in the URL query data
  • Auth token from the session
  • Auth secret from the session

So, the first thing to do in this script is validate this data and redirect if one of these variables is empty.

if(!empty($_GET['oauth_verifier']) && !empty($_SESSION['oauth_token']) && !empty($_SESSION['oauth_token_secret'])){
    // We've got everything we need
} else {
    // Something's missing, go back to square 1
    header('Location: twitter_login.php');
}

Now, if everything is set, inside the conditional we will be creating the TwitterOAuth instance, but with the tokens we just got as third and fourth parameters; after that, we will be getting the access token, which is an array. That token is the one we will be saving to the database. Finally, we’ll do a quick test to see if everything works out.

// TwitterOAuth instance, with two new parameters we got in twitter_login.php
$twitteroauth = new TwitterOAuth('YOUR_CONSUMER_KEY', 'YOUR_CONSUMER_SECRET', $_SESSION['oauth_token'], $_SESSION['oauth_token_secret']);
// Let's request the access token
$access_token = $twitteroauth->getAccessToken($_GET['oauth_verifier']);
// Save it in a session var
$_SESSION['access_token'] = $access_token;
// Let's get the user's info
$user_info = $twitteroauth->get('account/verify_credentials');
// Print user's info
print_r($user_info);

If nothing goes wrong, the print_r should show the user’s data. You can get the user’s id with $user_info->id, his or her username with $user_info->screen_name; there’s a bunch of other info in there as well.

It is very important to realize that the oauth_verifier hasn’t been used before this. If you see the user’s info correctly and then reload the page, the script will throw an error since this variable has been used. Just go back to twitter_login.php and it will automatically generate another fresh token.

Registering users

Now that we have the user’s info we can go ahead and register them, but first we have to check if they exist in our database. Let’s begin by connecting to the database. Add these lines in the script’s beginning.

mysql_connect('localhost', 'YOUR_USERNAME', 'YOUR_PASSWORD');
mysql_select_db('YOUR_DATABASE');

Modify the database info as required. Now, just below where we fetch the user’s info, we’ll have to check for the user in our database. If he or she is not there, we’ll enter the info. If the user has been registered, we must update the tokens, because Twitter has generated new ones and the ones we have in the database are now unusable. Finally, we set the user’s info to the session vars and redirect to twitter_update.php.

if(isset($user_info->error)){
    // Something's wrong, go back to square 1
    header('Location: twitter_login.php');
} else {
    // Let's find the user by its ID
    $query = mysql_query("SELECT * FROM users WHERE oauth_provider = 'twitter' AND oauth_uid = ". $user_info->id);
    $result = mysql_fetch_array($query);

    // If not, let's add it to the database
    if(empty($result)){
        $query = mysql_query("INSERT INTO users (oauth_provider, oauth_uid, username, oauth_token, oauth_secret) VALUES ('twitter', {$user_info->id}, '{$user_info->screen_name}', '{$access_token['oauth_token']}', '{$access_token['oauth_token_secret']}')");
        $query = mysql_query("SELECT * FROM users WHERE id = " . mysql_insert_id());
        $result = mysql_fetch_array($query);
    } else {
        // Update the tokens
        $query = mysql_query("UPDATE users SET oauth_token = '{$access_token['oauth_token']}', oauth_secret = '{$access_token['oauth_token_secret']}' WHERE oauth_provider = 'twitter' AND oauth_uid = {$user_info->id}");
    }

    $_SESSION['id'] = $result['id'];
    $_SESSION['username'] = $result['username'];
    $_SESSION['oauth_uid'] = $result['oauth_uid'];
    $_SESSION['oauth_provider'] = $result['oauth_provider'];
    $_SESSION['oauth_token'] = $result['oauth_token'];
    $_SESSION['oauth_secret'] = $result['oauth_secret'];

    header('Location: twitter_update.php');
}

Note that these queries are not validated; if you leave them as they are, you are leaving your database vulnerable. Finally, below the database connection, we should set a check to verify that the user is logged in.

if(!empty($_SESSION['username'])){
    // User is logged in, redirect
    header('Location: twitter_update.php');
}

You can now greet the user by his or her username.

<h2>Hello <?=(!empty($_SESSION['username']) ? '@' . $_SESSION['username'] : 'Guest'); ?></h2>

Let’s get to the fun side: updating, following and reading.


Step 3: Reading Statuses

There are over twenty categories of resources available: timeline, tweets, users, trends, lists, direct messages, etc. Each one has a bunch of methods, you can check them all in the official documentation. We’ll get to the basics, as most of these features are accessed in a similar way.

Just like the other two scripts, we’ll need to create the TwitterOAuth instance, including the variables in the session.

if(!empty($_SESSION['username'])){
    $twitteroauth = new TwitterOAuth('YOUR_CONSUMER_KEY', 'YOUR_CONSUMER_SECRET', $_SESSION['oauth_token'], $_SESSION['oauth_secret']);
}

We’ll begin with the user’s timeline. The reference tells us that the path is statuses/home_timeline; ignore the version and format, the library will take care of it.

$home_timeline = $twitteroauth->get('statuses/home_timeline');
print_r($home_timeline);

That will get you the timeline. You can fetch each item with a foreach loop. However, the reference specifies some optional parameters like count, which limits how many tweets will be fetched. In fact, get‘s second parameter is an array of every option needed, so if you want to fetch the latest forty tweets, here’s the code:

$home_timeline = $twitteroauth->get('statuses/home_timeline', array('count' => 40));

Also, you can see somebody else’s timeline, as long as it’s not protected. statuses/user_timeline requires either a user’s id or screen name. If you want to check @nettuts timeline, you’ll have to use the following snippet:

$nettuts_timeline = $twitteroauth->get('statuses/user_timeline', array('screen_name' => 'nettuts'));

As you can see, after authenticating, reading timelines is a breeze.


Step 4: Friendships

With friendships, you can check if a user follows another one, as well as follow or unfollow other users. This snippet will check if you are following me and and will create the follow if not.

But first, check the friendships/exists and friendships/create reference. Notice something? friendships/create method is POST. Fortunately, the library includes a post() function, which works just as the get() function; the main difference is that get() is for reading and post() is for creating, deleting or updating.

Anyways, friendships/exists requires two parameters: user_a and user_b, and friendships/create requires just one, either screen_name or user_id.

$follows_faelazo = $twitteroauth->get('friendships/exists', array('user_a' => $_SESSION['username'], 'user_b' => 'faelazo'));
if(!$follows_faelazo){
    echo 'You are NOT following @faelazo!';
    $twitteroauth->post('friendships/create', array('screen_name' => 'faelazo'));
}

You can unfollow an user with basically the same code that creates a follow, just replace create with destroy:

$follows_faelazo = $twitteroauth->get('friendships/exists', array('user_a' => $_SESSION['username'], 'user_b' => 'faelazo'));
if($follows_faelazo){
    echo 'You are following @faelazo! Proceed to unfollow...';
    $twitteroauth->post('friendships/destroy', array('screen_name' => 'faelazo'));
}

Step 5: Posting Updates

This is probably the most interesting section, since it’s Twitter’s core: posting an update, as you might have imagined, is pretty straightforward. The path is statuses/update, the method is POST (since we are not reading), and the one required argument is status.

$twitteroauth->post('statuses/update', array('status' => 'Hello Nettuts+'));

Now go to your Twitter profile page and you’ll see your tweet.

Let’s retweet @Nettuts’ update announcing the HTML 5 Competition; the status id is 19706871538 and the reference tells us that the path is statuses/retweet/:id, where the :id part is the status id we will be retweeting. The method is POST and it doesn’t require additional parameters.

$twitteroauth->post('statuses/retweet/19706871538');

To delete a tweet, you’ll have to pass the status id you’ll be destroying in the first parameter, just like retweeting. If the tweet’s id is 123456789, the code to destroy will be.

$twitteroauth->post('statuses/destroy/123456789');

Of course, this code can only delete tweets made by the authenticated user.


Conclusions

Twitter’s API is quite easy to understand; it’s far more documented than even Facebook’s (even though Facebook offers an in-house library). Unfortunately, the authentication is not as smooth as we might hope, depending on session data.

One thing worth noticing is that, once a Twitter user has been authorized (assuming the app has read and write permissions), you have plenty of control over this account. If you change something on behalf of the user without his permission, you’ll create trouble. Use it with caution!

The API changes coming to Twitter will deny basic authentication; Twitter is focusing on ceasing the countless scams that trick users into giving up their login credentials. OAuth is the solution; and, if you’ve worked through the Facebook Connect tutorial, you can now provide your website or app users with a quick login without credentials, using your choice of the two most used social networks. How cool is that?