Tracking AT&T Data Plan Use With PhantomJS / Part 2

by | Apr 8, 2015 | How To |

Tracking AT&T Data Plan Use With PhantomJS / Part 2

In Part 1 I covered the the framework needed to tracking my family data plan use. In the post I will cover the specific script I created. Once again, you will need to get PhantomJS on your Raspberry Pi see this post for the instructions.

The first thing I needed to do was add the command line parameters. The AT&T site uses a username, password and a security ID to sign in. I changed the framework parameters section to this:

if (system.args.length < 4) {
   console.log('Usage: att.js user pw code');
   console.log('Example: att.js myUser mip@ssw0rd 1234');
   phantom.exit(1);
} else {
   userID = system.args[1];
   userPW = system.args[2];
   userCode = system.args[3];
}

Next, I created the steps to sign in and then get the data. As stated above, the sign in process has several steps. First, I needed to provide the username and password, then I provide the security ID and last I get the data plan report.

Step 1 to 3 – Load the AT&T page and submit the username and password

First, here is the code placed in the steps[] array.

  function() {
    //Load Login Page
    page.open("https://www.att.com");
  },
  function() {
    //Enter search string
    page.evaluate(function(user,pw) {
      //console.log('sign in with ' + user + ' ' + pw);
      jQuery('form#ssoLoginForm input#userid').val(user);
      jQuery('form#ssoLoginForm input#userPassword').val(pw);
    }, userID, userPW);
  }, 
  function() {
    //Login
    page.evaluate(function() {
      jQuery('form#ssoLoginForm').submit(); 
      console.log('submit...');
    });
  },

The first function simply opens the AT&T site. The second function uses page.evaluate and jQuery to complete the username & password form. page.evaluate is a PhantomJS method to sandbox a web page and perform operations on that page. The last step simply submits the form.

Steps 4 & 5 – Submitting the passcode

NOTE: AT&T changed the login process for some accounts. If you do not need to submit the passcode, these steps should be removed from the script.

The AT&T passcode form required me to get a little creative. The main problem is that the site sometimes responded slowly and I needed to check to see if the passcode form was rendered before I filled in the information. To see if the form is rendered, I query the DOM looking for the form input element. If it is missing, I backup the step counter and wait for the next loop to try again. It it is there I complete the form in step 4 and submit it in step 5.

function() {
    var result, myLoop, goNow = false, formGood = false;
    
    // waitfor the passcode form to appear
    result = page.evaluate(
        function () {
            return document.querySelector( "form input#passcode" );
        });

    // Cheat - stay on this step until the form is complete
    if (!result) {
        testindex--;
    } else {
        page.evaluate(function(ucode) {
            console.log('sign in with ' + ucode);
            jQuery('form input#passcode').val(ucode);
            console.log('passcode...');
        }, userCode);
    }
  }, 
  function() {
    //Login
    page.evaluate(function() {
      jQuery('form[name="passcodeActionForm"]').submit(); 
      //$('form#ssoLoginForm input#tguardLoginButton').click(); 
      //$('form[name="search"] input#searchBtn').click();
      console.log('submit...');
    });
  },

Steps 6 & 7 – Getting the results

At this point PhantomJS is sign into the AT&T site. I could manage my whole account but all I wanted was to report the data use. Step 6 browses to the report page and step 7 finds the elements in the table and dumps it out.

  function() {
    //Load report page
    page.open("https://www.att.com/olam/showWirelessDashboardAction.myworld");
  },
  function() {
    // Dump results
    page.evaluate(function() {
        jQuery(".phoneItemContainerMid ul li").each( function()
                   { console.log(jQuery(this).find("#sdgCtn").text() + ":" +  jQuery(this).find("#sdgUsage .strong").text().replace(/\s+/g, ' ').trim()); });
      console.log('should be there...');
    });
  },

There are several phones on my data plan and i used jQuery.each to loop through every item and dump the phone number and the data use for that phone.

Running the script and the output

Copy the script to the PhantomJS directory and start it with this command. Be sure to replace the account number, password and security ID with your own information.

./phantomjs att.js 4845551212 password 1234

The script will run and give output similar to this. Not, the error messages are coming from javascript on the AT&T page, not from the PhantomJS script. The output is seen at the end.

step 1
ReferenceError: Can't find variable: ddo
https://www.att.com/media/cms/homepage/scripts/combined-top.min.js:49
https://www.att.com/media/cms/homepage/scripts/combined-top.min.js:49
https://www.att.com/media/cms/homepage/scripts/combined-top.min.js:49
ReferenceError: Can't find variable: jQuery
https://pixel.mathtag.com/event/js?01AD=3iB4ZEtFk2_xV_a3UQSJy4iu1IoZUoqCd
u0_cFB3vUarsJMw&01RI=5E22864BD7&01NA=&mt_id=1038&mm_bnc&mm_bct:43
step 2
step 3
SyntaxError: Parse error
ReferenceError: Can't find variable: jQuery
  https://pixel.mathtag.com/event/js?mt_id=108:43
step 4
step 5
step 6
step 7
484-555-1212:1.4
484-955-1212:0.3
484-559-1212:2.0
610-555-1212:0.1

That’s it

Getting the data was a little more complicated than my other web scraping scripts and it isn’t the fastest thing but I check once a day and then send myself an email if the use goes above 75% for the month. I have started to look into other sites I want to track such and my electric use and this script can be modified to get reports from that site too.

The complete steps[] array

var steps = [
  function() {
    //Load Login Page
    page.open("https://www.att.com");
  },
  function() {
    //Enter search string
    page.evaluate(function(user,pw) {
      console.log('sign in with ' + user + ' ' + pw);
      jQuery('form#ssoLoginForm input#userid').val(user);
      jQuery('form#ssoLoginForm input#userPassword').val(pw);
    }, userID, userPW);
  }, 
  function() {
    //Login
    page.evaluate(function() {
      jQuery('form#ssoLoginForm').submit(); 
      console.log('submit...');
    });
  }, 
  function() {
    var result, myLoop, goNow = false, formGood = false;
    
    // waitfor the passcode form to appear
    result = page.evaluate(
        function () {
            return document.querySelector( "form input#passcode" );
        });

    // Cheat - stay on this step until the form is complete
    if (!result) {
        testindex--;
    } else {
        page.evaluate(function(ucode) {
            console.log('sign in with ' + ucode);
            jQuery('form input#passcode').val(ucode);
            console.log('passcode...');
        }, userCode);
    }
  }, 
  function() {
    //Login
    page.evaluate(function() {
      jQuery('form[name="passcodeActionForm"]').submit(); 
      //$('form#ssoLoginForm input#tguardLoginButton').click(); 
      //$('form[name="search"] input#searchBtn').click();
      console.log('submit...');
    });
  }, 
  function() {
    //Load report page
    page.open("https://www.att.com/olam/showWirelessDashboardAction.myworld");
  },
  function() {
    // Dump results
    page.evaluate(function() {
        jQuery(".phoneItemContainerMid ul li").each(function() { console.log(jQuery(this).find("#sdgCtn").text() + ":" +  jQuery(this).find("#sdgUsage .strong").text().replace(/\s+/g, ' ').trim()); });
      console.log('should be there...');
    });
  }, 
];