Moving from NOOBS to a permanent filesystem and SD card

For my Raspberry Pi on my workbench I have a large SD card with NOOBS installed. This gives me the ability to switch between different operating systems without any trouble. But I recently finished a project and had to move the OS from the NOOBS SD to a permanent home on a smaller SD card. Basically, I needed to move the OS created by NOOBS to a new SD card. I did find some instructions in the forums but there are a few extra steps for Arch Linux to make the process smooth. BTW – all these commands need to be run as a superuser. 1. To start, trim out any extra packages you may have installed but do not need. This can be a pain so I focused on the biggest packages. Ironically, I had to installed the expac package (pacman -S expac) to list the packages by size. This command with list all packages with no dependencies and their sizes. It excludes the packages in the base and base-devel groups. expac -HM "%011m\t%-20n\t%10d" $( comm -23 <(pacman -Qqen|sort) <(pacman -Qqg base base-devel|sort) ) | sort -n 2. Pacman keeps a copy of every package downloaded in a cache. If your Arch Linux Raspberry Pi was installed a while ago, the pacman cache may be getting large with older versions. When your install is stable, you can erase the old, no-longer needed packages from the cache with this command: pacman -Sc On my system, these two steps freed almost 1GB. 3. I put the new, smaller SD card in a card-reader in the USB port. It needs to...
Tracking AT&T Data Plan Use With PhantomJS / Part 2

Tracking AT&T Data Plan Use With PhantomJS / Part 2

In Part 1 I covered the the framework needed to tracking my family data plan use. In the post I will cover the specific script I created. Once again, you will need to get PhantomJS on your Raspberry Pi see this post for the instructions. The first thing I needed to do was add the command line parameters. The AT&T site uses a username, password and a security ID to sign in. I changed the framework parameters section to this: if (system.args.length < 4) { console.log('Usage: att.js user pw code'); console.log('Example: att.js myUser mip@ssw0rd 1234'); phantom.exit(1); } else { userID = system.args[1]; userPW = system.args[2]; userCode = system.args[3]; } Next, I created the steps to sign in and then get the data. As stated above, the sign in process has several steps. First, I needed to provide the username and password, then I provide the security ID and last I get the data plan report. Step 1 to 3 – Load the AT&T page and submit the username and password First, here is the code placed in the steps[] array. function() { //Load Login Page page.open("https://www.att.com"); }, function() { //Enter search string page.evaluate(function(user,pw) { //console.log('sign in with ' + user + ' ' + pw); jQuery('form#ssoLoginForm input#userid').val(user); jQuery('form#ssoLoginForm input#userPassword').val(pw); }, userID, userPW); }, function() { //Login page.evaluate(function() { jQuery('form#ssoLoginForm').submit(); console.log('submit...'); }); }, The first function simply opens the AT&T site. The second function uses page.evaluate and jQuery to complete the username & password form. page.evaluate is a PhantomJS method to sandbox a web page and perform operations on that page. The last step simply submits the form. Steps 4...
Tracking AT&T Data Plan Use With PhantomJS / Part 1

Tracking AT&T Data Plan Use With PhantomJS / Part 1

When my family moved to the cheaper AT&T Shared Data Plan we saved hundreds of dollars every year. But, now we had to better manage the data use because it was shared by 4 people. Most months we don’t have a problem but I would like to know when we are over budget early enough to tell everyone to conserve. No Problem, I thought. I will use my Raspberry Pi to scrape the AT&T web pages and send me an alert when the use is too high. I already use my RPi to gather information like the outside temperature and stock quotes and this shouldn’t be any different. I was wrong. The AT&T site makes heavy use of javascript and jQuery and I was not able to the simple methods I used in the past. That is where PhantomJS comes into the picture. PhantomJS if a full webkit browser that can be scripted with javascript. So it renders all the jQuery goodness AT&T puts on the page AND I can create a simple automation script to scrape the data. In Part 1, I will cover the automation framework I created. PhantomJS has a full testing framework called CasperJS but I though that was overkill for my project. I just needed to step through a few pages and grab some content. In Part 2 I will cover scraping the AT&T site and some of the unique steps I had to take to script the site. Step 0 – Get PhantomJS If you need PhantomJS, see this post on how to compile it on the Raspberry Pi. The Basic Loop The...
How To Compile PhantomJS on the Raspberry Pi 2

How To Compile PhantomJS on the Raspberry Pi 2

PhantomJS is an awesome tool for web testing, web scraping and complex scripting. In a future post I will be talking about how to use PhantomJS to check and report on cell plan use. Because the Raspberry Pi 2 is new, no one has created a PhantomJS repository entry so before I could use it on my Raspberry Pi 2 I had to compile it from scratch. The PhantomJS team has done a super job automating most of the process (check out the instructions here) but a few tweaks are needed for the RPi2. Some Groundwork To start, PhantomJS is a big project. On the Raspberry Pi 1 it takes 2 days to compile. The RPi2 is faster and doing some preliminary work will reduce the compile time to a few hours. To start, don’t use a SD drive or a flash drive except for the initial boot. Move your main OS to a USB hard drive or SSD drive. See these instructions. This will make a huge difference since the compile process is disk intensive. Next, you will need swap space. The Pi2 has 1GB of ram but it isn’t enough. One compile step takes about 1.1GB and it crashes without swap. Follow these instructions. Required Packages If you are running Raspian install these packages: sudo apt-get install build-essential g++ flex bison gperf ruby perl \ libsqlite3-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev \ libpng-dev libjpeg-dev On Arch Linux you will need to use pacman to get these packages: pacman -S gcc make flex bison gperf ruby openssl fontconfig \ sqlite libpng libjpeg git python perl Download the Code and...
My Raspberry Pi 2 arrived

My Raspberry Pi 2 arrived

I was not part of the inner circle that was able to preview the Raspberry Pi 2 and my order finally arrived. My first impression is that NOOBS does not have Arch Linux for the Pi2. So I installed Arch based on the simple instructions found here. The result – a perfectly working, faster Arch RPi. More to come on this over the next few...
Adding swap to the Raspberry Pi

Adding swap to the Raspberry Pi

Recently I was working on a project that required more RAM than my Raspberry Pi has. After all, the model B only shipped with 512MB and part of that needs to be allocated to the video card. But Linux has a method of dealing with running out of memory called the swap. The swap is a file on disk that serves as ‘overflow’ RAM space. It is slower than regular RAM but is a better solution than simply having the program crash. At the time I was using Arch Linux which doesn’t setup any swap by default. Raspbian does setup swap space but doesn’t do it in a recommended manner. Let me put this right up front. Using swap space on SD cards and USB flash drives is not a good idea. It creates more problems than it solves. To start, it will be too slow to be useful. Second, and more importantly, Linux swapping can cause many write cycles which will tax your SD/flash drive quickly causing failures. I only use swap when I have a USB hard disk. A portable hard disk solves the speed and longevity problems quite well. I happened to have a old 6GB drive in my parts bin that I put into a portable enclosure. The Basics First, check how much memory you have with free -h (the -h means human readable). You should see something like: total used free shared buff/cache available Mem: 371M 16M 300M 340K 54M 286M Swap: 0B 0B 0B Meaning the board has 371MB of available RAM and no swap. I needed about 400MB of RAM and the...
The TinyCore distribution is now available on the Raspberry Pi

The TinyCore distribution is now available on the Raspberry Pi

For advanced users – If you need a minimal distribution that includes a GUI and run entirely out of RAM, TinyCore might be right for you. It includes only a minimal set of tools that you can use to build up a complete system with only what you need. The complete announcement can be found here http://forum.tinycorelinux.net/index.php/topic,16551.msg98689.html and a review is here http://www.linuxuser.co.uk/reviews/picore-5-1-review-tinycore-comes-to-raspberry-pi. I will say that I do not entirely agree with the review and I give TinyCore higher marks than the reviewer. UPDATE TinyCore v6.0 is now available...
Stack Exchange now has a Raspberry Pi site

Stack Exchange now has a Raspberry Pi site

Stack Exchange, the incredibility popular question and answer site now has a board dedicated to the Raspberry Pi. Check it out http://raspberrypi.stackexchange.com/. On this site (and every Stack Exchange site) users post questions and answers. The best answer get voted to the top. There is a sophisticated reputation system to keep spam and worthless exchange to a minimum. So the site always has high quality and fresh...
Emulating Arch Linux For the Raspberry Pi in QEMU

Emulating Arch Linux For the Raspberry Pi in QEMU

In my last post about QEMU I showed how easy it was to emulate the Raspberry Pi in Windows. If you are just getting started with the Pi, that is the way to go. Personally, I prefer to use Arch Linux over Debian. There is nothing wrong with Raspbian, I am just used to how Arch Linux works. But, emulating Arch Linux under QEMU is not that simple. To start, not everything can be found in one easy download package. But, I worked through the problems and here are the instructions: To start,you will need a working QEMU working Rasbian to modify the Arch Linux img file. Follow the instructions in this post to get Raspbian running. Next, download QEMU 1.6 from QEMU FOR WINDOWS (52MB). Extract in a different location from the Rasbian install. Download the latest Linux kernel for QEMU from http://xecdesign.com/downloads/linux-qemu/kernel-qemu NOTE: http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/ has some great information about QEMU and the Raspberry Pi Download the latest Arch image from http://archlinuxarm.org/platforms/armv6/raspberry-pi Before we boot the Arch image we are going to make a few changes to the img file. One, it needs to be bigger and have a swap partition and two, the way the Arch image specifies the boot partition does not work with QEMU. We need to modify the /etc/fstab to make it work. Add 2 GB of space to the fake SD card with the following command (in Windows) qemu-img <Arch img filename> resize +2G Next, we will add the Arch img as a second drive in Rasbian. Add the following to the BAT file that starts the Rasbian QEMU session -hdb ArchLinuxARM-20##.##-rpi.img Add...