Jenkins Continuous Integration server on Raspberry Pi

From a fresh install of Raspbian “wheezy” image running headless (with no desktop) : 

$ cat /etc/debian_version
wheezy/sid
$ uname -a
Linux raspberrypi2 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l GNU/Linux

• Default Pi speed is slow for Jenkins so we need to overclock

$ sudo raspi-config

• Select 'Overclock -> Modest 800MHz ARM, 250MHz core, 400MHz SDRAM, 0 overvolt -> OK -> Reboot when selected'

 

• Post reboot check the frequency is 800 MHz :

$ vcgencmd get_config arm_freq
arm_freq=800

• Check default port of 8080 is free

$ netstat -an | grep -i 8080
<- nothing should show as bound

•  Install OpenJDK Java 6 runtime

$ sudo apt-get update
$ sudo apt-get install -y openjdk-6-jre

• Check Java version

$ java -version
 java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.4) (6b27-1.12.4-1+rpi1)
OpenJDK Zero VM (build 20.0-b12, mixed mode)

•  Install Jenkins

$ sudo apt-get install -y jenkins

• Post install check port 8080 is now bound for all IP's: 

$ netstat -an | grep -i 8080
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN  

• Open in GUI : http://< Raspberry Pi IP>:8080/

Note - Initial load of GUI make take a few minutes with high CPU but this is fine once it loads

• To monitor peformance htop is a good tool :

 $ sudo apt-get install -y htop

Arch Linux Raspberry Pi Monitoring Webcam Setup

The following method describes how to set-up a webcam that can be used on Arch Linux. Its useful to already have :

• NTP configured and running
• Configured correct timezone e.g.

$ sudo ln -s /usr/share/zoneinfo/America/Toronto /etc/localtime

• Update the latest packages

$ sudo pacman -Syu 

Webcam Setup

• Connect the webcam to the USB port
• You should see it listed under :

 $ sudo lsusb
Bus 001 Device 004: ID 19ff:0102 Dynex 1.3MP Webcam

•  If the camera is not listed check dmesg

$ dmesg | tail
[    5.401492] uvcvideo: Found UVC 1.00 device Dynex 1.3MP Webcam (19ff:0102)
[    5.414733] input: Dynex 1.3MP Webcam as /devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3:1.0/input/input0
[    5.414990] usbcore: registered new interface driver uvcvideo

Install Motion

• Motion is a software motion detector which grabs images from video4linux devices and/or from webcams

$ pacman -S motion

• This will install ~116MB of sotfware and dependacies 

Install ffmpeg

• Ffmpeg is a complete and free Internet live audio and video broadcasting solution for Linux

$ pacman -S ffmpeg

• This will install ~30MB of sotfware and dependacies 

Create log

• Create the log file under /var/log

$ sudo touch /var/log/motion.log
$ sudo chmod 777 /var/log/motion.log

Motion Configuration

• Backup existing config file :

$ sudo cp /etc/motion/motion.conf /etc/motion/motion.backup 

• Change /etc/motion/motion.conf file with the following as an example :

daemon off
minimum_frame_time 0
pre_capture 0
post_capture 0
output_normal on
quality 75
ffmpeg_cap_new off
ffmpeg_timelapse 0
ffmpeg_variable_bitrate 0
target_dir /data/webcam/motion
webcam_port 8081
control_port 8080
webcam_localhost off
width 320
height 240

• target_dir : target base directory for pictures 
• Create target base directory for pictures

$ sudo mkdir -p /data/webcam/motion

• Start motion and go to port 8081 of your local IP and the web cam should be working :

$ sudo motion &

Startup Script

• Example Startup script

$ cat StartWebCamMonitor.sh
#!/bin/bash
LOG=/var/log/motion.log
sudo echo "Start Motion Script" `date` >> /var/log/motion.log
sudo /usr/bin/motion 2>&1 | tee -a $LOG &

Shutdown Script

• Example Shutdown script

$ cat StopWebCamMonitor.sh
#!/bin/bash
LOG=/var/log/motion.log
# Script details
sudo echo "Stop Motion Script" `date` >> /var/log/motion.log
sudo pkill motion 2>&1 | tee -a $LOG

 

Clean Up Script

• Example Clean up script

$ cat CleanUpWebCamMonitor.sh
#!/bin/bash
> /var/log/motion.log
sudo rm -rf /data/webcam/motion/
sudo mkdir /data/webcam/motion/

 Schedule WebCam to Run daily

• Use 'crontab -e' to update when you want the scripts to run

$ crontab -l
00 07 * * 1-5 ~/bin/StartWebCamMonitor.sh > /dev/null 2>&1 &
00 19 * * 1-5 ~/bin/StopWebCamMonitor.sh > /dev/null 2>&1 &
20 19 * * 1-5 ~/bin/CleanUpWebCamMonitor.sh > /dev/null 2>&1 &

Scheduled RSS Podcast Downloader for Raspberry Pi

I have a number of RSS podcasts that I frequently listen to. I can listen to them on a number of devices like my phone or tablet. My requirement was a setup where during the night the podcasts would be downloaded to the Raspberry Pi so I could then quickly download them to my phone or tablet.

I looked at a number of solutions but the one I liked the best was BashPodder which essentially is a small bash script that download the file. It conforms to the KISS rule (Keep It Simple Stupid). Its easy to set-up, configure and set a cron job to run.

Set-up

• There are three files :
• bashpodder.shell - The main program
• parse_enclosure.xsl - xsl style sheet
• bp.conf - podcast config list

• Once you download these to your Pi you can edit the 'bp.conf' file for the RSS feeds you want. They are a number of example podcasts in this file already but you can edit as necessary. 
• I removed all and added two e.g.

$ cat bp.conf    
http://www.howstuffworks.com/podcasts/stuff-you-missed-in-history-class.rss
http://feeds.feedburner.com/JupiterBroadcastingVideos

• After that its simple to run the downloader :

$ cd /data/podcasts/bashpodder/ && ./bashpodder.shell

• The podcats will be downloaded to a date stamped folder in the directory the script was run :

$ ls -lh 2013-10-20/
total 206M
-rw-r--r-- 1 cwishaw users 206M Oct 14 12:23 cr-0071-432p.mp4

• The name of the folder can be changed if required in 'bashpodder.shell'

• Note - The first time you run the ./bashpodder.shell it will download all the podcasts which may be massive. One way to avoid this is to update the 'podcast.log' file which is a list of the files that have been downloaded by BashPodder.

$ tail -n 2 podcast.log
http://www.podtrac.com/pts/redirect.mp3/podcasts.howstuffworks.com/hsw/podcasts/symhc/2013-10-14-symhc-elsa-lanchester-1.mp3
http://www.podtrac.com/pts/redirect.mp3/podcasts.howstuffworks.com/hsw/podcasts/symhc/2013-10-16-symhc-elsa-lanchester-2.mp3

• To create the list you can use Python. For example on Arch :

$ pacman -S python-pip
$ sudo pip install feedparser

• Run wget to obtain the RSS file e.g.

$ wget http://www.howstuffworks.com/podcasts/stuff-you-missed-in-history-class.rss

• Create a python script :

#!/usr/bin/python
import feedparser

RSSFEED = '~/tmp/stuff-you-missed-in-history-class.rss'
OUTPUTFILE = '~/tmp/RSSFeedParser.tmp'

## End of Configuration

d = feedparser.parse(RSSFEED)

text_file = open (OUTPUTFILE, "w")

for post in d.entries:
    #text_file.write(post.title + ": " + post.link + "\n")
    text_file.write(post.link + "\n")
   
text_file.close()

text_file = open (OUTPUTFILE, "r")
print (text_file.read())
text_file.close()

• You can then append the file :

$ cat RSSFeedParser.tmp >> podcast.log 

• Final Task is to set up a cron job to run at 2 AM every morning :

$ crontab -e

00 02 * * * cd /data/podcasts/bashpodder/ && ./bashpodder.shell > /dev/null 2>&1 &

• On Android I use an app called 'Turbo Client' to download the podcasts locally via SFTP.

Arch Linux on Raspberry Pi

After only using Raspbian on my Pi I wanted to try a different OS and as i've heard a lot of good things about Arch Linux decided to give it a try. I am using an SD card for the image.

Image Setup from Linux PC

• Download the latest Arch Linux ARM image
• Verify the file downloaded

 $ sha1sum archlinux-hf-2013-07-22.img.zip
e273d680e89ca9f79c2252c0b00ccc3084cd9b14  archlinux-hf-2013-07-22.img.zip

• The long string can be compared with the SHA-1 given on the page the image was downloaded from
• Extract image (img file will be ~ 1.9GB)

$ unzip archlinux-hf-2013-07-22.img.zip

• Run command 

$ df -h

• Connect SD card to your computer via dedicated port or SD card reader
• Run command

$ df -h 

• A new device will have appeared and this is the SD card to apply the image to shown in the left most column e.g.

 /dev/sdb1        56M   17M   40M  30% /media/cwishaw/3312-932F

• For /dev/sdb1 the 1 refers to the partition number. We will format the whole disk e.g. /dev/sdb
• Umount the device

$ sudo umount /dev/sdb1

• Format SD card
• bs = block size of 4MB 
• if = image file downloaded and unziped
• of = device name (important to get correct)

$ sudo dd bs=4M if=archlinux-hf-2013-07-22.img of=/dev/sdb

•  Once format is complete (it may take a while) insert SD card into Raspberry Pi

Arch Linux on Pi

• By default an IP will be obtained from DHCP and SSHD is enabled
• Username : root
• Password : root
• The Pacman Rosetta guide is a great resource for Software Installations on Arch Linux

glances - a simple but cool monitoring tool

I always am on the lookout for tools to help with monitoring a diagnosing issues. The best tools I find are those that quickly show you the information you need to know in an easy to read manner. I just discovered the excellent tool glances which is a a cross-platform curses-based monitoring tool. It shows a lot of useful information but in a manner that is easy to read. It can even be used in client/server mode!

Information shown includes :

• Disk Usage per mount point
• Disk I/O
• Network Rx/s + Tx/s
• Memory Usage
• CPU Usage
• Process
• Uptime

I really like that it has color codes for state :

• Green : OK
• Blue : Careful
• Magenta : Warning
• Red : Critical

Installation

Arch Linux

• From standard Arch or Raspberry Pi based ARM :

$ pacman -S glances

Ubuntu (13.04+)

• From standard apt-get :

$sudo apt-get -y install glances


Debian/Ubuntu (pre 13.04)

•  From python's 'pip'

$ sudo apt-get install python-pip python-dev

$ sudo pip install glances

CentOS/Red Hat

• Configure the EPEL repository
• Then install via yum

# yum -y install glances

Refresh Time

• The default is 3 seconds but this can easily be changed e.g.

$ glances -t 10

Interactive Commands

• a : Sort process list automatically
• b : Switch between bit/s or Byte/s for network I/O
• c : Sort processes by CPU usage
• d : Show/hide disk I/O stats
• f : Show/hide file system stats
• h : Show/hide the help screen
• i : Sort processes by I/O rate
• l : Show/hide log messages
• m : Sort processes by MEM usage
• n : Show/hide network stats
• p : Sort processes by name
• q : Quit
• s : Show/hide sensors stats (only available with -e flag)
• t : View network I/O as combination
• u : View cumulative network I/O
• w : Delete finished warning log messages
• x : Delete finished warning and critical log messages
• y : Show/hide hddtemp stats (only available with -y flag)
• 1 : Switch between global CPU and per-CPU stats
   

 Client/Server mode

• In the server :

$ glances -s
Glances server is running on 0.0.0.0:61209

• On Client

 $ glances -c @server

• The footer will display the connection information e.g. Connected to @pi is shown below