Building My Linux Box…The Plan

I concluded it was time to retire my current PC (a Dell Dimension XPS 400, Intel Pentium D 820, 2.8GHz w/Dual Core Technology, 2G of memory, purchased in January 2006 for  ~ $2,500) and replace it with something better, aka faster.  It’s performing poorly, but most of all, I want to do some experimenting with iPythonmatplotlib, as well as, GNU Radio & digital signal processing.  To do this justice, I need a faster box and ideally loaded with Linux and X Windows.

I also think its time for me to build my own box, instead of purchase it already packaged and assembled.  They say you get more for your dollar, or at least you can invest the same money into things that will make a better system (instead of some more crappy speakers, mice, keyboards, etc.).  Given I’m building it, I want to pay special attention to getting the performance up.  I don’t need the top of the line CPU (I don’t believe you get sufficient value for your money), but I would invest in improving the predominate enemy of computer performance, that is I/O.  Also, I want the graphics to be fast and smooth.  The work on matplotlib and signal processing is likely to be graphically intensive.  I’m not going to be playing games on the system, but I’m going to keep an eye on the gaming communities hardware preferences.   I’m willing to go with a good (but not super, and therefore expensive) graphics board.

A fancy sound system isn’t a priority for me.  While my interest in GNU Radio & digital signal processing may have some use for a good sound system, I think I could live with the integrated sound system that will come on the motherboard.  Like graphics, if  it proves unacceptable, I’ll make the investment on another occasion.

Since I’m buying components and assembling it, I want to do some cherry picking.  When a vendor sells you an assembled box, they are often using cheaper / less functional components to save them cost.  I’m picking an Intel Core i5 CPU (I believe it has the best value for me).  Also, I’m going to get a -K Series Intel CPU and motherboard designed for overclocking and provide optimal performance for gamers and high-power users.  While I’m not doing gaming, I like the option to tuning the hardware and I have found the -K Series to be only slightly more expensive.  For a small amount of additional cash, I get some cool capabilities.

To get the better I/O that I want, I’m going to purchase one solid-state drive (SSD) from which I plan to boot Linux.  SSD’s are very much more expensive byte-for-byte when compared to a standard hard drive but boy they can fly!  Faster CPU and faster drives are my most valuable investment.  With the  financial investment in the SSD, I’m not going to purchase new hard drives but reuse the ones I have in my current PC (I’ll need them since the SSD I can budget for will not be large).  They are newer then the original Dell box, and besides  the hard drive has over 6 years of stuff on it, including a MS Windows environment that I have grown dependent on.  That brings me to the next point.

I want to dual boot the system with both MS Windows and Linux.  The Linux will be on the SSD, and it is here that I plan to spend much of my time.  I want to pick a Linux distribution that is well supported, popular, and a good graphic desktop environment.   I grew up on the Linux command line (then it was Unix) and I feel at home there, but I would like to try out the X Windows desktop.  If at all possible  I want the MS Windows to boot from my current hard drive.  I don’t want to have to reload software or copy an image … you never can get it to be the same again.  If I must, I’ll make the MS windows drive the default boot as it is now (and Microsoft seems to insist on being default).  If I can do this, moving into my new PC will be painless.

With my objectives and priorities fully articulated, lets explore what I intend to build.

Central Processing Unit (CPU)
For my CPU I have picked the Intel Core i5-3570K Ivy Bridge 3.4GHz (3.8GHz Turbo Boost and 6M Cache) LGA 1155 77W Quad-Core Desktop Processor which includes Virtualization Technology (VT-x), and Intel HD Graphics 4000.  What a mouth full …. here is what this all means:intel core i5 3570k with stock cooler

  • Intel Core - Intel Core (sometimes refereed to as Core 2) is a brand name that Intel uses for various mid-range to high-end consumer and business microprocessors. In general, processors sold as Core are more powerful variants of the same processors marketed as entry-level Celeron and Pentium. Similarly, identical or more capable versions of Core processors are also sold as Xeon processors for the server and workstation market.
  • i5-3570K – The very first question after you concluded that your going with the Intel Core line, is Core i3 vs. i5 vs. i7 – Which one is right for me?   I’m driven to the i5 since the current Intel Core i5 models are generally considered the best price/performance choice for a gaming system and the i7  do NOT have built-in graphics capability (this would force me to buy a graphics card). Why K versions you ask? Well, the default Ivy Bridge processors are much harder to overclock, where the K series are unlocked and come with tools for overclocking.  This processor has overclocked test results running at a stable 4.7GHz.
  • Ivy Bridge -  Ivy Bridge is the latest generation of processors within the Intel Tick-Tock Development Model.   Intel introduced its Sandy Bridge desktop and laptop processors at the start of 2011 as there new micro-architecture …the tock.  Intel introduced Ivy Bridge in April 2012 new processor technology … the tick.  There is a school of thought that says you shouldn’t buy the last generation CPU because you can get more value (aka performance/dollar) from the previous generation.  But the prices I have seen and the comparative reviews have given me the courage to go with the latest generation.  Also check out this review with a description of Ivy Bridge.
  • Turbo Boost - This feature increases performance of both multi-threaded and single-threaded workloads.  Intel Turbo Boost Technology 2.0 allows the processor core to opportunistically and automatically run faster than its rated operating frequency/graphic render clock if it is operating below power, temperature, and current limits. It can boost the frequency up to 3.8GHz.
  • 6M Cache – This refers to cache used by the central processing unit of a computer to reduce the average time to access memory. The cache is a smaller, faster memory which stores copies of the data from the most frequently used main memory locations.  It comes it three types: L1, L2, and L3. L1 cache (sometimes called primary cashe) is the fastest cache and it usually comes within the processor chip itself.  L2 cache comes between L1 and RAM (processor-L1-L2-RAM) and is bigger than the primary cache. The L1 and L2 cache is per core but the last-level cache (L3), is shared among all cores and sometimes call Smart Cache since cache can be dynamically assigned to a core as it needs it. The “6M” refers to the number of bytes of data that the L3 cache can hold.
  • LGA 1155 - LGA 1155, also called Socket H2, is an Intel microprocessor compatible 1155 pin socket which supports Intel Sandy Bridge and Ivy Bridge microprocessors.
  • 77W - This is the  Thermal Design Power (TDP) is the maximum power consumed by the CPU under normal / regular use. In other words, the TDP is the max power a device will dissipate when running real applications. What’s more the TDP is given for graphics card default clocks.  TDP is a manufacturer’s data, and thanks to this information, CPUcooler manufactures can size their CPU coolers.
  • Quad-Core Desktop Processor - This is a multi-core processor (in fact a quad or 4 core)  computing component with four independent actual central processing units (called “cores”).  Intel manufacturers the four cores onto a single integrated circuit die (known as a chip multiprocessor or CMP), or onto multiple dies in a single chip package.  The proximity of multiple CPU cores on the same die allows it to operation at a much higher clock-rate than is possible if the signals have to travel off-chip.
  • Virtualization Technology – VT-x  (i.e. x86 virtualization or Vanderpool) is the facility that allows multiple operating systems to simultaneously share x86 processor resources in a safe and efficient manner, a facility generically known as hardware virtualization. With virtualization, you can have several operating systems running in parallel, each one with several programs running.  Each operating system runs on a “virtual machine”, i.e. each operating system thinks it is running on a completely independent computer. Note that on the Intel Core technology, the virtual machines are not assigned specific CPUs among the multiple CPUs but are shared by all.
  • Intel HD Graphics 4000 - Before the introduction of Intel HD Graphics, Intel integrated graphics were built into the motherboard’s northbridge.  HD Graphics 4000 is Intel’s 3rd generation of this integrated GPU technology.  The HD 4000 was completely redesigned and offers many improvement. The IPC (instructions per clock) can therefore be even 2x as fast  and overall up to 60% more performance should be possible.

One highly desirable feature missing from the Intel i5 line is Hyper-Threading Technology.  Hyper-Threading (HT) is a means for improving processor performance by supporting the execution of multiple threads (two is the current limit) on the same processor at once: the threads share the various on-chip execution units.  HT is available on the i7 line of processors but I just can’t justify the cost of this additional functionality.

Motherboard (Mobo)
I need to match the CPU to a -K Series motherboard and I picked the Intel Desktop Board DZ77GA-70K with Intel Z77 Express Chipset family which includes Intel High Definition AudioIntel Desktop Board DZ77GA-70K. The Intel high-definition audio chip allows you to use your computer to send digital audio signals to speakers, headphones, telephones and other audio equipment. Early computer audio systems could only produce simple stereo sound reproduction. The Intel HD audio system supports surround sound up to Dolby 7.1.  It supports the  usual 32GB of RAM, Smart Response TechnologySmart Connect Technology, Intel Rapid Storage Technology (Intel RST) for RAID 0, 1, 5, and 10. The “GA” in the motherboard’s name means that it contains Bluetooth/WiFi and the a front panel USB 3.0 module  (“GAL” means it has no Bluetooth/WiFi). The board supports Intel’s Fast Boot Technology which can dramatically reduce the time to boot the system. Support for high end graphics boards. It has eight  USB 3.0 ports (4 external/4 header), ten USB 2.0 ports (4 external (2 Hi-Current/Fast Charging) / 6 internal), four Serial ATA 6.0 Gb/s ports, one eSATA 6.0 Gb/s, four Serial ATA 3.0 Gb/s ports and many other features.

The Mobo comes with the Intel Visual BIOS graphical interface and animated controls, which allow you to configure settings faster and take full advantage of your Intel -K processors. The Visual BIOS uses the Unified Extensible Firmware Interface (UEFI) which defines a software interface between an operating system and Mobo firmware. UEFI is meant to replace the Basic Input/Output System (BIOS) firmware interface, present in all IBM PC-compatible personal computers.

The form factor for this Mobo is ATX.  ATX (Advanced Technology eXtended) is a motherboard form factor specification developed by Intel in 1995 to improve on previous de facto standards like the AT form factor.  There are different form factor of motherboards including microATX, Standard ATX and XL-ATX.  This is important to keep in mind when picking a case.

The selling feature for me was that its an Intel product (motherboards are new to me and I need the emotional support), seems easy to setup, it has gotten reasonable reviews (and here is another), and reasonable price.  It isn’t most feature full Mobo nor what a die-hard overclocker would buy but it seems a solid, stable product that will not give me any troubles or support problems and will perform well.

Memory (RAM)
According to the September 2011 Steam hardware survey, 4GB of RAM is currently the most popular configuration among gaming PCs. This may be sufficient for a light home user, however, many power users and enthusiasts find 4GB to be insufficient. Many recommend at least 6GB for any enthusiast PC, especially in light of the relativelyCorsair Vengeance low cost of memory.  I’m going with 8GB of RAM in an effort to assure good I/O and 12GB or 16GB just seems like more than I will need.  That amount of memory seems sufficient for you average gamer and should work for me.  And the reviews I have seen also claim they have successively overclocked this memory.

I also want to keep open the option to do some overclocking, so I need to consider memory based on Intel’s  Extreme Memory Profile (XMP).  I also want a memory provider with a solid reparation.  The Intel Core i5-3570K processor requires DDR3-1333/1600 memory.  With all this in mind, I choose the 8GB (2X4 GB) Corsair Vengeance Blue, 9-9-9-24, 1.5V  PC3-12800 1600MHz DDR3 240-Pin SDRAM Dual Channel Memory.   They are not top of the line memory but seem a good fit for my needs and have gotten solid reviews.

DDR3 or DDR3 SDRAM, an abbreviation for double data rate type three synchronous dynamic random access memory, is a modern kind of dynamic random access memory (DRAM) with a high bandwidth interface, and has been in use since 2007.  The primary benefit of DDR3  over its immediate predecessor (i.e. DDR2), is its ability to transfer data at twice the rate (eight times the speed of its internal memory arrays), enabling higher bandwidth or peak data rates. The next generation,  DDR4 expected to be released to the market sometime in 2013.  Its primary benefits compared to DDR3 is the higher range of clock frequencies (200MHz vs 166MHz) and data transfer rates (400MT/s vs 333MT/s).

DDR3 memory is classified according to the maximum speed at which it can work, as well as their timings. Timing are numbers such as 3-4-4-8, 5-5-5-15, 7-7-7-21, 9-9-9-24 where lower is better. Memory speed is specified via a number like this: DDR3-xxx / PC3-yyy or DDR3-xxx/yyy. The xxx number indicates the maximum clock speed that the memory chip supports. Therefore, DDR3-1333 can work up to 1,333MHz. Note this isn’t the real clock speed but twice that speed. So the real clock speed of DDR3-1333 is 666MHz. The yyy indicates the maximum transfer rate that the memory can reach. So memory labeled as DDR3-1333/10664 has a transfer rate of 10,664MB/s or 21,328 MB/s if they are running under dual channel mode. Most current boards have dual with the Intel socket 1336 has triple channel.

The memory timings x-x-x-x indicates the number of clock cycles that it takes for the memory to perform something. The smaller the number, the faster the memory. These set of four numerical parameters are called CL, tRCD, tRP, and tRAS. Sometimes there a fifth value which is voltage. Check out Understanding RAM Timings for more information.

Memory is sold in “kits” which are simply multiple single, similar (identical as possible) RAM modules packaged together. The intention is for them to be used in motherboards that have dual and triple (etc.) RAM channel capabilities.

Graphics Processing Unit (GPU)
I’m going to try and live with the on-board graphics processing unit (GPU) integrated with the CPU and invest that money elsewhere.   The reviews of  Intel’s newest integrated GPU that comes with the i5-3570K (HD Graphics 4000 or HD 4000) have been favorable (also see this).   This is the  third and latest generation of HD Graphics (now with 16 execution units) and appears to be a real contender to low end graphics cards.  If it proves less than acceptable, I’m invest in a graphics board another day.

Sound Card
Here again, I’m not buying a separate card but using the Intel High Definition Audio integrated into the motherboard.  Frankly, I’m not sure if this will limit my GNU Radio & digital signal processing objectives but I’ll take the risk.  If I’m unhappy for any reason, I’ll buy myself a sound board.

Solid State Drive (SSD)Samsung-SSD-840-Pro
I have chosen the Samsung Electronics 840 Pro Series 2.5-Inch 128GB SATA 6GB/s for my Solid State Drive (SSD).  Samsung has a great repatriation in this space and it has gotten solid reviews.

What most people who use SSDs do (and what I plan to do) is to buy one large enough to hold the OS and applications, and also buy a hard drive to hold the rest of your documents, photos, videos, etc. That’s a good compromise which puts the most performance-critical files on the fastest drive and has the cheapest cost-per-byte for your voluminous data files which typically have much lower performance requirements.  But keep in mind that as soon as the amount of data written reaches the stated capacity of the device, the write bandwidth immediately drops.  In fact, write bandwidth reduced by up to 70-80% once fully loaded with data and continued to operate under those conditions.  Therefore, don’t fill the SSD drive.

Also, one of the most publicized downsides of SSDs is that they have a limited number of writes before they wear out—however, with most newer SSDs, this isn’t actually a problem. Most modern SSDs will become outdated before they die, and you’ll probably have upgraded by then, so there’s not really a huge need to worry about writing to the drive too many times.

Because of the high speed of the SSD, your going to want to use the 6GB/s SATA ports on the motherboard.  The standard 3GB/s SATA ports don’t have the throughput, never the less, studies show the SSD still give you benefits.

Power Supply Unit (PSU)Corsair 650W PSU
I choose the popular  Corsair TX650 650-Watt Power Supply as my PSU. Most computers only consume around 150W, and even a high end computer might consume maybe 200W. That’s why most OEM computer manufacturers put small 250-350W PSUs in their systems. If you look at online reviews of highly overclocked systems with multiple video cards (SLI/Crossfire) they consume at most about 500-600W.   I don’t believe I’ll ever approach these levels so this PSU will give me much head room.  The review I have read seem to claim that he best way to take advantage of the TX650W’s quiet qualities is  to ensure that the PSU intake air does not exceed 30°C often, nor demand more than ~350W DC output.  I believe my usage will fit in this sweat spot.

Networking
Networking capabilities are built into the motherboard.  The Mobo comes with two Gigabit (10/100/1000 Mb/s) LAN subsystems using the Intel 82579V Gigabit Ethernet Controller. It also has a Bluetooth 2.1 & WiFi 802.11b/g/n module. There appears to have been some troubles with WiFi and Bluetooth module for DZ77GA-70K in 2012, but it has been reported to Intel and hopefully this has been worked out by now.  I’ll have to make sure I update the firmware on the board when I get it.

CD-ROM / DVD Drive / Hard Drives
I’m not going to worry about this now.  I anticipate loading all my software / data from the Web or transferring from my existing hard drives.  Also, I’ll reuse my existing hard drives in this box.

Cooler
The Intel Core i5-3570K comes with a stock cooler.  If do over clock the CPU, I’m likely to need a better cooler, but this is fine for now.

CaseCooler Master HAF 912 - Mid Tower Computer Case
Picking a case has been the hardest thing for me to select.  I guess this is because its not as much a technical decision but an aesthetic choice.  I have narrowed my choose to the Cooler Master HAF 912 – Mid Tower Computer Case with High Airflow Design (19.5 x 9.1 x 18.9 inches ; 17.8 pounds).  It has gotten good reviews with the main complaint being that it needs more fans (much room for more installation but only two are provided).  The front panel comes with the older USB 2.0 ports but the Mobo comes with a USB 3.0 panel that could be install if desired.  The case isn’t expensive but still has a sharp look and seems very versatile in its use and cooling.

Monitor
The Dell LCD monitor I presently have dates back to 2006 and isn’t equipped with HDMI, which is the only way to interface with the Mobo.  I presently use my monitor via its Digital Video Interface Digital (DVI-D), but it also has Video Graphics Array (VGA) and Composite Video inputs.  So if I wish to continue to use the monitor, I’ll need a converter of some type.  I found that the DVI-D to HDMI can be done via an inexpensive cable, so that is the way I’m going.  I specifically need a HDMI Type A to DVI-D Dual Link Male to Male cable.

Operating System (OS)
I plan to install Ubuntu Linux on the SSD drive.  Picking the Linux distribution was nearly as hard as picking the case.  I choose Ubuntu because of its popularity and I wanted to experience its desktop environment once again, GNOME.  I used GNOME many years ago when it was very young, I saw potential, and I would like to see how it has grown. I plan to spend the vast majority of my time within Xterm at the command prompt, but I also want to get familiar with Ubuntu/GNOME.  I’ll also do most of my systems administration at the command prompt, but again, getting familiarity with Ubuntu would be good to know.

How do I plan to installing Ubuntu, given that I will not have a OS already installed and I will not have a CD-ROM/DVD?  Ubuntu does have an ability to be installed via an USB stick.

I’m not going to abandon MS Windows.  I have many tools that I use in Windows and its not practical to just abandon them for something else, at least not right now.  I would like to dual boot the system with Linux and MS Windows.  Ideally, I’ll keep my old Windows image on my current PC’s hard drive, put that drive in my new system, and have the hard drive be my second OS on the bootloader’s chain of operating systems.  I know this could be done if I choose to re-install MS Windows and all my applications but I don’t know the challenges I’ll face given I’m using an establish image …. it will be a learning opportunity!

If I’m forces to do a re-install of MS Windows, I might us Oracle VM VirtualBox, which is a x86 virtualization software package. VirtualBox can be installed on an existing host operating system as an application; this host application allows additional guest operating systems, each known as a Guest OS, to be loaded and run, each with its own virtual environment.  The typical way of installing a guest operating system is to install it from the ground up. In general, you don’t see VirtualBox running a guest operating system from an existing drive or partition.  Never the less, a search of the Web does show evidence that people have made it work this way (1, 2, 3, 4, 5, 6, 7).

The last, and least desirable approach (for my needs) is to use Windows applications is via Wine.  Wine (originally an acronym for “Wine Is Not an Emulator”) is a compatibility layer capable of running Windows applications on Linux. Instead of simulating internal Windows logic like a virtual machine or emulator, Wine translates Windows API calls into Linux calls on-the-fly, eliminating the performance and memory penalties of other methods and allowing you to cleanly integrate Windows applications into your desktop.  Since it doesn’t create a virtual machine programs perform faster than in a VM.  However, you’ll need to test it with your application since it doesn’t support all programs.  Also, you’re not running MS Windows, just the MS Windows compatible applications.  This is fine if your interest in running Excel standalone, but you can’t perform anything that requires the MS operating system.

Boot Loader
A boot loader is the first software program that runs when a computer starts. It is responsible for loading and transferring control to the operating system kernel software. The kernel, in turn, initializes the rest of the operating system.  GRUB (GRand Unified Bootloader) is a boot loader package developed to support multiple operating systems and allow the user to select among them during boot-up.  GRUB is often the default boot loader for Linux and is preferred to MS Windows since it makes up for numerous deficiencies while providing full-featured command line and graphical interfaces.  GRUB is the default boot loader for Ubuntu, making it an easy choose.  GRUB is powerful and complex so check out How I configured grub as the default bootloader on a UEFI Boot systems.

What does it Cost
Now that I have a plan, what will all this cost?  I estimate it will be $765, less than one-third the cost of my present system.  Granted I’ll be reusing the monitor, key board, mouse, and some drives but this is a substantial price difference for what will be a much more capable machine.

cost

A Better Mobile Display for the Raspberry Pi (and my journey to a better understanding of X)

As I described in an earlier post, I run my Raspberry Pi (RPi) as a headless system, using Cygwin/X‘s xterm for command line interaction with the RPi, with my PC being my X server to support any X Window applications.  I can move files between the PC and the RPi via my pseudo-Dropbox.  I really recommend this configuration and its working perfectly for … Continue reading

The Simplest XBee Network

As I continue my investigation of the XBee radio, I’m impressed by the functionality compressed into the the small package, but I have been frustrated by one fact.  It has been a hard road to understand the device and to make it do something useful.  There is a confusing mass of commands and options that can be … Continue reading

Configuration Utilities for XBee Radios

My ultimate aim is to wirelessly network several Arduino based platforms with a centralized Raspberry Pi controller. There is much for me to learn to get this operational, not the least of which is the radio device I plan to use, the Xbee.  To get up to speed on the Xbee, I found the tutorials at AdafruitSparkfun, and Parallax helpful.   More detailed references are listed at the end of this post, but the very first challenge is to configure the XBee radios for operation.  This post provides insight on how this can be done, and my main mission, create a few simple utilities that make that job easy.

Xbee Radios
I purchased two XBee Series 1 Module (Freescale 802.15.4 firmware) from Adafruit.  These are manufactured by Digi and are low-power module with wire antenna  (XB24-AWI-001).  They have a 250 kbps RF data rates and operate at 2.4 GHz.  These radios use the IEEE 802.15.4 networking protocol and can perform point-to-multi-point or peer-to-peer networking , but as configured here, they do not mesh.  The Digi models that handle meshing are Digimesh, ZNet2.5 and Zigbee (ZB).  Digimesh is a version of firmware that runs on Series 1 hardware. So, if you choose to, you can upgrade these modules to Digimesh firmware to get meshing.

Xbee Adapter Board
Along with the XBee radios, I purchased adapter boards designed to make it easier to work with the radios. The adopter provides on-board 3.3V regulator power from a 5 volt source, voltage level shifting circuitry so you can connect  5V circuitry to the XBee, commonly used pins are brought out along the edge (making it easy to breadboard), and engineered to be interface via FTDI cable to a computer via USB.  The image and the text below describe the pin-out for the Adafruit  XBee Adapter:

Adafruit Xbee Adapter Pinout

  1. 3V pin - This is either an input power pin (if 5V is not provided) or an output from the 250mA regulator if 5V is provided
  2. DTR - “Data terminal ready”  is a flow control pin used to tell the XBee that the microcontroller or computer host is ready to communicate.
  3. RST - “Reset”  pin can be used to reset the XBee.  By default it is pulled high by the 10K resistor under the module. To reset, pull this pin low.’
  4. Ground - common ground for power and signal
  5. CTS - “Clear to Send” this is a flow control pin that can be used to determine if there is data in the XBee input buffer ready to be read
  6. 5V - This is the power input pin into the 3.3V regulator. Provide up to 6V that will be linearly converted into 3.3V
  7. RX - “Receive Data” is the XBee’s serial recieve pin. Serial data is sent on this pin into the XBee to be transmitted wirelessly
  8. TX - “Transmit Data” is the XBee’s serial transmit pin. Serial data is sent on this pin out of the XBee, after it has been transmitted wirelessly from another module
  9. RTS - “Ready to Send” is a flow control pin that can be used to tell the XBee to signal that the computer or microcontroller needs a break from reading serial data.
  10. see pin #1

The DTR, RTS, RST and RX pins (going into the XBee) pass through a level converter chip that brings the levels to 3.3V. Adafruit claims you can use pretty much anywhere between 2.7 to 5.5V data to communicate with the XBee. The breakout pins on the bottom of the board are not level shifted and you should try to keep data going directly into the XBee pin under 3.3V

Xbee radio installed in a XBee Adapter with the USB FTDI TTL-232 Cable attached

XBee Initial Configuration and Testing 
You need a way to communicate withe the Xbee, via it adapter,  to set it up.  This can be done via Adafruit’s  USB FTDI TTL-232 Cable, and the Digi X-CTU serial terminal program.  By the way, the X-CTU user guide describes the many more things it can do beyond the configuration shown here.

  1. Plug in the USB FTDI TTL-232 Cable into a PC USB port.  If drivers are not installed automatically (it didn’t for me), follow the steps at the FTDI site.
  2. Download the X-CTU, double click on the executable file, and follow the instructions to install the program.
  3. Now connect the USB FTDI TTL-232 Cable to the Xbee Adapter as shown in the picture to the right and insert the USB end of the cable to you PC.  Start the X-CTU.
  4. To connect, configure and upgrading the Xbee, follow the Adafruit instructions for the Xbee Adapter board. Note that if you follow the instructions (I didn’t – I kept it at 9600 baud), the modem’s serial interface is now set to 19,200 baud, not the default 9600 used by X-CTU.  Remember this next time you use X-CTU with this Xbee.
  5. If your instructed by X-CTU to reset the Xbee, you can do this by shorting the reset pin, RST pin,  to ground.

The configuration can be touchy, it can go badly, or not at all.  In my case, I seem to have one Xbee Adapter that can reliably perform a firmware upgrade but the other one took some time due to a lose fitting between the Adaptor and  Xbee.  If you run into configuration problems, check out these sites: Using XCTU to Invoke the BootloaderThe Unofficial XBee FAQ,  How to recover from a failed firmware upgrade.

Quickly Getting the Xbee’s Communicating
The next step for me was just do a basic test of getting two XBee device communicating with each other. This is just a sanity test to see evidence of communication between the devices. Basically, I just followed the instructions provided by Adafruit.
simple test

  1. Using the X-CTU, set the PAN ID to the same value on the two Xbee’s.
  2. Select an Ardunio that has been programmed to send repeated brief messages to its serial port.  I used the standard LED Blinking sketch but put in some write statements in the loop.
  3. Using an Arduino and breadboard, connect +5V and ground to provide power. Make sure the XBee’s LED is blinking.
  4. Connect the RX line (input) of the XBee to the TX line (output) of the Arduino. Connect the RX line (input) of the Arduino to the TX line (output) of the Xbee. Plug the Arduino into your PC’s serial port.
  5. Now take the second Xbee and connect the  USB FTDI TTL-232 Cable to the Xbee and the PC.  The cable is doing nothing but appling power to the Xbee.
  6. Now you should see the receive LED periodically light on the USB FTDI TTL-232 Cable tethered Xbee.
  7. You now got proof that the two Xbee’s are communicating.  The Arduino connected Xbee is sending data to its serial port and the USB FTDI TTL-232 Cable tethered Xbee is receiving it.

Above you’ll find a picture of the configuration, and below is the Arduino sketch I used.

/*
 *  Xbee Test via Blink LED
 *
 *Turns on an LED on for one second, then off for one second, repeatedly.
 *Also increase brigthness of analog LED.
 *
 *The circuit:
 * LED1 connected from digital pin 13 to ground.
 * LED2 connected from analog pin 9 to ground.
 * Note: On most Arduino boards, there is already an LED on the board
 * connected to pin 13, so you don't need any extra components for this example.

 *Created 1 June 2005
 *By David Cuartielles
 *http://arduino.cc/en/Tutorial/Blink
 *based on an orginal by H. Barragan for the Wiring i/o board
 *Modified by Jeff Irland in December 2012
 */

int ledPin1 =  13;    // LED connected to digital pin 13
int ledPin2 =  9;     // LED connected to analog pin 9
int brightness = 0;

// The setup() method runs once, when the sketch starts
void setup()   {
    Serial.begin(9600);
    pinMode(ledPin1, OUTPUT);     // initialize the digital pin as an output
    Serial.println("Arduino done with setup()");
}

// the loop() method runs over and over again,
// as long as the Arduino has power
void loop()
{
    digitalWrite(ledPin1, HIGH);   // set the LED on
    Serial.println("LED set HIGH.");
    delay(1000);                   // wait for a second

    digitalWrite(ledPin1, LOW);    // set the LED off
    Serial.println("LED set LOW.");
    delay(1000);                   // wait for a second

    brightness = brightness + 5;
    analogWrite(ledPin2, brightness);
    Serial.println("LED brightness increased.");
}

Installing XBee Python Tools for the RPi
While the MS Windows based Digi X-CTU tool is just fine, I want to use the RPi’s and Python to access the XBee serial communication API, and its advanced features, for one or more XBee devices.  I prefer simple utilities, that can be scripted within the Linux shell.  Call me a Linux snob if you wish, but I don’t care for MS Windows!

In my post “Selecting XBee Radios and Supporting Software Tools“, I referenced a Python package that could be used to create my utilitiesd, call python-xbee, and I will be using it here. It claims to provides a semi-complete implementation of the XBee binary API protocol and allows a developer to send and receive the information they desire without dealing with the raw communication details. It also claims the  library is compatible with both XBee 802.15.4 (Series 1) and XBee ZigBee (Series 2) modules, normal and PRO.

First, we need to load some additional required Python Packages, that being pySerial and Nose. pySerial extends python’s capabilities to include interacting with a serial port and Nose is a package providing a very easy way to build tests, based on the Python class unittest.  (Don’t let this all scare you away, these are necessary but your not going to use them directly).  To load these package:

sudo pip install pySerial
sudo pip install nose

Download the python-xbee tools from Google Code or Python Org and place them into the RPi’s $HOME/src.  The README file provides installation instructions.  It states that the following command automatically test and install the package for you:

sudo python setup.py install

There is a simple to use RPi platform tool that I have modified for my needs, that is a XBee serial command shell for interacting with XBee radios.  It performs the core functions of the official configuration tool, X-CTU, which only runs on Windows. (There happens to be a cross-platform version of X-CTU called moltosenso Network Manager but I don’t need all this horse power.)  I’ll use this X-CTU-alternative to configure the individual XBee radios.  With the X-CTU, you can update firmware, etc. but most of the time you need the program to do simple configuration tasks. You could use Linux’s minicom, but I prefer a simpler tool which can be scripted so I can configure several XBee radios identically.  I found much of what I wanted in an existing Python XBee tools for configuration.  I made some modification/improvements, I call it the XBeeTerm, and its listed below:


#!/usr/bin/env python

"""XBeeTerm.py is a XBee serial command shell for interacting with XBee radios

	This command interpretors establishes communications with XBee radios so that AT Commands can be sent to the XBee.
	The interpretors output is color coded to help distinguish user input, from XBee radio output, and from
	interpretors output. This command-line interpretor uses Python modual Cmd, and therefore, inherit bash-like history-list
	editing (e.g. Control-P or up-arrow scrolls back to the last command, Control-N or down-arrow forward to the next one,
	Control-F or right-arrow moves the cursor to the right non-destructively, Control-B or left-arrow moves the cursor
	to the left non-destructively, etc.).

	XBeeTerm is not a replacement for the Digi X-CTU program but a utility program for the Linux envirnment.  You can pipe
	scripts of XBee configration commands, making it easy to multiple radios.  Also, XBeeTerm wait for the arrival of a
	XBee data packet, print the XBee frame, and wait for the next packet, much like a packet sniffer.

	XBeeTerm Commands:
		baudrate <rate>		set the baud rate at which you will communicate with the XBee radio
		serial <device>		set the serial device that the XBee radio is attached
		watch				wait for the arrival of a XBee data packet, print it, wait for the next
		shell or !			pause the interpreter and invoke command in Linux shell
		exit or EOF			exit the XBeeTerm
		help or ?			prints out short discription of the commands (similar to the above)

	Just like the Digi X-CTU program, the syntax for the AT commands are:
		AT+ASCII_Command+Space+Optional_Parameter+Carriage_Return
		Example: ATDL 1F<CR>

	Example Session:
		baudrate 9600			# (XBeeTerm command) set the baudrate used to comm. with the XBee
		serial /dev/ttyUSB0		# (XBeeTerm command) serial device which has the XBee radio
		+++ 					# (XBee command) enter AT command mode on the XBee
		ATRE					# (XBee command) restore XBee to factory settings
		ATAP 2					# (XBee command) enable API mode with escaped control characters
		ATCE 0					# (XBee command) make this XBee radio an end device
		ATMY AAA1				# (XBee command) set the address of this radio to eight byte hex
		ATID B000				# (XBee command) Set the PAN ID to eight byte hex
		ATCH 0E					# (XBee command) set the Channel ID to a four byte hex
		ATPL 0					# (XBee command) power level at which the RF module transmits
		ATWR					# (XBee command) write all the changes to the XBee non-volatile memory
		ATFR					# (XBee command) reboot XBee radio
		exit					# (XBeeTerm command) exit python shell

	Referance Materials:
		XBee 802.15.4 (Series 1) Module Product Manual (section 3: RF Module Configuration)
			ftp://ftp1.digi.com/support/documentation/90000982_A.pdf
		python-xbee Documentation: Release 2.0.0, Paul Malmsten, December 29, 2010

http://python-xbee.googlecode.com/files/XBee-2.0.0-Documentation.pdf

		cmd - Support for line-oriented command interpreters

http://docs.python.org/2/library/cmd.html

		cmd - Create line-oriented command processors

http://bip.weizmann.ac.il/course/python/PyMOTW/PyMOTW/docs/cmd/index.html

		Easy command-line applications with cmd and cmd2

http://pyvideo.org/video/306/pycon-2010--easy-command-line-applications-with-c

	Orginally Created By:
		Amit Snyderman (amit@amitsnyderman.com), on/about August 2012, and taken from

https://github.com/sensestage/xbee-tools

	Modified By:
		Jeff Irland (jeff.irland@gmail.com) in January 2013
"""

# imported modules
import os						# portable way of using operating system dependent functionality
import sys						# provides access to some variables used or maintained by the interpreter
import time						# provides various time-related functions
import cmd						# provides a simple framework for writing line-oriented command interpreters
import serial					# encapsulates the access for the serial port
import argparse					# provides easy to write and user-friendly command-line interfaces
from xbee import XBee			# implementation of the XBee serial communication API
from pretty import switchColor	# colored text in Python using ANSI Escape Sequences

# authorship information
__author__ = "Jeff Irland"
__copyright__ =	"Copyright 2013"
__credits__ = "Amit Snyderman, Marije Baalman, Paul Malmsten"
__license__ = "GNU General Public License"
__version__ = "0.1"
__maintainer__ = "Jeff Irland"
__email__ = "jeff.irland@gmail.com"
__status__ = "Development"
__python__ = "Version 2.7.3"

# text colors to be used during terminal sessions
CMD_INPUT_TEXT = 'normal'
CMD_OUTPUT_TEXT = 'bright yellow'
XBEE_OUTPUT_TEXT = 'bright red'
SHELL_OUTPUT_TEXT = 'bright cyan'
WATCH_OUTPUT_TEXT = 'bright green'

class ArgsParser():
	"""Within this object class you should load all the command-line switches, parameters, and arguments to operate this utility"""
	def __init__(self):
		self.parser = argparse.ArgumentParser(description="This command interpretors establishes communications with XBee radios so that AT Commands can be sent to the XBee.  It can be used to configure or query the XBee radio", epilog="This utility is primarily intended to change the AT Command parameter values, but could be used to query for the parameter values.")
		self.optSwitches()
		self.reqSwitches()
		self.optParameters()
		self.reqParameters()
		self.optArguments()
		self.reqArguments()
	def optSwitches(self):
		"""optonal switches for the command-line"""
		self.parser.add_argument("--version", action="version", version=__version__, help="print version number on stdout and exit")
		self.parser.add_argument("-v", "--verbose", action="count", help="produce verbose output for debugging")
	def reqSwitches(self):
		"""required switches for the command-line"""
		pass
	def optParameters(self):
		"""optonal parameters for the command-line"""
		pass
	def reqParameters(self):
		"""required parameters for the command-line"""
		pass
	def optArguments(self):
		"""optonal arguments for the command-line"""
		self.parser.add_argument(nargs="*", action="store", dest="inputs", default=None, help="XBeeTerm script file with AT Commands to be executed")
	def reqArguments(self):
		"""required arguments for the command-line"""
		pass
	def args(self):
		"""return a object containing the command-line switches, parameters, and arguments"""
		return self.parser.parse_args()

class XBeeShell(cmd.Cmd):
	def __init__(self, inputFile=None):
		"""Called when the objects instance is created"""
		cmd.Cmd.__init__(self)
		self.serial = serial.Serial()
		if inputFile is None:
			self.intro = "Command-Line Interpreter for Configuring XBee Radios"
			self.prompt = "xbee% "
		else:
			self.intro = "Configuring XBee Radios via command file"
			self.prompt = ""			# Do not show a prompt after each command read
			sys.stdin = inputFile

	def default(self, p):
		"""Command is assumed to be an AT Commands for the XBee radio"""
		if not self.serial.isOpen():
			print "You must set a serial port first."
		else:
			if p == '+++':
				self.serial.write('+++')
				time.sleep(2)
			else:
				self.serial.write('%s\r' % p)
				time.sleep(0.5)

			output = ''
			while self.serial.inWaiting():
				output += self.serial.read()
			if output == '' :
				print 'XBee timed out, so reissue "+++". (Or maybe XBee doesn\'t understand "%s".)' % p
			else:
				switchColor(XBEE_OUTPUT_TEXT)
				print output.replace('\r', '\n').rstrip()

	def emptyline(self):
		"""method called when an empty line is entered in response to the prompt"""
		return None		# do not repeat the last nonempty command entered

	def precmd(self, p):
		"""executed just before the command line line is interpreted"""
		switchColor(CMD_OUTPUT_TEXT)
		return cmd.Cmd.precmd(self, p)

	def postcmd(self, stop, p):
		"""executed just after a command dispatch is finished"""
		switchColor(CMD_INPUT_TEXT)
		return cmd.Cmd.postcmd(self, stop, p)

	def do_baudrate(self, p):
		"""Set the baud rate used to communicate with the XBee"""
		self.serial.baudrate = p
		print 'baudrate set to %s' % self.serial.baudrate

	def do_serial(self, p):
		"""Linux serial device path to the XBee radio (e.g. /dev/ttyUSB0)"""
		try:
			self.serial.port = p
			self.serial.open()
			print 'Successfully opened serial port %s' % p
		except Exception, e:
			print 'Unable to open serial port %s' % p

	def do_shell(self, p):
		"""Pause the interpreter and invoke command in Linux shell"""
		print "running shell command: ", p
		switchColor(SHELL_OUTPUT_TEXT)
		print os.popen(p).read()

	def do_watch(self, p):
		"""Wait for the arrival of a XBee data packet, print it when it arrives, wait for the next"""
		if not self.serial.isOpen():
			print "You must set a serial port first."
		else:
			print "Entering watch mode..."
			switchColor(WATCH_OUTPUT_TEXT)
			while 1:
				packet = xbee.find_packet(self.serial)
				if packet:
					xb = xbee(packet)
					print xb

	def do_exit(self, p):
		"""Exits from the XBee serial terminal"""
		self.serial.close()
		print "Exiting", os.path.basename(__file__)
		return True

	def do_EOF(self, p):
		"""EOF (end-of-file) or Ctrl-D will return True and drops out of the interpreter"""
		self.serial.close()
		print "Exiting", os.path.basename(__file__)
		return True

	def help_help(self) :
		"""Print help messages for command arguments"""
		print 'help\t\t', self.help_help.__doc__
		print 'shell <cmd>\t', self.do_shell.__doc__
		print 'EOF or Ctrl-D\t', self.do_EOF.__doc__
		print 'exit\t\t', self.do_exit.__doc__
		print 'watch\t\t', self.do_watch.__doc__
		print 'serial <dev>\t', self.do_serial.__doc__
		print 'baudrate <rate>', self.do_baudrate.__doc__

# Enter into XBee command-line processor
if __name__ == '__main__':
	# parse the command-line for switches, parameters, and arguments
	parser = ArgsParser()							# create parser object for the command-line
	args = parser.args()							# get list of command line arguments, parameters, and switches

	if args.verbose > 0:							# print what is on the command-line
		print os.path.basename(__file__), "command-line arguments =", args.__dict__

	# process the command-line arguments (i.e. script file) and start the command shell
	if len(args.inputs) == 0:	# there is no script file
		shell= XBeeShell()
		shell.cmdloop()
	else:							# there is a script file on the command-line
		if len(args.inputs) > 1:
			print os.path.basename(__file__), "will process only the first command-line argument."
		if os.path.exists(args.inputs[0]) :
			inputFile = open(args.inputs[0], 'rt')
			shell = XBeeShell(inputFile)
			shell.cmdloop()
		else:
			print 'File "%s" doesn\'t exist. Program terminated.' % args.inputs[0]

The XBeeTerm.py module imports functions from the pretty.py package, specifically to colorize the output for xterm on the Raspberry Pi.  This package is provided here:

#!/usr/bin/env python

"""pretty.py will color text for Xterm/VT100 type terminals using ANSI Escape Sequences

	A library that provides a Python print, stdout, and string wrapper that makes colored terminal
	text easier to use(e.g. without having to mess around with ANSI escape sequences).

	Referance Materials:
		Colored text in Python using ANSI Escape Sequences

http://nezzen.net/2008/06/23/colored-text-in-python-using-ansi-escape-sequences/

	Orginally Created By:
		Copyright (C) 2008 Brian Nez <thedude at bri1 dot com>

	Modified By:
		Jeff Irland (jeff.irland@gmail.com) in January 2013
"""

# imported modules
import sys

# authorship information
__author__ = "Jeff Irland"
__copyright__ =	"Copyright 2013"
__credits__ = "Brian Nez"
__license__ = "GNU General Public License"
__version__ = "1.0"
__maintainer__ = "Jeff Irland"
__email__ = "jeff.irland@gmail.com"
__status__ = "Production"

# Dictionary of ANSI escape sequences for coloring text
colorCodes = {
	'black':	'0;30',		'bright gray':	'0;37',
	'blue':		'0;34',		'white':		'1;37',
	'green':	'0;32',		'bright blue':	'1;34',
	'cyan':		'0;36',		'bright green':	'1;32',
	'red':		'0;31',		'bright cyan':	'1;36',
	'purple':	'0;35',		'bright red':	'1;31',
	'yellow':	'0;33',		'bright purple':'1;35',
	'dark gray':'1;30',		'bright yellow':'1;33',
	'normal':	'0'
}

def printc(text, color):
	"""Print in color"""
	print "\033["+colorCodes[color]+"m"+text+"\033[0m"

def writec(text, color):
	"""Write to stdout in color"""
	sys.stdout.write("\033["+colorCodes[color]+"m"+text+"\033[0m")

def switchColor(color):
	"""Switch terminal color"""
	sys.stdout.write("\033["+colorCodes[color]+"m")

def stringc(text, color):
	"""Return a string with ANSI escape sequences to color text"""
	return "\033["+colorCodes[color]+"m"+text+"\033[0m"

# Simple test routine to validate thing are working correctly
if __name__ == '__main__':
	printc("Welcome to the pretty.py test routine!", 'white')

	printc("I will now try to print a line of text in each color using \"writec()\"", 'white')
	for color in colorCodes.keys():
		writec("Hello, world!", color)
		print "\t", color

	printc("\n\nI will now try to print a line of text in each color using \"switchColor()\"", 'white')
	for color in colorCodes.keys():
		switchColor(color)
		print 'Hello World #2!'

	printc("\n\nI will now try to print a line of text in each color using \"printc()\"", 'white')
	for color in colorCodes.keys():
		printc('Hello World #3!', color)

	printc("\n\nI will now try to print a line of text in each color using \"stringc()\"", 'white')
	for color in colorCodes.keys():
		print stringc('Hello World #4!', color)

Identifying the RPi USB device used by the XBee
Since the python-xbee library wants to talk to the via a Linux serial devices, I'm using the USB FTDI TTL-232 Cable (FTDI is the USB chip manufacturer) used in the XBee configuration step done earlier.  I connected the cable to the RPi USB port  and then we need to find the serial tty the cable is associated with.  To do this, it takes a bit of detective work. Run the commands:

lsusb
dmesg | grep Manufacturer
dmesg | grep FTDI

A better command might be (but I'm not sure it will work every time):

dmesg | grep -i usb | grep -i tty

The interpretation of the output tells us the cable is attached to serial device /dev/ttyUSB0.  See the output below.

best use of dmesg

Another possibility is to use udevadm to gather information about specific devices but I never figured out exactly how to use it to answer my question.  Python also has a package called PyUSB that might provide some help, but also here you'll still need the vendor and product identification information.

Chances are that when you plug the cable into the same USB port the next time, it will default to the same tty but there is no certainty.  To assign a permanent tty name to the device, and never do any of this again, check out Persistent names for usb-serial devices.

Configuring the XBee Radios for API Mode
The configuration and testing of the XBee's done earlier was done in AT Command mode (Transparent Mode). In AT mode, everything sent to the RX line of the XBee radio will be sent out via the antenna, and all the incoming data from antenna will go to the XBee's TX line.  This is why we could check the sanity of the XBee radios in the earlier section, XBee Initial Configuration and Testing using a simple Arduino sketch.  We sent junk to the XBee and it transmitted it!

Now we'll configure two XBee radios (with a Coordinator and a single End Device) to form a network using API Mode.  In API Mode, XBee won't send out anything until it received the correct form of commands from the serial interface.  The XBee AT Command Set (page 27), specifically the  ATAP 2 command, allows you to configure the XBee radio for API Mode.   So why API Mode, consider the following:

  • When sending a packet, the transmitting radio receives an ACK, indicating the packet was successfully delivered. The transmitting radio will resend the packet if it does not receive an ACK.
  • Receive packets (RX), contain the source address of transmitting radio
  • You can configure a remote radio with the Remote AT feature
  • Easily address multiple radios and send broadcast TX packets
  • Receive I/O data from 1 or more remote XBees
  • Obtain RSSI (signal strength) of an RX packet
  • Packets include a checksum for data integrity

The XBeeTerm utility will  easily configure the XBee radios for API mode and set the appropriate network parameters.  To get a deeper appreciation of configuring the XBee radios, see the References at the end.  For here, I'll just run through the steps using the XBeeTerm.py tool and the configuration commands used, documented in file scripts.

Coordinator Configuration File: Config-Coordinator.txt

# To remove comments, white spaces, and blank lines, use the following:
#		sed '/^#/d; s/\([^$]\)#.*/\1/' Config-Coordinator.txt | sed 's/[ \t]*$//' > coord.txt
# Run this script to configure the XBee radio using the following:
#		python XBeeTerm.py coord.txt
#
baudrate 9600			# (XBeeTerm command) set the baudrate used to comm. with the XBee
serial /dev/ttyUSB0		# (XBeeTerm command) serial device which has the XBee radio
+++ 					# (XBee command) enter AT command mode on the XBee
ATRE					# (XBee command) restore XBee to factory settings
ATAP 2					# (XBee command) enable API mode with escaped control characters
ATCE 1					# (XBee command) make this XBee radio the network coordinator
ATMY AAA0				# (XBee command) set the address of this radio to eight byte hex (must be unique)
ATID B000				# (XBee command) Set the PAN ID to eight byte hex (all XBee's must have this same value)
ATCH 0E					# (XBee command) set the Channel ID to a four byte hex (all XBee's must have same value)
ATPL 0					# (XBee command) power level at which the RF module transmits (0 lowest / 4 highest)
ATWR					# (XBee command) write all the changes to the XBee non-volatile memory
ATFR					# (XBee command) reboot XBee radio
exit					# (XBeeTerm command) exit python shell

End Device Configuration File: Config-End-Device.txt

# To remove comments, white spaces, and blank lines, use the following:
#		sed '/^#/d; s/\([^$]\)#.*/\1/' Config-End-Device.txt | sed 's/[ \t]*$//' > endd.txt
# Run this script to configure the XBee radio using the following:
#		python XBeeTerm.py endd.txt
#
baudrate 9600			# (XBeeTerm command) set the baudrate used to comm. with the XBee
serial /dev/ttyUSB0		# (XBeeTerm command) serial device which has the XBee radio
+++ 					# (XBee command) enter AT command mode on the XBee
ATRE					# (XBee command) restore XBee to factory settings
ATAP 2					# (XBee command) enable API mode with escaped control characters
ATCE 0					# (XBee command) make this XBee radio an end device
ATMY AAA1				# (XBee command) set the address of this radio to eight byte hex (must be unique)
ATID B000				# (XBee command) Set the PAN ID to eight byte hex (all XBee's must have this same value)
ATCH 0E					# (XBee command) set the Channel ID to a four byte hex (all XBee's must have same value)
ATPL 0					# (XBee command) power level at which the RF module transmits (0 lowest / 4 highest)
ATWR					# (XBee command) write all the changes to the XBee non-volatile memory
ATFR					# (XBee command) reboot XBee radio
exit					# (XBeeTerm command) exit python shell

As they stand right now, these files could not be processed by XBeeTerm.py because of the comments (included to make the contents understandable).  To clean this up, the command sed '/^#/d; s/\([^$]\)#.*/\1/'will remove all shell type comments from a file and sed 's/[ \t]*$//'will remove unneeded white space.  Putting this all together and you can use this to prepare the above files for XBeeTerm.py:

sed '/^#/d; s/\([^$]\)#.*/\1/' Config-Coordinator.txt | sed 's/[ \t]*$//' > coord.txt
sed '/^#/d; s/\([^$]\)#.*/\1/' Config-End-Device.txt | sed 's/[ \t]*$//' > endd.txt

Now execute the following python XBeeTerm.py coord.txt and you get the output below:

XBeeTerm Script

The yellow text is responses back from the XBee serial terminal and the red text is from the XBee radio itself.  Since all the red text is “OK”, all the commands took and the XBee radio is now configured as a Coordinator.  Now repeat this for the End Device XBee radio.

In this example, I have one End Device but what if you have multiple devices, do you need a Config-End-Device.txt file for each end device?  The only change within the configuration file is the radio’s address, which is established via the ATMY command.  Here is a trick to avoid the need for multiple files.  First, configure all your End Devices using the configuration file.  Then, for each radio, modify the ATMY use the following:

echo -e "baudrate 9600\nserial /dev/ttyUSB0\n+++\nATMY AAA1\nATWR\nATFR\nexit" | python XBeeTerm.py

but for each End Device radio, increment the ATMY address by one (e.g. AAA2, AAA3, …).

Querying XBee for Configuration
Now that we believe the XBee radios are properly configured, lets verify that by query the radios.  You could use XBeeTerm to perform this function by including only the AT Command without the parameter but I wanted a more informative tool. For this, I have created another utility that can take a list of AT Commands as arguments and query the XBee radio for the AT’s parameter value.  This utility, call XBeeQuery.py, is listed below:


#! /usr/bin/python

"""
	This utility will query a XBee radio for some of it's AT Command parameters and print their values, as well as
	optional discriptive information.  It has a set of default AT Commands or the use can provide the desired set
	of AT Commands on the command-line.

	The dictionay of AT Command descriptive information is limited but can be easily expanded.  The descriptive
	information was taken from the first referance given below. Also note that if this utility appears to hang,
	it is almost certainly waiting on a response from the XBee. To continue processing the AT Command list, use Ctrl-C.

	Reference Materials:
		XBee/XBee-PRO OEM RF Modules: Product Manual v1.xCx - 802.15.4 Protocol
			ftp://ftp1.digi.com/support/documentation/90000982_A.pdf
		python-xbee Documentation: Release 2.0.0, Paul Malmsten, December 29, 2010

http://python-xbee.googlecode.com/files/XBee-2.0.0-Documentation.pdf

		Parser for command-line options, arguments and sub-commands

http://docs.python.org/2/library/argparse.html#module-argparse

"""

# imported modules
import os						# portable way of using operating system dependent functionality
import sys						# provides access to some variables used or maintained by the Python interpreter
import serial					# encapsulates the access for the serial port
import argparse					# provides easy to write and user-friendly command-line interfaces
from xbee import XBee			# implementation of the XBee serial communication API
from pretty import stringc		# provides colored text for xterm and VT100 type terminals using ANSI Escape Sequences

# authorship information
__author__ = "Jeff Irland"
__copyright__ =	"Copyright 2013"
__credits__ = "Paul Malmsten"
__license__ = "GNU General Public License"
__version__ = "0.1"
__maintainer__ = "Jeff Irland"
__email__ = "jeff.irland@gmail.com"
__status__ = "Development"
__python__ = "Version 2.7.3"

# text colors to be used during terminal sessions
NORMAL_TEXT = 'normal'
CMD_TEXT = 'bright red'
NAME_TEXT = 'bright yellow'
CAT_TEXT = 'bright yellow'
DESC_TEXT = 'normal'
RANGE_TEXT = 'bright cyan'
DEFAULT_TEXT = 'bright green'

class ArgsParser():
	"""Within this object class you should load all the command-line switches, parameters, and arguments to operate this utility"""
	def __init__(self):
		self.parser = argparse.ArgumentParser(description="This utility will query a XBee radio for some of it's AT Command parameters and print their values.  It has a set of default AT Commands or the use can provide the desired set of AT Commands on the command-line.", epilog="This utility is for query only and will not change the AT Command parameter values.")
		self.optSwitches()
		self.reqSwitches()
		self.optParameters()
		self.reqParameters()
		self.optArguments()
		self.reqArguments()
	def optSwitches(self):
		"""optonal switches for the command-line"""
		self.parser.add_argument("--version", action="version", version=__version__, help="print version number on stdout and exit")
		self.parser.add_argument("-v", "--verbose", action="count", help="produce verbose output for debugging")
		self.parser.add_argument("-n", "--name", required=False, action="store_true", help="print the name (i.e. short description) of the XBee AT Command")
		self.parser.add_argument("-d", "--description", required=False, action="store_true", help="print the full description of the XBee AT Command")
	def reqSwitches(self):
		"""required switches for the command-line"""
		pass
	def optParameters(self):
		"""optonal parameters for the command-line"""
		self.parser.add_argument("-b", "--baudrate", required=False, action="store", metavar="RATE", type=int, default=9600, help="baud rate used to communicate with the XBee radio")
		self.parser.add_argument("-p", "--device", required=False, action="store", metavar="DEV", type=str, default='/dev/ttyUSB0', help="open this serial port or device to communicate with the XBee radio")
	def reqParameters(self):
		"""required parameters for the command-line"""
		pass
	def optArguments(self):
		"""optonal arguments for the command-line"""
		self.parser.add_argument(nargs="*", action="store", dest="inputs", help="AT Commands to be queried")
	def reqArguments(self):
		"""required arguments for the command-line"""
		pass
	def args(self):
		"""return a object containing the command-line switches, parameters, and arguments"""
		return self.parser.parse_args()

class ATDict():
	"""Within this object class you should load a dictionary of { "AT Command" : [ "Name", "Category", "Description", "Parameter Range", "Default Value" ] }"""
	# Networking & Security, RF Interfacing, Sleep Modes (NonBeacon), Serial Interfacing, I/O Settings, Diagnostics, AT Command Options
	def __init__(self):
		self.commands = {
		"CH" : [ "Channel", "Networking", "Set/Read the channel number used for transmitting and receiving data between RF modules.", "0x0B - 0x1A", "0x0C" ],
		"ID" : [ "PAN ID", "Networking", "Set/Read the PAN (Personal Area Network) ID. Use 0xFFFF to broadcast messages to all PANs.", "0 - 0xFFFF" , "0x3332" ],
		"DH" : [ "Destination Address High", "Networking", "Set/Read the upper 32 bits of the 64-bit destination address. When combined with DL, it defines the destination address used for transmission. To transmit using a 16-bit address, set DH parameter to zero and DL less than 0xFFFF. 0x000000000000FFFF is the broadcast address for the PAN.", "0 - 0xFFFFFFFF", "0" ],
		"DL" : [ "Destination Address Low", "Networking", "Set/Read the lower 32 bits of the 64-bit destination address. When combined with DH, DL defines the destination address used for transmission. To transmit using a 16-bit address, set DH parameter to zero and DL less than 0xFFFF. 0x000000000000FFFF is the broadcast address for the PAN.", "0 - 0xFFFFFFFF", "0" ],
		"MY" : [ "16-bit Source Address", "Networking", "Set/Read the RF module 16-bit source address. Set MY =0xFFFF to disable reception of packets with 16-bit addresses. 64-bit source address(serial number) and broadcast address (0x000000000000FFFF) is always enabled.", "0 - 0xFFFF", "0" ],
		"SH" : [ "Serial Number High", "Networking", "Read high 32 bits of the RF module's unique IEEE 64-bit address. 64-bit source address is always enabled.", "0 - 0xFFFFFFFF [read-only]", "Factory-set" ],
		"SL" : [ "Serial Number Low", "Networking", "Read low 32 bits of the RF module's unique IEEE 64-bit address. 64-bit source address is always enabled.", "0 - 0xFFFFFFFF [read-only]", "Factory-set" ],
		"RR" : [ "XBee Retries", "Networking", "Set/Read the maximum number of retries the module will execute in addition to the 3 retries provided by the 802.15.4 MAC. For each XBee retry, the 802.15.4 MAC can execute up to 3 retries.", "0 - 6", "0" ],
		"RN" : [ "Random Delay Slots", "Networking", "Set/Read the minimum value of the back-off exponent in the CSMA-CA algorithm that is used for collision avoidance. If RN = 0, collision avoidance is disabled during the first iteration of the algorithm (802.15.4 - macMinBE).", "0 - 3 [exponent]", "0" ],
		"MM" : [ "MAC Mode", "Networking", "Set/Read MAC Mode value. MAC Mode enables/disables the use of a Digi header in the 802.15.4 RF packet. When Modes 0 or 3 are enabled(MM=0,3), duplicate packet detection is enabled as well as certain AT commands.", "0 = Digi Mode, 1 = 802.15.4 (no ACKs), 2 = 802.15.4 (with ACKs), 3 = Digi Mode (no ACKs)", "0" ],
		"NI" : [ "Node Identifier", "Networking", "Stores a string identifier. The register only accepts printable ASCII data. A string can not start with a space. Carriage return ends command. Command will automatically end when maximum bytes for the string have been entered. This string is returned as part of the ND (Node Discover) command. This identifier is also used with the DN (Destination Node) command.", "20-character ASCII string", "-" ],
		"NT" : [ "Node Discover Time", "Networking", "Set/Read the amount of time a node will wait for responses from other nodes when using the ND (Node Discover) command.", "0x01 - 0xFC [x 100 ms]", "0x19" ],
		"CE" : [ "Coordinator Enable", "Networking", "Set/Read the coordinator setting. A value of 0 makes it an End Device but a value of 1 makes it a Coordinator.", "0 = End Device, 1 = Coordinator", "0" ],
		"SC" : [ "Scan Channels", "Networking", "Set/Read list of channels to scan for all Active and Energy Scans as a bitfield. This affects scans initiated in command mode (AS, ED) and during End Device Association and Coordinator startup", "0 - 0xFFFF [bitfield](bits 0, 14, 15 not allowed on the XBee-PRO)", "0x1FFE (all XBee-PRO Channels)" ],
		#"SD" : [ "Scan Duration", "Networking", "Set/Read the scan duration exponent.  For End Device - Duration of Active Scan during Association.  For Coordinator - If ‘ReassignPANID’ option is set on Coordinator [refer to A2 parameter],  SD determines the length of time the Coordinator will scan channels to locate existing PANs. If ‘ReassignChannel’ option is set, SD determines how long the Coordinator will perform an Energy Scan to determine which channel it will operate on.  ‘Scan Time’ is measured as (# of channels to scan] * (2 ^ SD) * 15.36ms). The number of channels to scan is set by the SC command. The XBee can scan up to 16 channels (SC = 0xFFFF).", "0-0x0F [exponent]", "4" ],
		"A1" : [ "End Device Association", "Networking", "Set/Read End Device association options. bit 0 - ReassignPanID (0 - Will only associate with Coordinator operating on PAN ID that matches module ID / 1 - May associate with Coordinator operating on any PAN ID), bit 1 - ReassignChannel(0 - Will only associate with Coordinator operating on matching CH Channel setting / 1 - May associate with Coordinator operating on any Channel), bit 2 - AutoAssociate (0 - Device will not attempt Association / 1 - Device attempts Association until success Note: This bit is used only for Non-Beacon systems. End Devices in Beacon-enabled system must always associate to a Coordinator), bit 3 - PollCoordOnPinWake (0 - Pin Wake will not poll the Coordinator for indirect (pending) data / 1 - Pin Wake will send Poll Request to Coordinator to extract any pending data), bits 4 - 7 are reserved.", "0 - 0x0F [bitfield]", "0" ],
		#"XX" : [ "", "", "", "", "" ],
		#"XX" : [ "", "", "", "", "" ],
		}
	def dictionary(self):
		"""return the whole dictionary of command"""
		return self.commands
	def command(self, cmd):
		"""return the colorized AT Command"""
		return stringc(cmd, CMD_TEXT)
	def name(self, cmd):
		"""return the colorized name of the AT Command"""
		return stringc(self.commands[cmd][0], NAME_TEXT)
	def category(self, cmd):
		"""return the colorized category of the AT Command"""
		return stringc(self.commands[cmd][1], CAT_TEXT)
	def description(self, cmd):
		"""return the colorized description of the AT Command"""
		return stringc(self.commands[cmd][2], DESC_TEXT)
	def range(self, cmd):
		"""return the colorized range of the AT Command"""
		return stringc(self.commands[cmd][3], RANGE_TEXT)

	def default(self, cmd):
		"""return the colorized devault value of the AT Command"""
		stringc(self.commands[cmd][3], DEFAULT_TEXT)

class FrameID():
	"""XBee frame IDs must be in the range 0x01 to 0xFF (0x00 is a special case).
	This object allows you to cycles through these numbers."""
	def __init__(self):
		self.frameID = 0
	def inc(self):
		if self.frameID == 255:
			self.frameID = 0
		self.frameID += 1
		return self.frameID

def byte2hex(byteStr):
	"""Convert a byte string to it's hex string representation"""
	hex = []
	for aChar in byteStr:
		hex.append("%02X " % ord(aChar))
	return ''.join(hex).strip()

if __name__ == '__main__':
	# parse the command-line for switches, parameters, and arguments
	parser = ArgsParser()							# create parser object for the command-line
	args = parser.args()							# get list of command line arguments, parameters, and switches
	if args.verbose > 0:							# print what is on the command-line
		print os.path.basename(__file__), "command-line arguments =", args.__dict__

	# create required objects
	frameID = FrameID()								# create a cycling frame ID sequence number to be used in XBee frames
	at = ATDict()									# create a dictionary of XBee AT Commands definitions
	ser = serial.Serial(args.device, args.baudrate)	# Open the serial port that has the XBee radio
	xbee = XBee(ser)								# Create XBee object to communicate with the XBee radio

	# if user provided desired AT Commands, use them for query, otherwise use the whole dictionary
	if len(args.inputs) == 0:
		args.inputs = at.dictionary()

	# for the command list provided, query the XBee radio AT Command's parameters
	for cmd in args.inputs:
		xbee.send('at', frame_id=chr(frameID.inc()), command=cmd)
		try:
			response = xbee.wait_read_frame()
			if args.name:
				print at.command(cmd) + " = " + byte2hex(response['parameter']) + "  " + at.name(cmd)
			else:
				print at.command(cmd) + " = " + byte2hex(response['parameter'])
			if args.description:
				print at.description(cmd) + "\n"
		except KeyboardInterrupt:
			print "*** AT Command \"" + at.command(cmd) + "\" interrupted. ***"
			if args.description:
				print " "

	ser.close()

Here is a sample output for the XBeeQuery utility:

XBeeQuery Script

References
Configuring the XBee Radios

General Documentation

Selecting XBee Radios and Supporting Software Tools

DigiMesh_networking

My ultimate aim is to wirelessly network several Arduino based platforms with a centralized Raspberry Pi controller or gateway.  There is much for me to learn to get this operational, not the least of which is the selection of the radio device platform I plan to use. After reviewing other devices, I have settled on the XBee, in part because of its popularity, its mesh capabilities, and it power management.  To get up to speed on the Xbee, I found the tutorials at AdafruitSparkfun, and Parallax helpful.   Some additional good references are listed at the end of this post.

As I did in the post Raspberry Pi Serial Communication: What, Why, and a Touch of How, I have a desire (obsessive need?) to do some extensive researching before diving into implementing a project.  What is listed below are my research findings.

XBee Series 1 vs. Series 2
Digi’s XBee website gives you a confusing set of options for selecting radios but after reviewing multiple sources, it boils down to the XBee Series 1 vs. Series 2 for the DIY type applications I would do.

  • XBee Series 1 Module (Freescale technology with 802.15.4 firmware) have a 250 kbps RF data rates and operate at 2.4 GHz.  These radios use the IEEE 802.15.4 networking protocol and can perform point-to-multi-point networking.  You can also do peer-to-peer networking (a form of mesh network) but this will require a firmware  upgrade called DigiMesh designed specifically for the Series 1 hardware.
  • XBee Series 2 Module (Ember/Silicon Labs technology with ZigBee firmware) are similar to the Series 1 in many respects but use the ZigBee standard, and  therefore, have the potential for interoperability with devices made by different vendors.

Mesh networking (topology) is a type of networking where each node must not only capture and disseminate its own data, but also may serve as a relay for other nodes, that is, it must collaborate to propagate the data in the network.  This gives the network self configuringself healingdynamic routing, and other capabilities.  Wireless mesh networks can be implemented with various wireless technology including 802.11802.15802.16, cellular technologies or combinations of more than one type.  The mesh enabling capability for these technologies is the routing protocol being used. There are many routing protocols being used by mesh networks today for many different types of products, but I will concern ourselves with just the XBee Series 1 and Series 2 products.

The Zigbee Alliance is a group of more than 300 companies who is responsible for publishing and maintaining the Zigbee specification. In the ZigBee network topology,  there are different node types in the network (Coordinator, Router, End Device). DigiMesh is a proprietary peer-to-peer networking using a simplified topology (no need to define and organize coordinators, routers or end-nodes).  Digi has a white paper that does a nice comparison of the two types of meshing products.

To further clarify the similarity/difference between Series 1 & 2, see the table below:

Feature

XBee Series 1 / 802.15.4

XBee Series 2 / ZigBee

Ramifications

Price

~ $23

~ $26

Price shouldn’t be a driver but If you are looking for a simple point-to-point configuration, you should go with the Series 1. The Series 2 requires considerable setup and configuration.

Transmit Power Output 

Indoor/Urban Range 

Line-of-Sight Range

1 mW (0dbm)

up to 100 ft.

up to 300 ft.

2 mW (+3dbm)

up to 133 ft.

up to 400 ft.

The additional power of the Series 2 give you a ~25% increase in range.

Firmware

Comes standard with 802.15.4 firmware for point-point or star topology. Requires DigiMesh firmware to mesh.

does not offer any 802.15.4-only firmware; it is always running the XBee ZB ZigBee mesh firmware

DigiMesh, while proprietary, appears to have less overhead and easier to configure

Network Topologies

Point-to-Point, Star, and Mesh (with DigiMesh firmware)

Point-to-Point, Star, and Mesh

With a closer reading of the specs, you’ll find that the Series 1 with DigiMesh has a peer-to-peer topology but the Series 2 is hierarchical. I believe peer-to-peer is superior.

Routing Protocol

Ad hoc On-Demand Distance Vector (AODV) Routing + Hierarchical Tree Routing as last resort

Ad hoc On-Demand Distance Vector (AODV) Routing

I suspect there are more differences here but couldn’t uncover them.

RF Data Rate

250 Kbps

250 Kbps

RF data rate doesn’t have much practical meaning

Practical Maximum Throughput

~ 80kbps

ZigBee is significantly slower than the 802.15.4

ZugBee is a full OSI stack, and as a result, has significant overhead.

Power-Down Current

10 uA

1 uA

Series 2 was built for low power consumption.

After pondering this all for a bit, I believe the choose boils down to two questions:

  • Is the interoperability of ZigBee important to you?
  • What are the benefits of ZigBee vs. DigiMesh?

The answer to the first question is “no”.  While I find ZigBee’s interoperability seductive, the practical matter is that I just don’t see that many applications that I could envision integrating into a project.  Maybe some day, but not within my planning horizon.  As to the second question, I don’t see any advantages to ZigBee’s imposed topology of coordinators, routers, and end-nodes.  I believe the homogeneous types of nodes will give my applications the flexibility I may need. Of course, there are many other things one might want to consider, but I think this analysis is sufficient for my needs.  I’m going with the XBee Series 1 Module and I’ll install DigiMesh firmware when it comes time to build meshed networks.

AT Mode vs. API Mode
XBee modules support two modes of operation – AT mode and API mode.  In API mode, you communicate with the radio by sending and receiving packets. In AT (transparent) mode, the XBee radio simply relays serial data to the receiving XBee, as identified by the DH+DL address.  Series 1 radios support both AT and API modes with a single firmware version, allowing you switch between the modes with the  X-CTU software.

To create simple point-to-point links, XBee works nicely in AT mode without much coding. However, if your goal is to build a network consisting of more than two devices, AT mode becomes too difficult to bear. You will spend almost all the time switching in and out of AT mode, wasting time and draining batteries in the process. On the other hand, in API mode commands and data travel in specially formatted frames and no switching is necessary. Another advantage of API mode is that serial speed on transmitters doesn’t have to match – one can be configured for 115,200bps, another for 2400bps, third left with default 9600bps. There is another nice feature called remote command; you can remotely request the state of XBee module pins, for example, or change an output pin level.

It’s clear that I’ll want to work in API mode, but judging from the examples of XBee API mode code, it sure would be nice to have a library package that has designed in some basic utilities that I can leverage.  That is the next topic.

Supporting Software
One of the first packages I discovered was the Xbee Network Protocol (XNP).  I was impressed by the volume and quality of the documentation.  Never the less, I passed on it for two reasons.  First, it isn’t as mature as other packages I discovered (and not widely used), and most importantly, it appears to be implementing mesh networking in software.  Ether the author didn’t recognize that the Series 1 modules can mesh via the DigiMesh firmware (not a surprise since many websites wrongfully report that the Series 1 do not mesh) or he just wants to roll-his-own (what better way to learn about mesh routing protocols).

xbee-arduino is a C++ Arduino library for communicating with XBees in API mode, with support for both Series 1 and Series 2.  Judging from the documentation,  it appears that it could be ported to Raspberry Pi but it would be far easier to use something targeted for the RPi (see below).  With the latest beta software, the XBee’s serial communications can be handle via SoftwareSerial, freeing up the Arduino USB for debugging.  It also appears to that the author is experimenting  if not already supporting DigiMesh. The package is well documented, actively maintained, and equally important, appears to be popular.

Another possibility for the Raspberry Pi is the python-xbee. This Python package is not as well documented as the xbee-arduino but does appear to be actively used and supported. The fact that its on the Python Organization’s web site as a listed package gives it some additional credibility. See this and this with respect to DigiMesh support.

libxbee is another C++ library but targeted at Linux and Windows. Fewer users and the author states “development is coming to an end” may mean this platform isn’t as strong as the others.

While not a very rigorous analysis, I believe I’ll place my bets on xbee-arduino for Arduino development and python-xbee for Raspberry Pi development.  Using Python is intriguing in part because it appears to be the preferred software language for the RPi.  But what if you have some C++ code, a popular language for Linux, and that you want to use some existing libraries? There are tools that could make this happen. You could use Python’s C/C++ extension modules. Also, there is the Simplified Wrapper and Interface Generator (SWIG), which is a software development tool that connects programs written in C/C++ with a variety of high-level programming languages. SWIG is used with different types of target languages including common scripting languages such as Perl, PHP, Python, Tcl and Ruby.

References
An unspoken consideration in my analysis is documentation availability/quality, example implementations  for learning, and the availability of software libraries I could potential use. Here are some of the more interesting things I uncovered.

Tutorials (ZigBee but can be generalized to the XBee)

XBee Documentation

Example Implementations

  • Collection of XBee Projects – Digi’s collection of XBee projects done by the Hacker/DIY community
  • XBee Examples & Guides – Digi’s website giving you step-by-step instructions for simple projects
  • Playing Xbee: Part 1, 2, 3, 4 – author does a nice “teaching tour” on his introduction to XBee

Other Sources

Raspberry Pi Serial Communication: What, Why, and a Touch of How

I started exploring how to get a LCD display operational with a Raspberry Pi (RPi).  I checked out two hardware configurations: the bare bone LCD 16×2 display driven by the HD44780 chip set, which takes 6 wires to operate,  and another configuration that uses a Adafruit shield that requires only 2 wires.  It quickly became clear to me that I knew very little about how the LCD display works (maybe not an important topic for me) and I didn’t fully understands the RPi’s serial communications capabilities (something I must fully understand).  Adafruit does give some nice tutorial that  provide instructions on how to get both configurations work on the RPi, but I don’t like blindly follow tutorials without having a deeper understanding of my options and the underlining hardware.   So what are the serial communication options supported by the Raspberry Pi, under what situations would you use them, and how do you use them?

So I did my research, and for the moment, I’m not going to concern myself about LCD displays but I will dive neck deep into understanding RPi serial communications.  I’ll return to the LCD display topic later.

The Raspberry Pi Board
As our first step, lets take a quick scan of the interfaces we have on RPi for moving data in and out.  The picture below labels the most prominent components on the RPi board.

raspberry-pi

Of course, the “lead actor” on the RPi board is the Broadcom BCM2835, System on a Chip (SoC).  Broadcom bills this chip as a “multimedia applications processor for advanced mobile and embedded applications that require the highest levels of multimedia performance”.  Maybe more importantly for the DIY/Hacker community is that all the firmware running on the chip is now open source.

There is another big chip, that being the SMSC LAN9512/LAN9512i USB  Hub & Ethernet Controller.  This is the major work horse in getting data in/out of the RPi for USB peripheral devices and IP networking.

Now note that most of the board’s other large components are for data I/O.  Specifically, the HDMI, RCA Video, and Audio Out (3.5mm jack) are output only (at least for general purpose data communication perspective), and therefore, not part of our discussion.  On the other hand, the USB and Ethernet ports, are powerful  and widely supported serial communication devices, but I will only lightly touch the USB. I want to focus on serial communication with simple/bare-bone devices, like an LCD 16×2 display, which generally are not interface via something as sophisticated at USB or Ethernet.  But the USB can be used to some for some types of device interfacing, so I will cover that as a final topic.

There are three other board components that concern themselves with data communications but will not be covered in this post:

  • JTAG Header: The JTAG Header is used as a debug port. Embedded systems developer relies on debuggers communicating with chips via the JTAG to perform operations like single stepping and break-pointing.  In time, I suspect people will find ways to exploit these pins for both good & evil, but for now, this is for the RPi hardware development community to use.
  • CSI Camera Connector: The Camera Serial Interface (CSI)  specification is a standard interface between a camera and a host processor for mobile device applications.  This will be where you’ll connect cameras and video devices to the RPi.
  • DSI Display Connector: The Display Serial Interface (DSI) is a specification  aimed at reducing the cost of display sub-systems in a mobile device. This is commonly targeted at LCD and similar display technologies. RPi may be able to interface to some of these.  Some graphical LCD/OLED displays might be attached to it.  DSI would seem like a good candidate for implementing my LCD display but there is a problem.  It appears the DSI isn’t supported at this time.

This leaves one remaining board input/output component standing, that be the General Purpose Input/Output (GPIO).  On some electronics boards, GPIO pins have no special purpose defined, and some pins go unused by default.  The RPi developer has identified a handful of digital control lines and provided services on them for you to use.  By having these available can save you the hassle of having to arrange additional circuitry to provide them or implement functionality in software.

Raspberry Pi’s GPIO as a Data Bus
The first thing to get your head around, is how is data moved around the RPi, or in any general purpose computer.  In the most general sense in electronics, a bus or data bus is used to move data words of any type from one place to another. Computing is based on data words made up of collections of data bits. These “words” can contain as few as four data bits and often much larger.

The task of a bus designer is to devise circuitry that passes these data words from one circuit to another.  These words can be communicated serially  (i.e. serial communications) or in parallel.

  • Serial Bus: The least expensive method in terms of wire cost is to send the bits one at a time over a single pair of wires. This is called serial data transmission.  Data words start as sets of bits that exist in parallel. In order to ship these words on a serial basis they must be converted to a serial stream of bits at the transmit end and then reconverted to a parallel word at the receive end. The common name for the circuitry that does this conversion is a SerDes circuit which stands for serializer/deserializer. Integrated circuits are more expensive when they have more pins. To reduce the number of pins in a package, many ICs use a serial bus to transfer data when speed is not important. Some examples of such low-cost serial buses include Serial Peripheral Interface (SPI)Inter-Integrated Circuit (I²C)UNI/O, and 1-Wire.
  • Parallel Bus: At some point, it is more cost effective to add a wire for each bit in the word and send it in parallel on a data bus. Parallel buses have a limited data rate and distance at which they can be reliably run (more so than a serial bus).  Some widely used parallel bus standards are Parallel Bus Interface (PBI), Peripheral Component Interface (PCI), Small Computer Systems Interface (SCSI), the VMEbus used in instrumentation, the Rambus interface used in memories, and others.

As the name implies, GPIO pins can be configured through software  to provide some specific function or purpose within the hardware device design.  The GPIO pins connect directly into the core of the processor, and the Raspberry Pi developers implemented several alternate functions for the GPIO pins.  Several are desirable because of the multiple standards and types of devices you may wish to interface.  On boot-up,  the RPi board GPIO is in alternate function state “ALT0″ and will support I2C, SPI, and UART.  This is shown below:

Raspberry Pi (Rev 1) GPIO Pin Out

It can be confusing to call the RPi’s whole 26 pin array GPIO and also some specific pins GPIO.   In reality, all the GPIO pins can be reconfigured to provide alternate functions. 

While PCM isn’t a topic for this posting, in the figure above, you’ll also see references to PCM on pins 18 & 21.  PCM stands for Pulse-code modulation and is s a method used to digitally represent analog signals.  It is often used to control light intensity or motors.  RPi’s native PCM capabilities are not well documented but it appears that people have has some success.  Generally, to do anything useful, you need multiple PCM channels so people have resorted to adding hardware or software to get the desired functionality from the RPi.  My guess is these RPi pins don’t have much of a future.

I2C, SPI, UART … Say what?
Much has been lightly covered so far, including terms like I2C, SPI, and UART.  So what is the significance?  Well, I2C, SPI, and UART are the heart of our quest to understand RPi’s serial communications capability. Via their exposure on the GPIO pins, these capabilities are what can be used to integrate things like LCD displays to the RPi.  Now lets dive deeper into each one of them.

Universal Asynchronous Receiver/Transmitter (UART)
The Universal Asynchronous Receiver/Transmitter (UART) takes bytes of data and transmits the individual bits in a sequential fashion. The device changes processor’s parallel information to serial data which can be sent on a communication line. A second UART (maybe on another processor)  can be used to receive the serial information. The UART performs all the tasks, timing, parity checking, etc. needed for the communication. The only extra devices attached are line driver chips capable of transforming the TTL level signals (0/5 volts) to line voltages (on RS-232 line this could be as +/- 25 volts) and vice versa.

Each UART contains a shift register, which is the fundamental method of conversion between serial and parallel forms. Serial transmission of digital information (bits) through a single wire or other medium is much more cost effective than parallel transmission through multiple wires.  The UARTs transmit/receive one bit at a time at a specified data rate (i.e. 9600bps, 115,200bps, etc.). This method of serial communication is sometimes referred to as TTL serial communications.

Asynchronous transmission allows data to be transmitted without the sender having to send a clock signal to the receiver. Instead, the sender and receiver must agree on timing parameters in advance and special bits are added to each word which are used to synchronize the sending and receiving units. When a word is given to the UART for Asynchronous transmissions, a bit called the “Start Bit” is added to the beginning of each word that is to be transmitted. The Start Bit is used to alert the receiver that a word of data is about to be sent, and to force the clock in the receiver into synchronization with the clock in the transmitter.

For a further description of synchronous and asynchronous line communications, check out this tutorial.

Raspberry Pi’s Mini-UART
Warning: Misleading information ahead – See Warren Gay’s comments.
The Raspberry Pi actually has two UARTs. One UART  is part of the internal ARM architecture of the Broadcom BCM2835 chip, in the core of the Raspberry Pi and not accessible externally.  The other UART is sometimes called the RPi’s “Serial Port” (even thou the USB supports serial communications, and therefore a serial port).  The serial port being reference here is serviced by a UART, sometime refereed to as the  “Mini-UART” since it doesn’t appear to be very rich in functionality.  It is basically be used as a console port for access to the Raspberry Pi.  The serial console is a convenient way to interact with the Raspberry Pi for debugging or your network is down and it is the destination of console messages (including boot-up messages).  From the Raspberry Pi pinout and the eLinux wiki, I can see that the serial port (aka Mini-UART) on the Pi is on GPIO Pin 14 (TX) and GPIO Pin 15 (RX):

Mini-UART

Since the GPIO pins give access to the Mini UART, you can establish a serial console, which can be used to log in to the Pi, and many other things.  However, normal console device communicate with -12V (logical “1″) and +12V (logical “0″) RS-232, which may just fry something in the 3.3V Pi. Even “TTL level” serial at 5V runs the same risk.  See this tutorial for one example on how to build 3.3V to RS-232 levels converter with a MAX3232CPE and a few passive components.

You can reconfigure the RPi so that the Mini UART isn’t acting as a serial console and use it for outer purposes (e.g. communicate with an attached Arduino or Xbee).  Using the Raspberry Pi’s serial port requires some Linux reconfiguration and the abandonment of the serial console, and potentially some level conversion, but it could be useful. The Mini-UART pins to provide access to Linux’s /dev/ttyAMA0 serial port.  To be able to use the serial port to connect and talk to other devices, the serial port console login needs to be disabled and the post “Raspberry Pi and the Serial Port” shows you how.

Again, keep in mind that RX and TX lines are available on the GPIOs but operate at 3.3 volts. You’ll need a board or cable to level convert 3.3 volt UART signals to connect with other devices (e.g. RS-232, USB).

Serial Peripheral Interface Bus (SPI) — aka 4-Wire Serial Bus
The Serial Peripheral Interface Bus or SPI (pronounced as either S-P-I or spy) bus is a synchronous serial data link standard, named by Motorola, that operates in full duplex mode.  SPI is much simpler than I2C. Master and slave are linked by three data wires, usually called MISO, (Master in, Slave out), MOSI (Master out, Slave in), the SCLK clock line (sometimes called M-CLK), and an optional SS (Slave Select; sometimes known as the Chip Select or CS line or Chip Enable or CE line) is the slave select or chip select line.  Its optional only if you have one slave, otherwise one or more SS lines are provided.  The Raspberry Pi has two Slave Select lines: CE0 and CE1.

SPI Diagram and Sescription

Usually the transfer sequence consist of driving the SS line low, sending X number of clock signals with the proper polarity and phase, then driving the SS line high to end the communication. As the clock signals are generated, data is transferred in both directions, therefore in a “transmit only” system the received bytes have to be discarded and in a “receive only” system a dummy byte has to be transmitted.

Many SPI-enabled ICs and Microcontrollers can cope with data rates of over 10MHz, so transfer is much faster than with I2C. Since it is synchronous communications, it is not limited to 8-bit words so you can send any message sizes with arbitrary content and purpose. The SPI interface does not require pull-up resistors, which translates to lower power consumption. The downside is that SPI normally has no addressing capability; instead, devices are selected by means of a SS signal which the master can use to enable one slave out of several connected to the SPI bus. If more than one slave exists, one chip select line is required per device, which can use precious GPIO lines on the Master.

Inter-Integrated Circuit (I2C) — aka 2-Wire Serial Bus
Inter-Integrated Circuit or I2C (pronounced as either I-squared-C or I-2-C) is generically referred to as a “two-wire interface”.  It’s a multi-master serial single-ended computer bus invented by Philips that is used to attach low-speed peripherals to a motherboard, embedded system, cellphone, or other electronic device.

I2C can be used to connect up to 127 nodes via a bus has two data wires, called SCL and SDA.  SCL is the clock line. It is used to synchronize all data transfers over the I2C bus. SDA is the data line. Of course, there is a third wire being ground. There may also be a 5 volt wire to distribute power to the devices. Both SCL and SDA lines are “open drain” drivers. What this means is that the chip can drive its output low, but it cannot drive it high.  For the line to be able to go high you must provide pull-up resistors to the 5v supply. There should be a resistor from the SCL line to the 5v line and another from the SDA line to the 5v line. The value of the resistors is not critical.  Anything from 1800 ohms to 47K ohms used (1.8K, 47K and 10K are common values). You only need one set of pull-up resistors for the whole I2C bus, not for each device, as illustrated below:

I2C Diagram and Description

In theory the I2C bus can support multiple masters, but most micro-controllers can’t. A master is usually a microcontroller, although it doesn’t have to be. Slaves can be ICs or microcontrollers.  When the master wishes to communicate with a slave it sends a series of pulses down the SDA and SCL lines. The data that is sent includes an address that identifies the slave with which the master needs to interact. Addresses take 7 bits out of a data byte; the remaining bit specifies whether the master wishes to read (get data from a slave) or write (send data to a slave).

Some devices have an address that is entirely fixed by the manufacturer; others can be configured to take one of a range of possible addresses. When a micro-controller is used as a slave it is normally possible to configure its address by software, and for that address to take on any of the 127 possible values.  The address byte may be followed by one or more byes of data, which may go from master to slave or from slave to master.

When data is being sent on the SDA line, clock pulses are sent on the SCL line to keep master and slave synchronised. Since the data is sent one bit at a time, the data transfer rate is one eighth of the clock rate. The original standard specified a standard clock rate of 100KHz, and most I2C chips and micro-controllers can support this. Later updates to the standard introduced a fast speed of 400KHz and a high speed of 1.7 or 3.4 MHz.  The Arduino and Raspberry Pi can support standard and fast speeds.

The fast rate corresponds to a data transfer rate of 50K bytes/sec which is too slow for some control applications. One option in that case is to use SPI instead of I2C.

1-Wire — aka 1-Wire Serial Bus
On a 1-Wire bus (sometime refered to as a  ”MicroLan”), a single master device communicates with one or more 1-Wire slave devices over a single data line, which can also be used to provide power to the slave devices. Devices drawing power from the 1-wire bus are said to be operating inparasitic power mode.  When operating in parasite power mode, only two wires are required: one data wire, and ground. At the master, a 4.7k pull-up resistor must be connected to the 1-wire bus. With an external supply, three wires are required: the bus wire, ground, and power. The 4.7k pull-up resistor is still required on the bus wire.

1-Wire Diagram and Description

Each 1-Wire device contains a unique 64-bit  code, consisting of an 8-bit family code, a 48-bit serial number, and an 8-bit CRC. Before sending a command to a slave device, the master must first select that device using its code.

How do you use I2C, SPI, UART, or 1-Wire on the Raspberry Pi?
Now that we know the what & why for serial communications options on the Raspberry Pi, how do we use them?  This topic deserves technical details and examples but this post has already run too long.  I’m likely to do some specific implementation in the future, but for now I’ll reference some sources of information on the web.

First, lets be clear about the RPi software distribution I’m using, since not all will be supporting all these serial communications options.  I’m using Adafruit’s Occidentalis distribution (based on “Wheezy”) which comes with hardware SPI, I2C, and 1-wire support. In the Occidentalis distribution, Adafruit has included in the Linux kernel the needed drivers. SPI and I2C has been implement on the GPIO pins as outline above.  RPi doesn’t have a predetermined GPIO pin assignment for 1-Wire, but Adafruit choose GPIO pin 4 for 1-Wire.  Note that this unassigned GPCLK0 (General Purpose Clock Voltage) function.

Given you have the Occidentalis distribution, you can check on the installation of I2C, SPI, and 1-Wire via the following:

  • To validate I2C, connect any I2C device to power, ground, SDA and SCL. Then run  sudo i2cdetect -y 0 to detect which addresses are on the bus. i2cdetect is a program to scan an I2C bus for devices.  Also, you can list the I2C device drivers via the command ls /dev/*i2c*.  This illustrates that RPi supports two I2C buses, 0 and 1.
  • To validate SPI, the command ls /dev/*spi* will list two SPI devices, one each for the 0 & 1 SS lines.  To go further, use the  spidev_test.c tool described in Getting SPI working on the Raspberry Pi.
  • Occidentalis implementation of 1-Wire isn’t done via a kernel installed driver but bitbanged.  Adafruit states that the implementation is  ”flakier than SPI or I2C” and they don’t have any tutorials.  Maybe 1-Wire should be shelved for now.

There are some good RPi SPI & I2C tutorials on Adafruits.  There are others on personal blogs but generally they are scares right now.

Attach Peripherals to the Raspberry Pi
Now that we know the in’s & out’s of serial communications with the RPi, what can be done to make the physical aspects of interfacing the board easier.  Dealing with the GPIO pins on the board can be a pain.  One solution is the Pi Cobbler which plug the GPIO pins into a solderless breadboard via a ribbon cable.  My personal favorite is the Pi Crust.  In this case, the confusing layout of  GPIO pins are much more clearly organized and supplied with more useful female headers.  A whole development environment will be supplied via PiFace Digital and Gertboard … when they become available.

The USB Serial Ports
As promised, the final topic is the USB port. The USB can be used for some types of device interfacing, particularly if you make it look like a simple serial port to the device.  The conventional serial port (not the newer USB port) is a very old I/O (Input/Output) port. It’s slow compared to newer Universal Serial Bus (USB) serial devices, but conventional serial ports are still in use and many devices you’ll hook to your RPi will want to use them.  Until around 2006, most new desktop PC’s had one but has been largely replaced by the USB. Conventional serial ports are still widely used in embedded systems, but the RPi choose to use the USB.  Never the less, it is possible to put a conventional serial port device on the USB bus by using a Serial to USB Adapter hardware or cables.  This could be necessary for some of the hardware hacking you’ll do with an RPi.  (For example, I’ll be posting a project where I’ll use the RPi’s USB to talk to a XBee radio.)

USB does synchronous communications (synchronous means that bytes are sent out at a constant rate one after another in step with a clock signal tick) and transmits in special packets like a network. Conventional serial ports are typically asynchronous (i.e. “not synchronous”). Just like a network, USB can have several devices physically attached to it, including serial ports. Each device on it gets a time-slice of exclusive use for a short time. A device can also be guaranteed the use of the bus at fixed intervals. One device can monopolize it if no other device wants to use it.

Under Linux, each and every hardware device, including USB ports, are treated as a file and call a device file.  A device file allows a user to access hardware devices, but shields the users from the technical details about the hardware. (This is unlike what we’ll see for the RPi GPIO interfaces where hardware level technical details must be address directly.) Under Linux, a conventional serial port will typically have a device file such as /dev/ttyS0, /dev/ttyS1, etc. but the USB serial ports will appear as /dev/ttyUSB0, /dev/ttyUSB1, etc.  When your device is plugged in, Linux assigns the filename as it sees fit and isn’t always predicable (it doesn’t have to be this way). If you need to know what device file your serial device is connected too (and your software often needs to know), using a combination of the commands lsusb, dmesg, and grep, plus some basic insight, will often do the trick.

The lsusb command can show you the hubs connected to your system, but you won’t necessarily see any entries in /dev until you plug something into it, and what that entry may be will be dependent upon the type of device you are plugging in.  The system doesn’t recognize new USB devices right away. It can take from a couple of seconds to as much as a minute.  If I plug my Serial to USB Adapter cable, using the lsusb command I can identify the cables .  Now using dmesg, and grepping for the “Manufacturer”, I can get the “FTDI”.  Now I grep again for “FTDI” to get the device file name “ttyUSB0“.  This is all illustrated below:

lsusb and dmesg

Speech Synthesis on the Raspberry Pi

Now that I can get sound out of my Raspberry Pi (RPi), the next logical step for me is speech synthesis … Right?  I foresee my RPi being used as a controller/gateway for other devices (e.g. RPi or Arduino).  In that capacity, I want the RPi to provide status via email, SMS, web updates, and so why not speech?  Therefore, I’m looking for a good text-to-speech tool that will work nicely with my RPi.

The two dominate free speech synthesis tools for Linux are eSpeak and Festival (which has a light-weight version called Flite). Both tools appear very popular, well supported, and produce quality voices.  I sensed that Festival is more feature reach and configurable, so I went with it.

To install Festival and Flite (which doesn’t require Festival to be installed), use the following command:

sudo apt-get install festival
sudo apt-get install flite

Festival
To test out the installation, check out Festival’s man page, and execute the following:

echo "Why are you in front of the computer?" | festival --tts
date '+%A, %B %e, %Y' | festival --tts
festival --tts Gettysburg_Address.txt

Text, WAV, Mp3 Utilities
Festival also supplies a tool for converting text files into WAV files.  This tool, called text2wave can be executes as follows:

text2wave -o HAL.wav HAL.txt
aplay HAL.wav

MP3 files can be 5 to 10 times smaller than WAV files, so it might be nice to convert a WAV file to MP3.  You can do this via a tool called lame.

lame HAL.wav HAL.mp3

Flite
Flite (festival-lite) is a small, fast run-time synthesis engine developed using Festival for small embedded machines. Taking a famous quote from HAL, the computer in the movie “2001: A Space Odyssey

flite "Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over."

Depending how the software was built for the package, you find that flite (and festival) has multiple voices. To find what voices where built in, use the command

flite -lv

To play a specific voice from the list, use the -voice parameter in the command

flite -voice kal "I'm now speaking kal's voice. By the way, please call me Dr. Hawking."

In my case, the default voice appears to be “kal”, which sounds somewhat like Stephen Hawking.  ”slt” appears to be a female version of the “kal” voice.

flite_time is a talking clock that can speak things like “The time is now, exactly two, in the afternoon.”

flite_time 14:00
flite_time `date +%H:%M`

Documentation
The documentation for Festival and Flite isn’t all that great but there does seem to be multiple sources.  Here is what I found most useful:

Blog Stats

  • 66,804 hits

Join 25 other followers

Jeff’s Skinner Box Gravatar

Follow

Get every new post delivered to your Inbox.

Join 25 other followers

%d bloggers like this: