Cell Annoyances: Simple, Clear, Direct, and Concise

Found this article "Peering Inside a Mobile Phone Network" written by Rich Mogull. It was very well written and cleared up some things that I did not know about the cell networks. I had the grave misconception that the cell networks are like IP networks; oh no sir. The truth is that they are much different. The article sort of takes the view that we (users of these networks) should think about cutting the operators and designers of these networks some slack. I say no way on that. Not our (users of these networks) fault that the cell networks are woefully inadequate to the task. Alas, the article described (from a very high level) what was taking place and it is worth a read even if you are very technical already.

Python Code:: Sort by Key and Subkey in a List of Dictionaries

Here is a semi clear way to perform this function. There is probably a more elegant way to do this or even some snazzy library method, but this shows some of the guts of how this might be done. I am confident this can be rewritten to perform this is linear time or better. Again this is quick and dirty, like "Ah crap, I need this now and I can optimize later" type of thing:

def sortResults(self, unsort_rs, orderby, desc, subsortkey=None):
   list_rs = list(unsort_rs)
   result_rs = sorted(list_rs, key=itemgetter(orderby))
   if subsortkey:
     sub_list, complete_list = [], []
     for i in xrange(len(result_rs)):
       sub_list.append(result_rs[i])
       try:
          if result_rs[i+1][orderby] != result_rs[i][orderby]:
            complete_list += sorted(sub_list, key=itemgetter(subsortkey))
            sub_list = []
        except IndexError:
            complete_list += sorted(sub_list, key=itemgetter(subsortkey))
     result_rs = complete_list
   if desc == 1:
     result_rs.reverse()
   sorted_rs = tuple(result_rs)
   return sorted_rs

Ipsecuritas, must allow ICMP, MAC OS X VPN

So we had this box nonstop pinging a server. This IP was private, so it was easy to tell it was from the VPN zone of the firewall, but I could not tell why someone was doing this. So I filtered this out and waited for the calls to come in. Well, the call came in about someone getting time outs from this server when pulling web traffic across the VPN. Surely, web traffic has nothing to do with ICMP that I filtered. Well, I would be WRONG. It turns out that Ipsecuritas VPN client uses a nonstop ping (once every 3 seconds) to a LAN host that it previously had traffic to in order to keep its tunnel open. Otherwise, IPsecuritas (client side) will tear the tunnel down. Game over for the VPN connection. Bunk! Shame on you Ipsecuritas (or Apple), are you so ghetto that you need to do this? It just seems so bush league. From a sysadmin point of view, you can't think of a better way to do this? How about a proper keep alive packet to the firewall?

Sharing stuff

Ok, how can I be of service to someone else? How can I enrich someone else life? How can I ease the suffering of others? These are the questions that I ask in my life. In pursuit of this, I think about experiences that I have had. For some reason, I think it noble to impart any scraps of wisdom that I have picked up along the way. Maybe, I can impart wisdom in the fashion of, "hey I did this and it hurt, so be smart and don't do it, at least not like I did".

With all this in mind, I am going start a series of "Life on nuclear submarine", so that people can get a sense of the tribulations being a crew member. This was a time when I was constantly undergoing strife. Maybe lessons can be taught about not being a sailor, or maybe there are just lessons about being in close proximity to 100 other men.

Bikes for Christmas

So I started this charity project called "www.bikesforchristmas.org". I totally forgot to explain what that was on this forum. Maybe some marketing value will come my way. Here is the thing: It is just me giving bikes away for Christmas. That's it. You got a youngster that needs a new bike, just sign up and I will provide.


Get a bike!

Bacula: Understanding its Pool Resources

I back up about 20 different data sets daily. Each set exists on a separate machine somewhere on our network. I struggled a bit to get Bacula to do what I wanted it to do. Maybe I was looking at the problem wrong, but I felt as though I should formulate my own back up strategies and work with Bacula's configuration to devise the tactics. It seems that you should strategize around Bacula's common usage to make it easier on yourself. Bunk! Bacula does the job, but you need to understand the configuration settings well to get Bacula to do what you really want it to.

First, the pool resource section of the bacula director configuration is a good place to start. This is the stuff I monkey with the most.

My strategy (not thinking about how Bacula does things). I want the following:

1. on the first Monday of the month I want a FULL backup of all data sets.
2. on the 3rd Monday, I want a differential backup everywhere
3. every night other than those two, I want incrementals done.
4. I want a pool for each machine (each data set).
5. I want a maximum of 2 volumes written for each data set
6. the data set will be a month of backups (FULL backup, incrementals, differential, incrementals)
7. once 2 volumes are written out in the above nature, I want the oldest volume recycled. This gives us 2 full months of backups, at best and 1 full month backup at worst, depending on where we are in the backup cycle.

To make this happen, you must do the following:

1. set your "Maximum Volumes" to 2 for all the pools (in pools resource section of bacula-dir.conf)
2. set your "Volume Use Duration" to 1 month.
3. set your "RecycleOldestVolume" to yes
4. set your "Recycle" to yes
5. set your "Purge Oldest Volume" to no
6. set your "AutoPrune" to yes
7. I recommend letting bacula auto-name your Volumes. I mean the pool has a descriptive name, so who cares what the volume is named? So set "LabelFormat" to "vol"

Then g'head and fire off your backups for a month. Then come back to the config and change one setting once your pools each have 2 volumes written in them. In this case 32 days after starting your backups. In bacula-dir.conf change the setting for "Purge Oldest Volume" to yes by doing the following:

1. edit your bacula-dir.conf file for each pool and restart the bacula director.
2. update each Volume in the already written volumes to honor this by issuing a "update volume" at the bconsole and changing this parameter for each volume.

Now you have a self-rotation backup schedule that will maintain itself, more of less.

VirtualBox PXE Boot or VirtualBox PXE / TFTP

Just started using Sun's VirtualBox. VMware, Parallels ought to shaking in their closed sourced boots. It was easier to install and configure than Parallels and VMware and is FREE (as in beer). Sun even released a open source version. Parallels is such a bane running on Mac OS X, but VirtualBox seems to run smoother. I was even able to PXE boot a guest OS without using a floppy image like I have to do with my older version of Parallels.

A couple of things make VirtualBox different than other hypervisors. First, at least on the mac version of VirutalBox (VB), the networking differs in the following ways. This is in the manual too.

1. The network between the host OS (the OS that is native to the machine and that VB is installed on) and the guest OS (the OS that you are installing inside VB) is implemented in user mode. What this means is that VB does not reach into the host OS kernel and try to glob onto the network stack, Instead, VB just takes network traffic from its guest OS's and sends it the same way any other app on the host does. VB decided to do this using NAT by default. So VB just needs to present one logical "port" to the host OS for the network. So you can think of the VB hypervisor as a router, just like the one you may use at home to have multiple computers behind a cable/DSL modem. So the hypervisor has the ip of the host OS and it masquerades all its guest OS's as this ip. Pretty easy, makes sense, but definitely less convenient.

2. You can "bridge" the network connections to your guest OS's too. This is more common in paravirualization and virtualization. There are directions to do this in the manual. There appears to be no way to do this for VB for Mac OS X (using VB on Mac OS X host OS). Oh well, maybe in later versions. You can still port forward while you use NAT. I would not need this feature at this time anyway.

3. I installed VB on Mac OS X host OS and then PXE booted the debian etch installer and noted some things that can save you some troubleshooting time. First, you must make a TFTP directory and copy your debian installer files and pxelinux.0 image there. Also, and this tripped me up good, you CANNOT pxe boot into an image that has a space in it. For example, I named my VB image "warehouse image". So when pxelinux tried to go and find its configuration (by default: pxelinux.cfg/), VB passed it a directory prefix of "warehouse" WITHOUT the 'image" part. Bad news. I renamed the image to just "warehouse" and it worked. So here is the list of what to do to make VB PXE boot.

a. create an image in VB (be sure not to use white space in image name, use foo and NOT foo bar)
b. configure the boot options to boot from network
c. create a new directory called TFTP in ~/Library/VirtualBox/
d. on your mac, rename "pxelinux.0" to foo.pxe and copy it to ~/Library/VirtualBox/TFTP/
e. copy the entire directories of pxelinux.cfg and debian-installer to ~/Library/VirutalBox/TFTP/
f. the pxe system can be obtained from debian in a netboot file called netboot.tar.gz (this contains all the files you need to pxe boot into a debian installer).


A couple of things I noted AFTER debian was installed and I reboot into it. I reboot after everything was installed and since it was set to network boot before, it tried to network boot again. This time, it got an IP from the internal VB dhcp server (yes, the hypervisor runs an internal dhcp server), it then found the tftp (next server) and boot attempted to find a configuration inside pxelinux.cfg but COULD NOT because "TFTP server does not support the tsize option". OK so I let it sit and about 10 minutes later, it booted anyway! I have seen this tsize error before and it is well documented in the pxelinux FAQ's. They recommend using a TFTP server that supports the tsize option such as "tftpd-hpa", but why the VB tftp server worked before, but not after the debian install is a mystery. Installing an OS should not have anything to do with this tftp server.