Tail recursion in Python

Problem: When marshalling an object to be able to transmit to or from an XMLRPC server and the xmlrpc lib is insufficient to the task because the objects within the main object do not implement a marshal method.

Admittedly the datetime, mxDateTime and None objects should just know what to do if asked by the xmlrpclib to marshal themselves, but in this case, I want to do this myself.

Answer: Here is a tail recursive way to perform this task. This should not use any stack space, but I cannot confirm this since I am not sure what is going on in the interpreter.

def __convertNonMarshalables(self, obj):
  if type(obj) == type({}):
    for key, value in obj.items():
       if value is None:
        obj[key] = ""
      elif type(value) is type(mx.DateTime.now()) or type(value) is type(datetime.datetime.now()):
        obj[key] = value.strftime('%Y-%m-%d')
      elif type(value) == type([]) or type(value) == type({}):
        value = self.__convertNonMarshalables(value)
  elif type(obj) == type([]):
    for value in obj:
      if value is None:
        value = ""
      elif type(value) is type(mx.DateTime.now()) or type(value) is type(datetime.datetime.now()):
        value = value.strftime('%Y-%m-%d')
      elif type(value) == type([]) or type(value) == type({}):
        value = self.__convertNonMarshalables(value)
  else:
    if obj is None:
      obj = ""
  return obj

Leopard killed TextMate, sort of

Just updated Leopard and now having "#!/bin/sh" in the "command(s)" section of the bundle editor for any bundle yields "bad interpreter: no such file or directory" if I remove the directive, it works fine, but now on the shell bundle, the run command script is a ruby script and TextMate will not take #!/usr/bin/env ruby, yields bad interpreter again, but I really need to tell TextMate this is a ruby script. So I ran the script with ruby -e'do something'. That is weak sauce but it works. I also noticed circular dependencies in the bundles. You need ruby to run a python script in the python bundle; you need bash to run a ruby script, you need ruby to run a shell script.

I am not sure why this is because the text in command(s) should be treated as a shell script. Anyway, if anyone else has this problem, then removing the shell directive line should help.

Functionalness

An object that has no function does not exist.

Auto adding DHCP host names to DNS (MaraDNS)

Ever wonder why a host that you offer a DHCP lease to just does not automagicly appear in your non recursive DNS server's zone file. Well, it should! Using this "glue" code, it will be a snap to update your zone files whenever a new host joins the network. In this example, I am using MaraDNS and dhcpd.

Overall concept:

dhcpd --> lease file --> cron script --> ruby script --> DNS zone file

Cron entry (/etc/cron.d/update_dns):

*/5 * * * * root sh /usr/local/bin/parse_leases_add_to_dns.sh

parse_leases_add_to_dns.sh:

#!/bin/bash

#check modification time
if [ ! -f /tmp/leases_mod_time ]; then
touch /tmp/leases_mod_time
fi

#see if the last check time is less than mod time
if [ /tmp/leases_mod_time -ot /var/lib/dhcp/dhcpd.leases ]; then
/usr/local/bin/ruby add_hosts.rb
touch /tmp/leases_mod_time
fi

add_hosts.rb:

leases = open('/var/lib/dhcp/dhcpd.leases')
dns = open('/etc/maradns/kleetus.1337.foo', 'w')

leases_string = leases.read
lease = ""
client_hostname = ""

leases_string.each do |line|
next if line[0..0]=="#"
unless (line =~ /lease/).nil?
lease = line.split[1].gsub("\"","").gsub(";","")
end
unless (line =~ /client-hostname/).nil?
client_hostname = line.split[1].gsub("\"","").gsub(";","")
end
if client_hostname!="" and lease!=""
w_string = "#{client_hostname}.% #{lease}"
dns.write("#{w_string}\n")
lease = ""
client_hostname = ""
end
end

statics = ["myth-test.% 10.20.10.120", "*.% 10.20.10.100"]

statics.each do |line|
dns.write("#{line}\n")
end

dns.close

load=`/etc/init.d/maradns force-reload`

So why are you using a ruby script, can't you just do it all with bash/awk?

Sure, but the ruby script is more fun to write and maintain; if you feel the need to use bash, then this is possible too.

Adam Carolla

If you have never heard of this man, you are for a treat, my friend! Adam is the Quintessential everyman's man. He can be heard here

Here are the reasons why I love Adam Carolla:

1. He connects with his listeners on a personal level because he doesn't act like some stuck up rich person. He recognizes that most of his listeners are just like he is. He worked construction and cleaned carpet. Now he has means, so it is nice that he can create a portal into the world of Hollywood, but he always remains genuine. He is sincerely connected to "the rest of us".

2. He is honest. This really should be number 1 on my list. You cannot connect with people if you are phony. People can see through this in a second and radio people are fooling themselves if they think that they put on a persona and retain an audience. Ace Man (Carolla) just puts himself out there, take it or leave it. He has flaws and those flaws can be endearing since we all have them too.

3. He is observant. This is more than schtick, Adam brings out the truly absurd in our culture. He comes off more genuine that the Seinfeld ("Who are these people") bits. He gets into more counter-culture routines that are really funny. He has one bit where is mimes a strip club MC. Callers will ask him to do an impromptu routine centered around them. Great bit.

4. Connections. Adam is plugged into Hollywood and its weirdness. He will call out weird shit too. Life is all about relationships. Adam's personality is likeable and most pop culture stars like Adam, so he can leverage this.

5. Adam is old school So. California. He was born in North Hollywood and lived amoung the regional culture. I find people who talk about subculture amoung the most interesting. I live in Atlanta, so this is far removed from this, but I love to hear about how "the Valley" is like or dislike my area.

How to get ffmpeg to transcode your over the air mpeg2ts to iphone 3G

I got this figured out, so I thought I would share this with anyone transcoding mpeg2ts (over the air high definition) to something that the iphone 3G can play from a streaming server such as apache. Here is my ffmpeg line:

ffmpeg -i "${directory}/${file}" -async 3 -r 29.97 -acodec libfaac -ar ${rate}
-ab ${abitrate} -ac 2 -s ${width}x${height} -vcodec mpeg4 -b ${vbitrate} -flags
+aic+mv4 -mbd 2 -cmp 2 -subcmp 2 -g 250 -maxrate 512k -bufsize 2M -title "${file}"
 "${directory}/${file}.mp4"

The key here is to ensure the playback is at no more than 30 frames a second, or your iphone will play like 1 second and then stop...like it can't buffer the stream. Of course, when you do this, you will lose audio/video sync since you just yanked some frames out out the stream, so you need to use -async 3 to fix this; 3 works for me, but you may want to experiment. Also, I am using a recent SVN trunk compilation of ffmpeg with libfaac and xvid enabled in the compilation. This made a huge difference in whether ffmpeg could work with me on this TS file.

Now the bash like variables are exactly that, bash variable interpolations. Rate is the sample frequency of the audio...something like 44100 or 48000, abirate is the audio bit rate...something like 128k, the vbirate is not crucial, the video bit rate, something like 480K or 378K. The vbirate will really make the resulting file smallish. The original was prolly 2 or 3 times that. The dimensions usually work out to be 320x240, but I have also used 432x240 with success for 1.78 ratios.

Ipsecuritas, Mac OS X, Sonicwall Enhanced Firmware

This was a tricky one. I always had some difficulty getting the free IPsecuritas connected; so I am sure others had problems too.

Using IPsecuritas 3.2 build 2501
MacBook Leopard

connecting to a:

TZ 190 Wireless Enhanced
SonicOS Enhanced 4.0.1.3-46e

Sonicwall side config (straight out of the tech support report)
--- SA 1 ---
Authentication Method : IKE with Preshared secret
VPN Policy Name : "WAN GroupVPN"; enabled
Policy Type : Client Policy
Pre-shared Key len : 14, value=
IKE Local Id : UNKNOWN
IKE Remote Id : ID_FQDN: (GroupVPN)
Local network :
Peer network :
IKE Exchange : Aggressive Mode
IKE Proposal : DH Group 2; Encrypt/Auth - 3DES/SHA1
IKE SA Life time : 28800 (seconds)
IPsec Proposal : DH Group 2; Encrypt/Auth - ESP: 3DES/HMAC SHA1
Ipsec SA Life time : 28800 (seconds)
Policy Options : PFS: on; Xauth: on; Netbios: on; Multicast: off
Management : HTTP: n; HTTPS: n; SSH: n
XAUTH user group : Trusted Users
Default LAN gateway : (0.0.0.0)
VPN policy : Bound to zone WAN

WAN GroupVPN Client Settings:
User Name and Password Caching:
XAUTH User Authentication is Required
Cache XAUTH User Name and Password on Client: Never
Client Connections:
Virtual Adapter Settings: DHCP Lease or Manual Configuration
Allow Connections to Split Tunnels
Set Default Route as this Gateway is Not Selected
Apply VPN Access Control List is Not Selected
Personal Firewall on Client Machine is Not Required
Client Initial Provisioning:
Use Default Key for Simple Client Provisioning is Selected
-----------------------------

Now the ipsecuritas config

General Tab:
--------------------
Remote IPSec Device: IP or host name of Sonicwall (must be reachable from Internet)
Endpoint Mode: Host (IP Address left blank)
Remote Mode: Network (Internal LAN network of the Sonicwall, such as 10.0.1.0 CIDR/Mask 24)

Phase 1:
--------------------
Lifetime: 8 hours
DH Group: 1024 (2)
Encryption: 3DES
Authentication: SHA-1
Exchange Mode: Aggressive
Proposal Check: Claim
Nonce Size: 16

Phase 2:
--------------------
Lifetime: 8 hours
PFS Group: 1024 (2)
Encryption: 3DES
Authenication: HMAC SHA-1

ID:
--------------------
Local Identifier: Address
Remote Identifier: FQDN.... just fill in the "Unique Firewall Identifier" from the Sonicwall VPN section
Authentication Method: XAuth PSK
Preshared Key:
Username: XAuth username
Store Password: checked if you would like the password to be stored
DNS: check "enable domain specific DNS servers"
Domains: fill in your domain name
Name Server Addresses: probably your domain controller ip address

Options
--------------------

Check off the following:
IPSec DOI
SIT_IDENTITY_ONLY
Initial Contact
Support Proxy
Request Certificate
Send Certificate
Unique SAs
IKE Fragmentation

NAT-T disable
do not check "enable connection check"
Action after connection timeout= Give up

--------------------


The key for me was Perfect Forward Secrecy was NOT enabled but it should have been! So ENABLE perfect forward secrecy. The reason for this was that IPSecuritas just does PFS without an option to turn it off or on, so you must turn it on, on the Sonicwall. Otherwise you will get "NO PROPOSAL WAS CHOSEN" when trying to negotiate phase 1. Always have your log file open when trying to debug these connections. Also, be wary of mapping multiple networks behind the Sonicwall, each has to build its own contract. Please contact me if you need help with your connection.