Functional Programming

Your brain is rolling along on that cold steel track of life. You process item after item, then list after list as if you were the building the chassis for a million robotic arms. Do this, then that, then this. Wake up, shower, go to work, go to lunch, come home, go to bed, rinse and repeat. Suddenly div/0, your brain stops and you forget every pattern that was pitted into your mind. The very wiring of the mind is a melted cacophony of cold snow and bloodied bird feathers. You know not who you are or what you are or why you are. Objects become functions and anonymous functions become parameters. Logic is modeled in actors passing messages. Actors mean nothing beyond what messages they pass. Cars on the highway become mail to mailboxes. Action and nonaction and not being. Colors become transformations yet to be decoded. You are what you do and nothing more. That is the primordial essence of the functional being.

Reading list

I am logging a reading list for myself. Most times, when reading something, the author references another work, so I want to log this list as a must read.

Joe Armstrong's doctoral thesis
Why do computers stop and what can be done about it. by Jim Gray

Perl one liner, backreferences, displaying only part of a line

I like this construct for searching through files in a directory and pulling out part of a line.

Just add the regular expression. The '?' designates a non-greedy expression, which means the finite statement machine will quit on the first successful match.


find . -name "*.xml" | xargs perl -ne '/(.*?)"/; print $1; print "\n";'

What I like about living in Atlanta, GA

I recently moved to the Atlanta area. @dneighbors on twitter asked the question "Why Do You Live Where You Live?" It may be that I just love Atlanta, or maybe cause it felt so good to get out of Phoenix. Let me paint a picture for you.

Here is my perspective. Being born and raised in Phoenix, AZ and then moving to the Atlanta area in my thirties was a monumental change. The culture, weather, terrain are all starkly different.

Phoenix: hot, dry, dusty, smoggy, transient culture, California rejects, kind-hearted souls, sunburn, sunny days, prickly cacti, rocky moon-like surfaces, smelly dairy farms, cotton fields, traffic cameras, migrants standing on corners, 90 degree low temperatures, blast oven heat in your face from April to November, the sweet smell of jasmine and orange blossoms on spring nights.

Atlanta: Cool breezes, smell of pine in the air, festivals in the park on beautiful days, a sense of community, long summer days by the pool, peach cobbler, wild rasberries, blackberries, and blueberries growing in summer, smell of chocolate in the spring when the Magnolias bloom, blooming wisteria, azelia flowers, the old narrow streets of mid town, the rolling hills of Atlanta, every street named "peachtree" downtown, the aquarium, the college football, the reddest red of the maples in the fall, the greeniest of green of the pines against the deep blue sky, the "oceans" of kudzu, the brisk autumn breezes delivering a blanket of red, brown, and yellow leaves, the southern hospitality, neighbors helping neighbors, the squirrels, the bright red cardinals, the shady paths in the park, the fireflies, the geese, the deer

Out of work

I recently lost my job. I am looking for a new way to earn a living. I have a couple of leads right now and I am also reading a lot and hacking on some rails apps for some of the business ideas that I have had over the last few years. It feels good to get out of on my own. I am learning about some things that I have wanted to know about for quite some time. Some Erlang topics are really interesting me right now. It would be really nice to start a consulting firm. I have a feeling that jumping into a long term full time job is not the cards right now.

I had an idea for a documentary. It would follow a small software engineering firm from inception to consulting and producing a piece of software to trying to paid. All the things that small businesses have to deal with. It would cronical the times we live in. Our culture here is very enterprising and this where creative projects go from a concept to a real living thing. I dream that I could be the one who starts the company and filmmakers can just hang around and record everything. I would love to watch that kind of a movie. I was fascinated with the documentary, "The Staircase". I highly recommend this film. These are the best sort of movies, I think.

DNSMASQ and multiple DHCP networks with DHCP relay

I switched from MaraDNS to DNSMasq. DNSmasq is an excellent tool for small networks. Small networks are defined as something less than 1000 hosts. This is most networks! I love DNSMasq's simplicity. I also like how it bundles DHCP, DNS recursive server, DNS caching. The DHCP is surprisingly simple to use and configure and incredibly feature-full. For example, the DHCP server handles PXE clients requests for the next server. It also handles requests that are relayed, so you can provide multiple subnets DHCP service and do it based on the tag provided by the relayer. I also like how you can reuse the local (to the DNSmasq) machine's /etc/hosts and /etc/resolv.conf to gather static entries and upstream recursive DNS caches. It seems simple to just reuse these files for the server and clients. You are essentially exporting the server's /etc/hosts and /etc/resolv.conf to your clients.

Tail recursion in Python

Problem: When marshalling an object to be able to transmit to or from an XMLRPC server and the xmlrpc lib is insufficient to the task because the objects within the main object do not implement a marshal method.

Admittedly the datetime, mxDateTime and None objects should just know what to do if asked by the xmlrpclib to marshal themselves, but in this case, I want to do this myself.

Answer: Here is a tail recursive way to perform this task. This should not use any stack space, but I cannot confirm this since I am not sure what is going on in the interpreter.

def __convertNonMarshalables(self, obj):
  if type(obj) == type({}):
    for key, value in obj.items():
       if value is None:
        obj[key] = ""
      elif type(value) is type(mx.DateTime.now()) or type(value) is type(datetime.datetime.now()):
        obj[key] = value.strftime('%Y-%m-%d')
      elif type(value) == type([]) or type(value) == type({}):
        value = self.__convertNonMarshalables(value)
  elif type(obj) == type([]):
    for value in obj:
      if value is None:
        value = ""
      elif type(value) is type(mx.DateTime.now()) or type(value) is type(datetime.datetime.now()):
        value = value.strftime('%Y-%m-%d')
      elif type(value) == type([]) or type(value) == type({}):
        value = self.__convertNonMarshalables(value)
  else:
    if obj is None:
      obj = ""
  return obj

Leopard killed TextMate, sort of

Just updated Leopard and now having "#!/bin/sh" in the "command(s)" section of the bundle editor for any bundle yields "bad interpreter: no such file or directory" if I remove the directive, it works fine, but now on the shell bundle, the run command script is a ruby script and TextMate will not take #!/usr/bin/env ruby, yields bad interpreter again, but I really need to tell TextMate this is a ruby script. So I ran the script with ruby -e'do something'. That is weak sauce but it works. I also noticed circular dependencies in the bundles. You need ruby to run a python script in the python bundle; you need bash to run a ruby script, you need ruby to run a shell script.

I am not sure why this is because the text in command(s) should be treated as a shell script. Anyway, if anyone else has this problem, then removing the shell directive line should help.

Functionalness

An object that has no function does not exist.

Auto adding DHCP host names to DNS (MaraDNS)

Ever wonder why a host that you offer a DHCP lease to just does not automagicly appear in your non recursive DNS server's zone file. Well, it should! Using this "glue" code, it will be a snap to update your zone files whenever a new host joins the network. In this example, I am using MaraDNS and dhcpd.

Overall concept:

dhcpd --> lease file --> cron script --> ruby script --> DNS zone file

Cron entry (/etc/cron.d/update_dns):

*/5 * * * * root sh /usr/local/bin/parse_leases_add_to_dns.sh

parse_leases_add_to_dns.sh:

#!/bin/bash

#check modification time
if [ ! -f /tmp/leases_mod_time ]; then
touch /tmp/leases_mod_time
fi

#see if the last check time is less than mod time
if [ /tmp/leases_mod_time -ot /var/lib/dhcp/dhcpd.leases ]; then
/usr/local/bin/ruby add_hosts.rb
touch /tmp/leases_mod_time
fi

add_hosts.rb:

leases = open('/var/lib/dhcp/dhcpd.leases')
dns = open('/etc/maradns/kleetus.1337.foo', 'w')

leases_string = leases.read
lease = ""
client_hostname = ""

leases_string.each do |line|
next if line[0..0]=="#"
unless (line =~ /lease/).nil?
lease = line.split[1].gsub("\"","").gsub(";","")
end
unless (line =~ /client-hostname/).nil?
client_hostname = line.split[1].gsub("\"","").gsub(";","")
end
if client_hostname!="" and lease!=""
w_string = "#{client_hostname}.% #{lease}"
dns.write("#{w_string}\n")
lease = ""
client_hostname = ""
end
end

statics = ["myth-test.% 10.20.10.120", "*.% 10.20.10.100"]

statics.each do |line|
dns.write("#{line}\n")
end

dns.close

load=`/etc/init.d/maradns force-reload`

So why are you using a ruby script, can't you just do it all with bash/awk?

Sure, but the ruby script is more fun to write and maintain; if you feel the need to use bash, then this is possible too.

Adam Carolla

If you have never heard of this man, you are for a treat, my friend! Adam is the Quintessential everyman's man. He can be heard here

Here are the reasons why I love Adam Carolla:

1. He connects with his listeners on a personal level because he doesn't act like some stuck up rich person. He recognizes that most of his listeners are just like he is. He worked construction and cleaned carpet. Now he has means, so it is nice that he can create a portal into the world of Hollywood, but he always remains genuine. He is sincerely connected to "the rest of us".

2. He is honest. This really should be number 1 on my list. You cannot connect with people if you are phony. People can see through this in a second and radio people are fooling themselves if they think that they put on a persona and retain an audience. Ace Man (Carolla) just puts himself out there, take it or leave it. He has flaws and those flaws can be endearing since we all have them too.

3. He is observant. This is more than schtick, Adam brings out the truly absurd in our culture. He comes off more genuine that the Seinfeld ("Who are these people") bits. He gets into more counter-culture routines that are really funny. He has one bit where is mimes a strip club MC. Callers will ask him to do an impromptu routine centered around them. Great bit.

4. Connections. Adam is plugged into Hollywood and its weirdness. He will call out weird shit too. Life is all about relationships. Adam's personality is likeable and most pop culture stars like Adam, so he can leverage this.

5. Adam is old school So. California. He was born in North Hollywood and lived amoung the regional culture. I find people who talk about subculture amoung the most interesting. I live in Atlanta, so this is far removed from this, but I love to hear about how "the Valley" is like or dislike my area.

How to get ffmpeg to transcode your over the air mpeg2ts to iphone 3G

I got this figured out, so I thought I would share this with anyone transcoding mpeg2ts (over the air high definition) to something that the iphone 3G can play from a streaming server such as apache. Here is my ffmpeg line:

ffmpeg -i "${directory}/${file}" -async 3 -r 29.97 -acodec libfaac -ar ${rate}
-ab ${abitrate} -ac 2 -s ${width}x${height} -vcodec mpeg4 -b ${vbitrate} -flags
+aic+mv4 -mbd 2 -cmp 2 -subcmp 2 -g 250 -maxrate 512k -bufsize 2M -title "${file}"
 "${directory}/${file}.mp4"

The key here is to ensure the playback is at no more than 30 frames a second, or your iphone will play like 1 second and then stop...like it can't buffer the stream. Of course, when you do this, you will lose audio/video sync since you just yanked some frames out out the stream, so you need to use -async 3 to fix this; 3 works for me, but you may want to experiment. Also, I am using a recent SVN trunk compilation of ffmpeg with libfaac and xvid enabled in the compilation. This made a huge difference in whether ffmpeg could work with me on this TS file.

Now the bash like variables are exactly that, bash variable interpolations. Rate is the sample frequency of the audio...something like 44100 or 48000, abirate is the audio bit rate...something like 128k, the vbirate is not crucial, the video bit rate, something like 480K or 378K. The vbirate will really make the resulting file smallish. The original was prolly 2 or 3 times that. The dimensions usually work out to be 320x240, but I have also used 432x240 with success for 1.78 ratios.

Ipsecuritas, Mac OS X, Sonicwall Enhanced Firmware

This was a tricky one. I always had some difficulty getting the free IPsecuritas connected; so I am sure others had problems too.

Using IPsecuritas 3.2 build 2501
MacBook Leopard

connecting to a:

TZ 190 Wireless Enhanced
SonicOS Enhanced 4.0.1.3-46e

Sonicwall side config (straight out of the tech support report)
--- SA 1 ---
Authentication Method : IKE with Preshared secret
VPN Policy Name : "WAN GroupVPN"; enabled
Policy Type : Client Policy
Pre-shared Key len : 14, value=
IKE Local Id : UNKNOWN
IKE Remote Id : ID_FQDN: (GroupVPN)
Local network :
Peer network :
IKE Exchange : Aggressive Mode
IKE Proposal : DH Group 2; Encrypt/Auth - 3DES/SHA1
IKE SA Life time : 28800 (seconds)
IPsec Proposal : DH Group 2; Encrypt/Auth - ESP: 3DES/HMAC SHA1
Ipsec SA Life time : 28800 (seconds)
Policy Options : PFS: on; Xauth: on; Netbios: on; Multicast: off
Management : HTTP: n; HTTPS: n; SSH: n
XAUTH user group : Trusted Users
Default LAN gateway : (0.0.0.0)
VPN policy : Bound to zone WAN

WAN GroupVPN Client Settings:
User Name and Password Caching:
XAUTH User Authentication is Required
Cache XAUTH User Name and Password on Client: Never
Client Connections:
Virtual Adapter Settings: DHCP Lease or Manual Configuration
Allow Connections to Split Tunnels
Set Default Route as this Gateway is Not Selected
Apply VPN Access Control List is Not Selected
Personal Firewall on Client Machine is Not Required
Client Initial Provisioning:
Use Default Key for Simple Client Provisioning is Selected
-----------------------------

Now the ipsecuritas config

General Tab:
--------------------
Remote IPSec Device: IP or host name of Sonicwall (must be reachable from Internet)
Endpoint Mode: Host (IP Address left blank)
Remote Mode: Network (Internal LAN network of the Sonicwall, such as 10.0.1.0 CIDR/Mask 24)

Phase 1:
--------------------
Lifetime: 8 hours
DH Group: 1024 (2)
Encryption: 3DES
Authentication: SHA-1
Exchange Mode: Aggressive
Proposal Check: Claim
Nonce Size: 16

Phase 2:
--------------------
Lifetime: 8 hours
PFS Group: 1024 (2)
Encryption: 3DES
Authenication: HMAC SHA-1

ID:
--------------------
Local Identifier: Address
Remote Identifier: FQDN.... just fill in the "Unique Firewall Identifier" from the Sonicwall VPN section
Authentication Method: XAuth PSK
Preshared Key:
Username: XAuth username
Store Password: checked if you would like the password to be stored
DNS: check "enable domain specific DNS servers"
Domains: fill in your domain name
Name Server Addresses: probably your domain controller ip address

Options
--------------------

Check off the following:
IPSec DOI
SIT_IDENTITY_ONLY
Initial Contact
Support Proxy
Request Certificate
Send Certificate
Unique SAs
IKE Fragmentation

NAT-T disable
do not check "enable connection check"
Action after connection timeout= Give up

--------------------


The key for me was Perfect Forward Secrecy was NOT enabled but it should have been! So ENABLE perfect forward secrecy. The reason for this was that IPSecuritas just does PFS without an option to turn it off or on, so you must turn it on, on the Sonicwall. Otherwise you will get "NO PROPOSAL WAS CHOSEN" when trying to negotiate phase 1. Always have your log file open when trying to debug these connections. Also, be wary of mapping multiple networks behind the Sonicwall, each has to build its own contract. Please contact me if you need help with your connection.