R (15) Admin (12) programming (11) Rant (6) personal (6) parallelism (4) HPC (3) git (3) linux (3) rstudio (3) spectrum (3) C++ (2) Modeling (2) Rcpp (2) SQL (2) amazon (2) cloud (2) frequency (2) math (2) performance (2) plotting (2) postgresql (2) DNS (1) Egypt (1) Future (1) Knoxville (1) LVM (1) Music (1) Politics (1) Python (1) RAID (1) Reproducible Research (1) animation (1) audio (1) aws (1) data (1) economics (1) graphing (1) hardware (1)

25 May 2009

If only it were that simple... Enforced upgrade from feisty to jaunty

Pl/R abjectly refuses to build under Intrepid (8.10). No package for it, requires postgresql source, and even then it fails. But wait! There's a Jaunty package. A good excuse to upgrade (and postpone real work for another few hours)? Can it be done over a remote ssh without knocking the box over? On the upside, Oh My Lord, the campus network is Mad-Fast. The new ubuntu install tools (do-release-upgrade) helpfully inform me: "You have to download a total of 1735M. This download will take about 3 hours 40 minutes with a 1Mbit DSL connection and about 2 days 19 hours with a 56k modem. " Um, yeah. 5 minutes later... Done jaunty/main postgresql-contrib-8.3 8.3.7-1 [81%] 9469kB/s 35s Holy mother-of-wow. I love my campus, and i highly recommend for West Coasters looking for a screaming mirror. Of course, unpacking takes a while. First problems arise therein: "dpkg: libjack0: dependency problems, but removing anyway as you request: stk depends on libjack0 (>= 0.109.2); however: Package libjack0 is to be removed. mplayer depends on libjack0 (>= 0.109.2); however: Package libjack0 is to be removed." NOOOOoooo! Give me back my libjack! This is a work computer, and libjack is definitely not for work, though the psychic relief of multitrack recording with rosegarden definitely helps. What gives? Total upgrade takes ~50 minutes. At last: System upgrade is complete. Restart required To finish the upgrade, a restart is required. If you select 'y' the system will be restarted. Continue [yN] y Broadcast message from xian@ocimum (/dev/pts/2) at 20:19 ... The system is going down for reboot NOW! It goes down, and doesn't come back up. Looks like i'll be driving into the lab today...

22 May 2009

Exploits in Brute Repetition - RODBC and Postgresql and Postgis and UnixODBC and Ubuntu

Database admin will make me old before my time... It's a wicked-cool toolchain, but byzantine to say the least. This is the third time i've gone through this as a fresh installation, and it sucks every time. Here i'm installing the database engine, setting up the database, adding db user, and restoring a dump. This time, I've documented it... May you never have to do this from scratch through the google-tron! #!/bin/bash ## steps to create frs database from scatch on ubuntu ## with system user as cluser superuser ## postgis setup included for package postgresql-8.3 sudo apt-get install postgresql-8.3 postgresql-8.3-postgis sudo -u postgres createuser --superuser $USER sudo -u postgres psql -c "alter role $USER encrypted password '*****'" createdb $DB createlang plpgsql $DB psql -d $DB -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql psql -d $DB -f /usr/share/postgresql-8.3-postgis/spatial_ref_sys.sql ## ## postgis version issues... # cd /usr/lib/postgresql/ # ln -s 8.3 8.2 # sudo ln -s # time psql -d frs -f db.full-2008-06-24 # psql -d frs -f /usr/share/postgresql-8.3-postgis/lwpostgis_upgrade.sql ## odbc sudo apt-get install r-cran-rodbc sudo apt-get install unixodbc psqlodbc sudo odbcinst -i -d -f /usr/share/psqlodbc/odbcinst.ini.template sudo su cat /usr/share/doc/odbc-postgresql/examples/odbc.ini.template >>~/.odbc.ini ### edit the latter - name used to call by, db, passwd, etc. ### unixODBC doesn't do ident authentication??? exit chmod 600 ~/.odbc.ini ### tune the server ### suggestions here: ### and here: ### this machine has 4gb ram, services small number of concurrent connections sudo vi /etc/postgresql/8.3/main/postgresql.conf sudo sysctl -w kernel.shmmax=268435456 sudo sysctl -w kernel.shmall=4194304 sudo vi /etc/sysctl.conf ## set shmmax and shmall across reboots # pulling data from postgres into R using a custom function - not too shabby for a half a million records! # system.time((tmp=query('v_dtw', clause="limit 500000" ))) # user system elapsed # 25.502 18.277 46.285

19 May 2009

Adventures in R Recursion

Spent the day on the coffee shop patio fielding OLPC questions - from "Wow, that's cool, what is it?" to "Where can i get one?" I demonstratively poured some tea on the keyboard at one point for waterproof emphasis to "coooool"s. Good times. Installed R on the green machine (which is now runing happy-fun-times modified ubuntu. Spent time working out of Venables and Ripley's "S Programming" book - trying to keep some finals week brain momentum going... Plunking example code into the green machine. I kept the screen backlight off, just using sunlight and low battery while the sun was up. When i took a break from coding, i could hit the power button to put it to sleep. So, i worked through some recursion examples, in the meantime discovering my own knowledge-gaps in indexing ( this[-index] is the compliment of this[index] for a numeric index vector, whereas this[!index] is the compliment for a logical index vector) and list packing. Also, found a test example of "=" not the same as "<-" a=1:10 > index1 = a>2 ; any(index2<-(a>2)) ; identical(index1, index2) [1] TRUE [1] TRUE > index1 = a>5 ; any(index2=(a>5)) ; identical(index1, index2) [1] TRUE [1] FALSE ### index2 assignment isn't made... why?! I find recursion confusing; debugging recursive functions is especially so! The fruits of my labor are here, with examples of all of the above... mk.recur = function(a,lim=5, fac=2){ ## takes a numeric vector, returns a list of numeric vectors ## with elements greater than lim, multiplied by fac recursively if(any(index<-a>lim)) {ret=list(a[index])} if(length(a[!index])==0) {return(ret)} ret=append(ret, Recall(fac*a[!index], lim, fac)) return(ret) }

12 May 2009

the trigonometry of seeing and being seen (and watching it all happen)

in day-to-day exchanges, we often indicate our attention to each other, whether or not we're actually listening. a nod of a head, "uhuh", raised eyebrows, smirks & frowns & winks. all these are directional; their target is clear.

at a distance, it's harder to know whether he's checking you out or reading your shirt. when 2 people are in close conversation, it's trivial to tell that they're "gazing into each other's eyes".

basic trigonometry shows that the farther the distance, the less the angle. further, to watch a watcher watching is even trickier - i'm uncertain of the angle between watcher, watched, and me, let alone where their eyes point.

all of this makes it devilishly hard to tell if someone's checking someone other than you out (and which part) if they're sufficiently far away from either. yet two people from afar notice quickly if they're checking each other out - i know where i am in space.

try it. i bet the best you can tell is whether someone's looking at you or not...