< min(), max() and the collation dependent ordering of character columns on Greenplum (Postgres 8.2) >

Recently I came across an interesting collation related problem while using a Greenplum (Postgres 8.2) database setup with an UTF-8 collation.

Each character column that undergoes an order by, min() or max() will be sorted according to that collation.

But in some cases, I needed to perform ordering or min(), max() aggregations based on the ASCII content of character columns - luckily, this is quite easily achieved on Greenplum using operator classes and custom aggregates:

create temp table col_test (a text);

insert into col_test (a) values ('"quoted"'),('Abc'),('123'),('Öfföff'),('Zürich'),('Zuhause'),('Zahnpasta');

--
-- order by UTF-8 content
--
select * from col_test order by a asc;

123
Abc
Öfföff
"quoted"
Zahnpasta
Zuhause
Zürich

select * from col_test order by a desc;

Zürich
Zuhause
Zahnpasta
"quoted"
Öfföff
Abc
123

--
-- order by ASCII content
-- 
select * from col_test order by a using ~<~;

"quoted"
123
Abc
Zahnpasta
Zuhause
Zürich
Öfföff

select * from col_test order by a using ~>~;

Öfföff
Zürich
Zuhause
Zahnpasta
Abc
123
"quoted"


The min(), max() aggregation can be taken care of by creating two custom aggregates:

drop aggregate ascii_max(bpchar) cascade;

drop aggregate ascii_min(bpchar) cascade;

create or replace function ascii_bpchar_larger(prev bpchar, curr bpchar) returns bpchar AS $$
begin if (prev ~>~ curr) then return prev; end if; return curr; end; $$ language plpgsql immutable strict;

create or replace function ascii_bpchar_smaller(prev bpchar, curr bpchar) returns bpchar AS $$
begin if (prev ~<~ curr) then return prev; end if; return curr; end; $$ language plpgsql immutable strict;

create aggregate ascii_max(bpchar) ( sfunc = ascii_bpchar_larger, stype = bpchar, sortop = '~>~' );

create aggregate ascii_min(bpchar) ( sfunc = ascii_bpchar_smaller, stype = bpchar, sortop = '~<~' );

The custom aggregates can be used like any builtin aggregate:

--
-- aggregate by UTF-8 content
--
select min(a), max(a) from col_test;
 min |  max 
-----+--------
 123 | Zürich

--
-- aggregate by ASCII content
--
select ascii_min(a), ascii_max(a) from col_test;
 ascii_min | ascii_max
-----------+-----------
 "quoted"  | Öfföff

See http://www.postgresql.org/docs/8.2/static/sql-createaggregate.html for more information regarding custom aggregates and http://www.postgresql.org/docs/8.2/static/indexes-opclass.html regarding operator classes.

< Evolution of a trading system - three years of software development visualized in under eight minutes >

We used gource to create a video from the svn logs of our trading system - the background song is available here for free.

 

< dlsym(), ISO, C++, gimme a break >

Today, while trying to implement a simple class that can dynamically load new C++ classes into a running program using dlopen() and dlsym() on Ubuntu 11.04 using g++ 4.5.2, I came across this error:

algorithm.cc: In static member function ‘static algo::algorithm& algo::algorithm::load(const std::string&)’:
algorithm.cc:24:67: error: ISO C++ forbids casting between pointer-to-function and pointer-to-object

Couldn't be clearer, could it? According to ISO, there is no way you can convert a pointer-to-object into a pointer-to-function - PERIOD! But wait! We can fix that by adding four simple green lines (comments not counted):

#include "algo/algorithm.h" 

#include <assert.h> 

#define dlsym __gimme_a_break__ 

#include <dlfcn.h> 

#undef dlsym 

#include <string>

// redeclare dlsym to be a function returning a function

// pointer instead of void *
extern "C" void *(*dlsym(void *handle, const char *symbol))();

algo::algorithm &
algo::algorithm::load(const std::string &algo)
{
    // to at least get the illusion of safety...
    assert(sizeof(void *) == sizeof(void (*)()));
    std::string nm = "libalgo-" + algo + ".so";
    void *lib = dlopen(nm.c_str(), RTLD_NOW);
    assert(lib != 0);
    algo::algorithm *(*mk)() =

        reinterpret_cast<algo::algorithm *(*)()>(dlsym(lib, "make"));
    algo::algorithm *a = mk();
    assert(a != 0);
    return *a;
}   // load()

Voilà, everyone is happy again (except the language advocates maybe)...

<The Joys of C#>

Today I tried to debug my C# code from within Visual Studio 2005 "Professional Edition" - I loaded all the necessary data into the database, massaged it for debugging, and then debugged the program for ca 30 min. While trying to expand a variable in the debug session, I got the following MDA exception:


LoaderLock was detected
Message: Attempting managed execution inside OS Loader lock. Do not attempt to run managed code inside a DllMain or image initialization function since doing so can cause the application to hang.

Yeah, great... just because of this, I had to rollback everything, restart the debugging session, just to find out that the variable expand feature worked this time... something like this never happened to me with neither Java/C/C++/Eclipse nor C/C++/gdb...

<Dilbertism>

What do you say to a manager that wants a guesstimate for porting an RWA calculator from VB to C#, although he demonstrably knows that I am fluent in neither of the two programming languages?

If I were more like my hero, Tom DeMarco, I would probably say that it'll be done when it's done - but since I'm suffering from good manners, I answered that it will take approximately four weeks +/- some uncertainty regarding possible problems that could stem from language specific constructs that might have no simple representation in the target architecture...

I never thought that I would ever find myself in a dilbertesque situation like this again, but here I am... and life goes on ;-)

Perhaps I should suggest the use of the famous Planning Poker game to our development team? There is even an online site for "playing" the planning poker game: http://www.planningpoker.com ;-)

<Finally! A vi plugin for NetBeans!>

http://bgmerrell.blogspot.com/2008/02/netbeans-ide-with-jvi-plugin.html

Yippie!

<Linus Torwalds vs. Alan Cox>

Today I came across this l.k.m.l thread, where Linus Torwalds bashed the seasoned tty maintainer Alan Cox...

I think the whole thing started off when Linus made a suggestion on how to fix a bug that Alan had been working on and Alan replied "wrong end of the stick" - Linus couldn't bear that and he started getting really mad at Alan, which culminated in his: "I think you're full of crap", and later Alan replied: "However I've had enough. If you think that problem is easy to fix you fix it. Have fun." - and he left the maintainers list...

To me, it doesn't really matter so much if Alan or Linus are right about the issue (actually, I think Alan might be right as he is the tty maintainer, but I'm to far away from the Linux kernel nowadays to base my opinion on technical facts) - what matters is that Linus seems to have some sort of ego problem as he chose to insult Alan based on his mere "you got it wrong" instead of discussing the matter on a technical and professional level.

What really makes me sad is that two purportedly grown-up and mature programmers don't seem to be able to get over their personal feelings towards each other - but I guess that's simply human.

Anyway, I would like to thank Alan and Linus for their great work - it's only sad, that you guys couldn't sort out your personal problems for the greater good of the Linux community.

<Code Quality>

Today I came across a very nice drawing:

(Image source: http://www.osnews.com/story/19266/WTFs_m)

Thom is frakking right about it - in my opinion code normally falls into three categories:

1. there is code that provokes no WTFs, because it is doing what it was intended to do in a very unspectacular way, self-documenting and as short as possible,
2. then there is code that provokes only some WTFs, because it is mainly category #1 code with some passages where only few programmers can tell at first glimpse if it is pure awesomeness or complete rubbish (for example duff's device and its usage of intertwined switch and do-while statements vs. the infamous shortened XOR swap),
3. finally, there is code that provokes a lot of WTFs, because it is so obviously wrong and ugly that almost anybody can see it upon first inspection.

Our goal as software engineers should be to write as much category #1 code as possible - and category #2 code only when it is absolutely neccessary, e.g. when one is forced to by performance and/or runtime constraints.

<Tom deMarco>

My favourite software guru has written a phantastic article about software engineering:

http://www2.computer.org/cms/Computer.org/ComputingNow/homepage/2009/0709/rW_SO_Viewpoints.pdf

Thank you Tom, you are my hero!

<Disaster Recovery Plan>

I have been asked to write a disaster recovery plan... The nightmare of every real programmer... as I don't like wasting my precious developer time with anything other than writing code, my first thought was to give the requestor a picture from my favorite comic strip "Dilbert", which looks like this:

Dilbert is the best!

<Regex with Eclipse>

Today, after working some time with eclipse, I discovered that eclipse is also very good at doing RE! Comming from a vi background, I was very amazed! The more I use eclipse as my main ide, the less I have to go back to vi/vim.
Yes, there are still cases where you have to use a text editor like vi, e.g. when you have to do some remote hacking via console. Makes one thing aparent: vi-users can use anything for editing files after mastering the beast ;-)