Tag Archives: programming

Fandro2

You heard it here first: I’ve started working on a successor to Fandro (at xsamplex), mainly because, firstly, I love diving into these kind of algorithms. Secondly, Fandro’s popularity (for an app written in the Windows XP age) surprised me. Thirdly, Fandro required a reboot, because well, while it still works, obviously, there are issues with the current app. We’re moving to computers with higher DPI screens and 64-bit computers.

For a second I actually considered rewriting Fandro in FreePascal but I’ve decided to focus on C# for now: Yes, that will mean that most likely, I’ll open-source it. And yes, if time permits, I’ll even add side comments about how NOT to do text searches.

No timelines as of yet, though, but technically, the main algorithm was finished a couple of days ago. Time permitting, I expect at least something out by the end of the month.

The State of the Machine

I get cranky when I see people use regular expressions or simple substring routines when extracting strings from, for example, e-mail addresses. The first method, while powerful, is memory hungry, the second method is plain childish. You should only use substring/copy methods if you’re hundred percent certain that the data is formatted and well-formed (that is, it comes through exactly as you expect it to. In the case of e-mail addresses, this is of course, not true. After all, e-mail addresses can come in any format. The following samples are all legal: “hey@you.com”, “hey@you.com (Hey You)”, “ Hey You”, “Hey, You “. Your simple substring copy function would most likely have troubles resolving all of these e-mail variants.

During my Roundabout tenure, we ran into issues where extraction of names/e-mail from e-mail headers didn’t work out as originally planned. I was not surprised to find those evil substring routines in the code and literally rewrote that into a state machine (look for HeaderAddressToStringList). Extremely elegant and very effective.

Why use a state machine then? Because with string operations like this, looping through a string is a lot faster than trying hundreds of “if conditions” to cover all these e-mail cases. Keep in mind that simplicity is the key though: the more states you define, the complexer the code.

The devil is in the SerialPort

I‘m surprised how many people look here for help on the .Net 2.0 SerialPort component (reference, which leads to this posting). I have plenty of example code, but I haven’t had time to re-test the project or even try com0com on this 64-bit Windows box. Com0Com (link) does support 64-bit environments, provided that you:

The com0com.sys is a test-signed kernel-mode driver that will not load by
default. To enable test signing, enter command:

bcdedit.exe -set TESTSIGNING ON

What is bcdedit?

No examples for now but my earlier mentioned posting should get you somewhere.

Here-o

For what it’s worth: Don’t bother looking for a complete IDE for Javascript development. They’re out there, but they don’t work and come with a huge payload of crap that needs to be installed for your popular browser. You may just as well do that the old way: in notepad, vi or edlin and with an abundance of alert dialogs and “printf lines”.

Via Alfons and others, I ended up reading an excellent analysis of Conficker”. This is the worm that has many network people worried because everybody knows it is out there (and there’s an official Microsoft patch out for it [pretty colours there]) but nobody knows what it exactly does. There’s a doomsday thread over at Metafilter, so if you’re into reading doomsday threads, go for it.

And last but not least, there was some kind of controversy around the current Canadian minister for Science and Religion (OK, that’s a joke). I can’t remember the context, because, I’ve not been following the news recently. The discussion there (at that CBC page) ended up in a evolution vs. creationism fight which seems to show quite some polarization and discontent at both sides of the aisle.

Is Canada turning into a pseudo-scientific country? As of now, I’m not too worried about this: I don’t think that there are people who believe that the Internet was created on the seventh day or that their WiFi connection (i.e., electromagnetic waves) was provided there by Jewish and Greek gods (and goddesses).

Update 1: This just in: Stallman on JavaScript

A circular thing

I‘m extremelyA some control impressed with the Google Chrome download indicator: for some kind of reason, a circular progress meter makes a lot more sense than a linear one. Maybe it’s because of Pi.

Since I was so impressed and because it was ‘Pi day’ the other day (Pi Day), I thought now was a good time to write a component that does just that: show the progress of something in a somewhat circular form. You can find the code right here: a reminder, it’s C# and that means it is meant for .Net. I’d be really surprised if it wouldn’t work on Mono (it should) [just tested and confirmed that it works].

A couple of notes:

  • FillColor is the colour that’s used to fill up the circle
  • ForeColor is used for the Text (progress text)

The code should be pretty clear in what it does: I was planning to also paint a bunch of “spokes” in the meter but couldn’t be bothered at the end1. It shouldn’t be too hard (use the sin/cos methods of the Math class to get the right x/y coordinates).

1 Also, this would have given it the Trivial Pursuit look: Google’s progress meter reminds me of that too.

03/16/09: Introduced drawing of Text/Label/Caption.

03/18/09: Good rewrite: Mostly introduction of ‘progressbar’ like properties and methods (Value, Min/MaxValue), Marquee. Introduction of a ProgressType enumeration (maybe should use the regular ProgressbarType). Also, same license (Commercial/personal free use, but please leave the credits in).

FFMPEG

I saw that ffmpeg 0.5 was released the other day (I don’t have any link here) and since my Msys/MingW environment was already ready to go I decided to compile my own ffmpeg executable.

I expected it to be ‘an exercise of eating your own hat out of frustation’, however, there are a couple of things you should do to prepare yourself:

  • Evacuate your wife, kids, dogs and cats.
  • Move your coffee brewer next to your computer.
  • Have Kleenex ready for all the spit you probably have to remove from your computer screen.

On a serious note, compiling and finding the sources is not overly hard: The hardest part is to get Msys and Mingw to work right and find the right set of gnu compilers/binaries. If you’ve got that all working, the fun part is compiling the needed packages one by one.

If I can make time, I’ll add an entry how to compile ffmpeg yourself.

Backup

One of the weirdest stories last month, was the one about JournalSpace losing its data and then just plain calling its quits because the inability to recover that lost data. There’s an endless discussion going on at Slashdot and at MetaFilter1. The moral of the story is to properly backup your SQL data: most databases come with excellent tools to dump your data to text, binary or whatever format your prefer. And while you backup, always ensure that a database dump goes off-location, in case a fire breaks out in your server room.

I’ve mentioned this before but I generally keep sources all together, organized by programming language and highly tagged by date and such. Every 6 months, I make a quick inventory and make the proper backups if needed. I never use my source control/revision as a backup means: that’s not the point of a revision control system2. In the years of moving sources off computers and loading them back on new ones, I’ve lost a couple of sources. From all the sources I lost, I regret losing the PHP-based Scr*bble/WordPlay server. Well, actually, I have a bare snapshot of the sources but they are incomplete and I can’t do anything with it. To make it worst, back in 2001 or 2002, I forgot to export and dump the data definition and data. Where things went wrong was that the specific sources and data were stored on completely different locations (I developed on both Caldera OpenLinux and Windows 2000). What happened in my case was that I thought that all the sources were on OpenLinux but that the actual ones were stored on the Windows 2000 computer. I recognized this, of course, after I had formatted the Windows 2000 computer3.

We humans are extremely good in storing stuff in our brains, but we tend to only remember those crucial things after disaster has struck.

1 JournalSpace’s official report is right here. 01/18/09: JournalSpace’s official report is gone. It looks like the domainname was sold to a third party.

2 A source control system is only for saving your sorry ass when you made an error in one of your versions of your software so that you can revert your stupid sorry ass-changes and start over again. Sure you can use it as a backup means (CVS historical data can be moved to different computers easily, for example) but if you thought it to be there for disaster-recovery reasons, you’re mistaking.

3 Sidenote: this was on the MyNote 930. That computer had a full history of whereabouts too, and was eventually passed on to the local recycling plant.

In Reverse

I thought this article (‘Learning the iTunesDB file format’) was an excellent background on the principles of reverse-engineering existing binary file-formats. I’ve done some ‘clean room’ reverse-engineering too and at the end of the project, I was able to extract and cross-reference data from an existing accounting package. The most interesting part of the process is that after figuring out the patterns and the essential algorithms (reading and writing the proper bytes) every byte just falls into place. To this day, I can still dream up the code, which is a kind of funny. That random-looking 8 or 9 in front of each formatted numeric amount? That was just a joke from the original designers of that accounting package.

So, I have a couple of left over links I’d like to get rid of: Way long ago, I noticed that the ODF (the people who are in charge of the Open Office document specifications) announced the (initial) release of the ODF Toolkit, which includes AODL, a .Net module that supports well, you know, Open Office documents. For a complete set of features, look here.

Months ago, Slashdot featured the topic ‘Please list your Useful Stupid Unix trick’. While the discussion (predictably) heads into the typical Slashdot direction, the posting actually has some good and useful commands and hints:

I’ve seen Windows people go slack-jawed in astonishment as I ssh to the other side of the world and run X programs over forwarding.

Some refuse to believe it, others shake their heads and walk away.

And believe it or not: Unix systems have been able to do that since decades.

Canvas

During my programming career, I’ve ran into several cases where I absolutely had to use owner-draw to accomplish customized drawing. Originally, as a Delphi programmer, this required knowledge of the specific graphic wrappers around the Windows GDI (and GDI+) functions which Borland (appropriately) called TCanvas. The C#/.Net equivalent is called ‘Graphics’, which (admittingly) does not sound as fancy as Canvas.

That said: in my never-ending quest to fill a niche craving, I decided to look into the basics of photo-editing; that is, on a much smaller scale. The first step was to create a component that (given a specific image), drops a frame on it, which you can use to crop a photo (by either moving it and/or resizing it). Additionally, I always liked how some photo-editors integrate the Rule Of Thirds during cropping of photos, so, that had to be part of the custom-draw routine too.

There are couple of common tasks that need to be taken care of when doing own-draw stuff, all in C# (however, should be similar in Delphi):

  • The first thing is to decide which control you’re going to ‘descend’ from, or rather, which control is going to be your base-class
  • If your control requires user-interaction (i.e. mouse/key input), you should probably override the control’s MouseUp/Down and MouseMove events. Most likely you’ll need a couple of (private) flags that track down if a mouse button is still ‘pressed’. Add to that a couple of variables that track down the last positions clicked on the screen.
  • Separate the drawing routines and call these routines from an overridden OnPaint event.
  • Debugging (owner-draw) graphical routines is extremely painful, so think through your drawing routines.

Boring sample code is about to follow.

Continue reading

Net. Net.

I read that the Mono developers officially released Mono 2.0, which promises compatibility with Microsoft .Net 2.0. The official announcement can be found at the Mono-project (releasenotes, if you’re impatient). The Slashdot discussion is right here.

There was a long thread (discussion, so you will) on the Postgres mailinglist about Debian, Postgres and backports (ugh!). The issue is complex and it’s a long story about why latest versions of Debian were stuck with older Postgres versions, while most platforms moved to the recent 8.x series of that database. Anyway, the good news is that it looks like that Debian Lenny will come with a more up-to-date version of Postgres. Or, something like that.

Going back to Slashdot, there’s an interesting thread about multi-threading, or rather, about recommendations about good books on this. I’ll be the first to admit that at one time I picked my nose up for .Net threading: I thought Delphi’s way was a lot more elegant, where you’re threading model essentially starts with creating descendants from Delphi’s TThread class. Threading, indeed is a different and difficult beast.

CVS meet Enscript

Last night, This sucks. TerriblyI was installing CVS on my Debian box. Actually, it was already installed: the CVSROOT was ready to go and the service itself had been listening since the early days. However, I never finished assigning rights to myself, so, with that done and by adding myself to the cvs group, I was finally able to check in my latest Helios sources using WinCVS (I despise TortoiseCVS1).

For work, I had once added viewcvs to the CVS repository engine: Viewcvs allows developers to view the repository via a webserver. Easy as is: During its install (apt-get install viewcvs), I ran into the normal installation checks and changes for this specific script: Most of them can be resolved by Debian’s viewcvs installer but (as usual) you should prepare yourself for some old-handy typework. The first action on your list is to add a couple of Aliases to Apache’s httpd.conf file (in my case I was only interested in one, the viewcvs ‘docroot’ parameter: I don’t like graphics2).

 Alias /viewcvs/ /usr/share/viewcvs/

The next step is to set the ‘docroot’ and ‘icons’ folder parameters in the viewcvs.conf file (/etc/viewcvs/): these folders refer to the folderstructure on the webserver and not to the physical folder structure on your Linux system (hint: check Alias). After this, you can set the use_cvsgraph and use_enscript variables in that same file to 1.

Enscript is a ‘source code highlighter’, but (obviously) it doesn’t come with a state file for C#: I found a good one at Adam Milazzo’s (Thanks Adam). This file should be copied to the enscript states directory (on Debian that is at /usr/share/enscript/hl). Also, you will have to make a couple of changes to the enscript.st file (which is in the same directory): add the csharp highlighter to the namerules section. The final step that needs to be undertaken is a change to the viewcvs.py file (you generally should be able to find this at or around /usr/lib/python2.3/site-packages/viewcvs/). Look for the enscript_extensions section. Notice the comment that one of the programmer left behind, just right above that extensions section:

### this sucks… we have to duplicate the extensions defined by enscript

You wouldn’t say.

1 Yeah, I was going to write something here, but I forget. Oh right, something about TortoiseCVS and how I hate it.
2 Uh. Right. What was that again?

When the Earth

A couple of links to keep you busy for now:

I’m not sure where I got this one from, but to me ‘Social Wallpaper’ is the only compelling social network. If you can call it a social network: basically, it allows everyone to upload their wallpaper, with mixed results, of course. Some of the pictures are truly amazing, some of them plainly suck. Some pictures come from the NASA archives and some, well, don’t. I have to admit that I haven’t been convinced to change my desktop wallpaper yet (that wallpaper comes from Alfons).

Earlier this week, I stumbled upon this 2004 discussion on the Physics Forum where someone asked for ‘Prove that the Earth revolves around the Sun’Geocentric vs heliocentric:

I find it obvious, as did many previous people, that the sun goes aroudn the earth. I of course know htis to be untrue as it has been drummed into me again and again for the past 16 years.
Can someone please prove to me (so that i can prove to others as well) that the earth goes around the sun. Do not be afraid to explain using mathematics, as i see that as the only way of undeniable proof.

The discussion is eye-opening: the most compelling (easy to verify) proof seems to be planetary retrograde motion, which is too complex to explain if our solar system was a geocentric system. Additionally, with the knowledge we have of gravity, it would have been scientifically impossible to explain why the sun would move around an object with less mass (more on heliocentric vs. geocentric, right here).

And last but not least: I was surprised to find out that OpenOffice comes with CLI/.Net assemblies (if you have OpenOffice installed, it’s right in your “Program Files\OpenOffice.org 2.0\program\assembly” folder). Sample code can be found at OpenDocument4All: the projects seem to be geared towards .Net 1.1 (Sourceforge project page).

Update: Slightly unrelated: Fulll moon eclipse for next Thursday.

Is there a statistician in the house?

I downloaded the Visual Studio 2008 Express Edition ISO from the Microsoft site and discovered that specific stuff I was looking for isn’t supported in that special edition. I guess there is (or rather was) a huge difference between ‘Orcas’ and the Express Editions (Earlier at xsamplex).

That said, there was a time for looking through code and then I discovered that at Microsoft’s WPF site somebody had uploaded a Peer-2-Peer version of Sudoku (see also this post, or rather, this photo). This reminded me that a couple of weeks ago, I was asked to help with an algorithm that generates a complete filled out Sudoku puzzle. This I did and after that I moved on. So, that Peer-2-Peer version apparently has an algorithm that does the same, so when I opened the particular source file, I was shocked to find the following code (see my tag):

for (int iRow = 1; iRow < = 9; iRow++)
{
    for (int iCol = 1; iCol <= 9; iCol++)
    {
        iNum = rnd.Next(1, 10);
        while ((IsInColumn(iNum, iCol) 
                  | IsInRow(iNum, iRow) | 
                   IsInThreeByThree(
                      iNum, 
                      GetThreeByThree(
                         GetThreeByThreeIndexFromRowCol(
                             iRow, iCol)))) 
                  == true)
        {
            // Generate a number to be put on the grid..
            iNum = rnd.Next(1, 10);
            iAttempts++;

            // AH01 -- OH NOES...
            if (iAttempts == 100)
            {
                bStuck = true;
                break;
            }
            // AH01 -- Ends.
        }
        if (bStuck)
            break;
        iAttempts = 0;
        arrPuzzle[iRow - 1, iCol - 1] = iNum;
        if (iRow == 9 && iCol == 9)
            bDone = true;
    }
    if (bStuck)
        break;
}

Let me go back a couple of steps. In Sudoku, a 9 by 9 grid is divided in nine 3×3 grids: the numbers in each 3×3 grid MUST be unique and between 1 and 9. Additionally the same rule applies to each horizontal and vertical line in the grid. A fairly good description can be found at Wikipedia.

The main algorithm for filling out a 3×3 grid with 9 unique numbers, is a statistical routine that should have came up during math classes as the ‘coloured balls in a basket problem’. There’s 10, no, lets say 9 different coloured balls in a basket. What are the chances of (blindly) picking out the red ball in the second turn?

Back to that source snippet: Random generating a number from 1 to 9 The author randomly draws a number between 1 and 9, checks if the number has already been drawn. If so, he keeps generating a random number until he gets a number that fits the needs. Here’s where the problem starts: the more numbers you have picked, the less chances there are going to be that your random generator is going to pick one of the numbers that has not been drawn yet. Take a look at the example on the right (an animated gif, opens in separate window): If the stars are aligned right and your computer is in a bad mood, it may just take a while to randomly generate those last numbers. Certainly, you could throw in a 8-core computer to speed up the random number generator but the problem will still be there. The programmer knew this too hence the reason for that ‘jailbreak condition’ (“After 100 attempts I think I should break it off otherwise it looks like the program has crashed”).

Randomly drawing numbers from 1 to 9The right way to do this is, by actually drawing these numbers from either a list or an array as shown in the animated GIF (click on the image on the left). This effectively means that your randomize Next routine needs to adjust dynamically after each draw: after all, every time you draw a number, the amount of available numbers decreases by one. There’s a minor catch here, but that’s up for you to figure out.

So: what’s the point of bringing this up? The Sudoku Peer-2-Peer application is a great showcase of the latest and greatest .Net technologies but fails horribly on the basics: using math to solve a complex problem. I do (for example) have more respect for this guy who threw in a genetic algorithm to solve the problem. This is overkill for sure because it can take up to 10 minutes to fill out a complete grid. At least he understands (and showcases) how genetic algorithms work.

Update (01/21/08): Related: Who killed the Software Engineer?