Blog archive

This is (or will be) a collection of my ponderings on computer science and life in general.

There is an atom feed here ;)

Clearly, as the site is not dynamic, there's no way to directly comment on the posts here. Your best bet is to send me an email.

Wed, 29 Sep 2010 11:53:16 +0100

Some notes on C#

So I have a job interview on friday for a .NET developer position. Having not really done much with .NET in the past (which the employer doesn't mind so long as I am willing to learn) I decided to brush up on the main differences between this and the other languages I use which these days are; C, C++, assembly and Java. There doesn't seem to be much point comparing C or assembly to the .NET framework, so I'll try and get the main points compared to the other two.

This is written for my own reference more than anything else, so YMMV. If a feature is very similar to that in another language I will just say that rather than explaining it.

Basics, object model etc

Like Java, the object model is rooted in a single class; object, which is a native type.

Operator overloading is allowed, similar syntax to C++ but without stuff like returning non const references to *this etc. || and && can't be overloaded. == and this->Equals() have different meanings (== is reference equality, Equals() is value equality.)

Class definition is similar to C++, except the default inherited visibility is public (don't seem to be many examples of using protected or private, but then it was rarely used in C++ as well - maybe it doesn't exist?). Same as Java in that it always inherits from System.object. There is no semicolon on the end of the class definition. No multiple inheritance, like Java has a special type of class called an interface (with the same keyword) which only contains pure virtual member functions.

public class Foo : SuperClass, IInterface1, IInterface2 {
}

Strings in C# are immutable in a similar manner to Java. However they can also contain null characters - presumably this is because of the weird double-null-terminated-string-list structure that Win32 loves so much. There is probably some API function to convert between these and collections of strings. .NET strings are UTF-16, which is fine as Windows NT has always used unicode internally (hence all the Ex/A suffixes in the API)

Generics in C# (and .NET in general) are more similar to Java's generics than C++ templates. They essentially cast up to object and back the whole time.

Collections in C# typically inherit from IEnumerable<T>. The for-each syntax is as follows:

foreach(Thing i in setOfThings) {}

compared to the Java:

for(Thing i : setOfThings) {}

Exceptions

C# has no checked exceptions, like C++. As the standard libraries and the runtime make use of exceptions, you have to make the same assumption as in C++ when working with exceptions. Namely, that any line of code could throw. This makes some sense, as checked exceptions in Java were not always used correctly, for example: 

String s = in.readLine();
int x=0;
try {
	x=Integer.parseInt(s);
} catch(NumberFormatException e) {
...
}

Where NumberFormatException is not a checked exception, even though it is likely to be thrown. Of course the argument could be that this code should not use exceptions in the first place, as an abitrary string is less than likely to be a valid integer. (of course they added tryparse, but that has it's own problems.

The syntax for exceptions is pretty much the same as Java, with try, catch and finally. All exceptions inherit from the base class Exception.

Naming conventions

C++ doesn't really have a standardised naming convention, with each project defining their own. Java does, so I'll point out the main differences for comparison.

UsageJavaC#
Class names Starts with a capital, subsequent words also. The same
Member function names camelCase PascalCase
Member variable names camelCase PascalCase
Paramter names camelCase camelCase

Will add more later...

Sun, 19 Sep 2010 14:40:17 +0100

Some changes

So I've moved some things around on this site. The intention was to upload some other code, but most of the software I've written the last year or so was for other people so I can't really post that here.

I added the slowdown program, which is of limited utility but it's better than pastebinning it any time somebody wants it. I may try and make it into a more general purpose filter so you can simulate being on a *nix box over an old 300 baud modem (though it'd probably be easier to use a serial console and just set the baud rate.)

I'm going to try and clean up my BrainFuck compiler to upload it - was written in a hurry a good while ago so not very good by way of commented code. It produced the fastest code of any other compiler I could find at the time. That said there are probably a few optimisations that could be made - say buffering IO rather than read()/write()-ing single bytes at a time (as system calls aren't cheap).

Finally I'll try and get some version of my OS up. The problem with that, however, is that most of the ideas in it are proof of concept and testing rather than anything useful. On the other hand there are a number of reasonable features:

But the code in the kernel itself is a bit fragmented, as I didn't know what I was doing when I started. I will get around to writing a new version at some point, but I've other drains on my time at the moment. The User-Mode side is ok though - however chunks of it won't be binary compatible with the rewrite. For the time being, anyone who wants a copy can ask for one, but it's not going on here yet.

Wed, 03 Feb 2010 06:52:38 +0000

well i was foolish

Heh, turns out it wasn't a firefox issue, but a Gentoo one (and in fairness, seeing as it's not the mozilla binaries I shouldn't have mentioned it anyway.) Gentoo uses the system SQLite libraries, which were not compiled with secure delete enabled (or availiable as a use flag).

My bad.

Wed, 03 Feb 2010 05:45:22 +0000

Firefox doesn't really clear your private data

While waiting to head into uni for a lecture, I was clearing out some things in my /home directory. Poking around in my Firefox profile directory, I noticed a few large .sqlite files (places.sqlite, cookies.sqlite etc). Now I set Firefox to clear private data on exit. It removes all traces of the history in the user interface, so it was a bit odd.

On opening places.sqlite with a text editor (btw these are binary files, but you can still make it out) , I saw practically every site I had visited since 2004. The same went for the cookies and site preferences. I killed firefox, deleted these files and started it again, clicked round a few sites, cleared private data and quit. Same thing, it had kept my history and cookies.

I have to admit, there were not a small amount of conspiracy theories flying around my head at that point. A simple cron job to delete the data would not suffice, as there are certain things I do want to keep (e.g. which sites to accept cookies from) and if FF is running when the job executes (as is likely to be the case), firefox will hold the lock on the file anyway, and next time it syncs to disk it'll all be restored.

I started thinking about a small tool to do the file wiping, which i could add to my /usr/bin/firefox script. Reading about the sqlite file format it soon became clear what was happening, rather than rebalance the whole B-Tree, sqlite was just adding the deleted nodes to a free list, not actually zapping them from the file. Reading further it says sqlite provides two options to solve this:

Neither of these would be difficult for the Firefox people to implement. I'm currently compiling the latest firefox and sqlite to verify this is still present (was still on 3.5.2). After which I'll file a bug report.

In the meantime though, you can either just delete the files by hand (with firefox not running) or VACUUM it yourself, again when firefox isn't running

This can be done quite easily in the shell, just load the database with sqlite's interface (sqlite3 in this case) and type VACUUM; then quit (ctrl+d works fine). Again, if you try doing this with firefox running, you'll get something like:

richard@armada ~/.mozilla/firefox/k6telub8.default $ sqlite3 places.sqlite
SQLite version 3.6.22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> VACUUM;
Error: database is locked
sqlite>

An easier way is probably to loop over them all and vacuum them (again from the firefox profile directory):

$ ls *.sqlite | while read
> do
> echo "VACUUM;" | sqlite3 "$REPLY"
> done

It's worth noting that this won't clear out stuff like cookie preferences. If you go onto the options and clear them, then run this script it will remove traces (as they haven't been dropped from the database).

Sat, 23 Jan 2010 20:13:25 +0000

Making ROX-Filer respect folder.jpg thumbs

As you may know, I'm a big fan of ROX filer as it's very easy to use, and fast. However my main workstation also functions as a file server for my media centre downstairs (XBOX running XBMC, which is very nice as well).

In XBMC you can set custom thumbnails for directories, placing a file folder.jpg inside. Presumably this is for compatibility with Windows Media Centre/Media Player, but couldn't say tbh (don't use Windows, so it's irrelevant). It also allows you to set icons for individual files with filename.tbn.

I wanted ROX to show these icons on my workstation when I viewed said directories, I knew there was a hook somewhere, as ROX has a feature known as 'AppDir's, which is similar to the way RISC-OS (from Acorn) did it.

The hook turned out to be, instead of looking at ./folder.jpg, it went to ./.DirIcon. So a simple loop in bash suffices: 

find -name folder.jpg|while read
do
	ln -s folder.jpg "`dirname "$REPLY"`/.DirIcon"
done

Easy eh? Fair enough it uses another inode, but with most of the files being >100mb, I'm not likely to run out.

Sat, 23 Jan 2010 18:53:32 +0000

First semester of MSc

After the summer i started a masters course in Advanced Computer Science (still at Liverpool). To be honest the first month and a half was pretty unbearable, as the university had not confirmed my place until the day i was meant to register. (whilst I was still in Caithness).

This caused a number of problems, the primary one being that I was in desperate need of accomodation. I checked out LSH, but there was nothing hopeful. Also, I've been burnt with LSH accredited landlords and properties in the past so thought it was best to avoid the whole thing. After about 6 weeks of couch surfing (with some surprisingly understanding/generous friends) I found a shared house not far from where I lived two years back.

The MSc itself seems to be going well.. Exams are this week though (but I had another post I wished to make and figured that this one should come first. Privacy and Security (COMP522) is quite enjoyable, and I'll be walking into the exam with 25% already because I got perfect scores on the assessments. Advanced algorithmic techniques is slightly worse, but most of that is due to the practicals being poorly specified, deliverables being misleadingly stated, and the ommision of a number of things (which actually account for most of the work - for example a simple graph algorithm is not difficult, writing something to read in a graph frmo a text file is much more fiddly than the algorithm itself.)

Anyway, later ;)

Mon, 20 Jul 2009 14:27:41 +0100

Initialisation of vtables in static globals

While working on my kernel, I came across an odd problem that had been making me consider rewriting in C (though I may end up doing that anyway to reduce complexity) - Virtual functions did not work, and the results were odd. 

extern Console *con;
class Base {
public:
	virtual void do_something() {
		con->print("\nCalled base class\n");
	}

};
class Sub : public Base {
public:
	virtual void do_something() {
		con->print("\nCalled derived class\n");
	}
};
Base b1;
Sub s1;
Base *b2;
void test_it() {
	s1.do_something();
	b1.do_something();
	b2=&s;
	b2->do_something();
	Sub::Sub(s1);
	s1.do_something();
	Sub s2;
	s2.do_something();
}

The odd thing is that even though the static instances do not have any attributes you could expect (at least for s1) that the vtable would be set as static data (or even rodata as the type can not change).

Of these, s2 and b2 perform as expected - on s1 however the vtable is uninitialised. This means that (on bochs/qemu anyway) you are dereferencing a null pointer (which will cause a panic).

Once the constructor is called (Sub::Sub(s1)) it works as you'd expect. Was just a bit weird when certain things worked and others didn't - I had seriously expected it to pre-initialise the classes as the data was known at compile time. Should have read the ABI spec more thoroughly. ;)

Tue, 30 Jun 2009 20:19:20 +0100

Stuff i need to get on this site

Here are a few things I want to get here eventually. Just a case of writing them up.

Tue, 30 Jun 2009 19:11:18 +0100

Graduated

Well, I'm done with university now. I got a 2:2 which was less than I was expecting. However it is still a degree. The job hunt is occuring and there are a few promising leads - just nothing quite yet.

As a result I have had to relocate this site, there are a few issues here:

Fasthosts subdomains don't work right. They frame the index page - which means you cannot link to somewhere inside a subdomain. Even due to my dislike of javascript I was forced to remedy this with small (2-line) script at the bottom of the page.

Google points to my university site, and having not messed with .htaccess in a long time I couldn't get it right. however it works now

On a different note I've been working on a couple of personal projects, such as a software 3d renderer. It is capable of loading simple (as in triangles only, no textures or normals - but then the normals for triangles are implicit). At the moment the only thing I have been testing with is the Stanford Bunny. I intend to add in at least gouraud shading and possibly texturing.

Relating to this, I'm going to have to write my own filling/shading/texturing system, which could probably be released as a seperate project - at the moment I am using SDL_gfx just so I could concentrate on the mathematical aspects. The other thing I see a lot in code that uses SDL is the number of times people switch(){} on the pixel format of a surface. Wouldn't it be better to write a seperate method for each format rather than doing a comparison each time?

Anyway, that's all for now - I will try and update this more often...

Sun, 21 Dec 2008 02:28:50 +0000

revisiting my OS kernel

So today I decided to try doing a bit more work on my operating system. A few months back I got to the ooint of multitasking and managing memory.

However I stopped as I couldn't quite get the startup code right - the main problem was I wanted to avoid introducing big chunks of assembly language into it (which would hurt portability).

The way I was doing things previously was mapping the entirety of physical memory into the kernel address space. Any page that needed to be copied or zeroed could be done easily - the problem is that this puts a limit on the maximum amount of physical memory accessible.

For those unfamiliar with operating system development, it is common practice to reserve a chunk of the address space for use by the kernel. As you create processes, this part of the page directory is copied for each user-mode task.

This makes it difficult for the kernel to allocate memory after the initial startup because some of the page directories will not include these mappings. This is undesirable because it would mean having to walk the task list and update the relevant address mappings (on x86 this is a 2kb copy per process, assuming a 2g/2g user/kernel split)

So today I go and dig through the linux memory manager (which is surprisingly well written/commented for linux). It turns out that this works exactly the way I was trying to avoid. Shock horror.

The need to map all of memory into the kernel partition does not seem to be all that convincing. Any memory besides internal kernel structures and hardware registers isn't very important. Anything that does need to be touched would only be so when servicing a system call - in which case it is part of the user-mode page directory which is still used while servicing a software interrupt.

So... here's the question - can anyone offer a good reason for doing this, or a simple alternative mechanism. Most of the complexity exists during initialisation where there is only one kernel thread. But as I say, I would like to avoid a lengthy startup procedure written in assembly language - portability is one of the aims I have for this project.

Mon, 29 Sep 2008 01:39:00 +0100

Getting an RT2500 based card working with WPA in Gentoo

For some reason this card doesn't work with wpa_supplicant, so you need to use iwpriv (essentially a command line interface to ioctl for wifi devices) to set the various parameters by hand.

If you read the readme, it says the kernel module will read a file /etc/wireless/RT2500STA/RT2500STA.dat which you should create.

Turns out it does not read this file at all. I wrote a small bash loop to do the relevant work, and tbh it did the job fine.

start_ra0.sh
#!/bin/sh
GATEWAY="192.168.2.1"
IP_ADDR="192.168.2.2"
rmmod rt2500 && sleep 5
# load the driver as it's not in the vanilla kernel
modprobe rt2500
# the first line of the config file is [Default]. Don't process it.
grep -v Default /etc/wireless/RT2500STA/RT2500STA.dat  | while read
do
# do the ioctl's
	iwpriv ra0 set "$REPLY"
done
# IP address (replace this with your dhcp client if you use dhcp)
ifconfig ra0 up "$IP_ADDR"
# default gateway
route add default gw "$GATEWAY"

This of course assumes that you read the README in the CVS repository. Some of the commands will error out, don't worry. Also in iwconfig it does not show the correct encryption key. Finally it can sometimes get confused with multiple access points - just bring down the interface, unload the module and reload it.

Wed, 17 Sep 2008 20:53:40 +0100

Abuse of the Ternary operator

A friend of mine asked if some code looked ok:

correct me if i'm wrong, but

	d[i][j] = (del>ins)?(sub>ins)?ins:sub:(sub<del)?sub:del;

will stick the smallest of del, ins and sub in d[i][j] right?

While would appear to be correct, the fact that it needed to be asked should have set off warning bells. If you can't understand your code when you write it, how do you expect to do so a year or more down the line?

I instead suggested to him something like: 

d[i][j] = Math.min(del,Math.min(sub,ins));

may be more appropriate. I thought to myself "java has varargs now..."

public <T extends Comparable> T min(T... values) {
    T ret = values[0];
    for(T i:values) {
        if(ret.compareTo(i)>=0)
            ret = i;
    }
    return ret;
}
Fri, 12 Sep 2008 19:27:11 +0100

finally passed math161!

So I passed it.... That is a real load off my mind. I can go back to concentrating on computery stuff rather than stressing about my future.

Granted it might not look good to future employers that I repeatedly failed an introductory module, However that is part of the purpose of this site - to create a portfolio. I will start working on writing a couple of my other projects up. Also I have a few Delphi components to write, which I believe may be useful to a certain company.

The CMS has undergone a major rewrite and documentation update. However I have not yet moved this site to the new system. Mostly because I am still unable to log into the linux farm at CSC. I have confirmed this problem with a number of my peers, and plan on popping into the helpdesk when I get back to the 'pool.

Tue, 09 Sep 2008 16:21:55 +0100

Firefox doesn't do self-closing tags

So turns out that Firefox doesn't respect self-closing tags, leading to all sorts of problems when dealing with anchors. You would presume something like: <a name="foo" /> would be legal, but not as far as the mozilla/netscape people thing.

This is what was causing the odd behaviour of the W3C core styles. I'm going to have to dig through with the DOM inspector and see if it disregards self-closing img tags too.

Thank god I'm not incorporating javascript into this CMS....

Sun, 07 Sep 2008 18:21:29 +0100

Rewrote the CSS

So I bit the bullet and rewrote the CSS for the site. I'm not sure how well it worked out, but will make the site layout more customisable. For those of you who want to stick with tables, there is an option USE_TABLES in the source.

As a result of the new colour scheme, the core styles don't really work any more.. I'd like to find a way to make them override the other styles rather than cascading... idk, it would sort of defeat the point of Cascading style sheets. Maybe I should just create a couple of example themes and work from there.

On a related note, I've come across some ideas on how to improve the presentation of some parts of the site (say, the blog) but it would require a couple of changes I'm not really comfortable with, and seem a bit hacky. Something along the lines of this. I would like to have this as a feature availiable (I don't really like the idea of using actual images), but think this would warrant some refactoring of the CMS source in order to do it in a sane way.

It is probably in need of some refactoring anyway, there're a few other things I would like to support, such as multiple 'blogs' (could be news etc), categories for projects and posts, additional pages which can be user defined.

Sat, 06 Sep 2008 04:36:07 +0100

added a few alternative stylesheets

I've added the W3C's Core Styles as alternative stylesheets. This is mostly because I reckon mine needs some improvement and was looking for an easy way to check out alternative looks.

I'm also thinking I should drop the table-based layout and go for CSS and DIV tags. I'm not sure on this though. I'd presume all the modern browsers support this quite well, however I'm sort of unable to test anything other than Gecko at the moment because I'm on 64 bit gentoo, so no MSIE, Opera seems to have real problems (I think it's because I tried to install the flash plugin :o). Granted I could throw these things on my BSD box, but then it becomes a bit more of a pain.

Anyway to check out these styles, (in firefox this is, presumably the same on other netscape-based browsers) go to View->Page Style and pick one. Personally I don't really like the way they add emphasis onto text seemingly at random (as in, some paragraphs but not others).

Sat, 06 Sep 2008 00:00:28 +0100

ATOM feed support

So I added atom (RFC5023) support to this blog. I thought it would be quite funny as it is technically a static website. Not linked to it from anywhere yet though.

There was an interesting problem when trying to pass the validator, it was falling over on the date strings, even though I was generating them with the --rfc-3339 option to date(1). Turns out either the spec is inaccurate, or there's a bug in the date command.

I ended up solving it with a format string to date:
date --date "1970-01-01 ` stat -c "%X" "$1"` sec" +%FT%T%:z
Which is a bit messy, but I don't see another way to get date to take a time in seconds since the epoc.

Also, I changed the stat call today, as I discovered that on systems with access times disabled, the atime provides the only copy of the creation time (no, surprisingly enough, ctime changes along with mtime :/ )

Going to have to update the README again....

Thu, 04 Sep 2008 16:14:52 +0100

Getting my compiler working on 64 bit

I decided to have a stab at writing up my Brainfuck compiler. I didn't get very far though. It it probably as complex as the logic program, and there are two seperate versions of it (one in C, which produces code which uses the C library) and one in C++ (which produces plain old Linux assembly language).

So I tried running the compiler, I've a couple of handy Makefile macros to make working with it easier: 

%.o: %.c %.h
	$(CC) $(CFLAGS) -c $<
%.o: %.asm
	$(ASM) $(ASMFLAGS) -l $<.lst $<
%.asm: %.bf bfc
	./bfc  $< > $@
%: %.o
	ld $(LDFLAGS) -o $@ $<

So say there is a bf source file 'mandelbrot.bf' then you can simply do make mandelbrot and it will spew out the binary thanks to these rules. Well, that's the idea anyway.

The problem came when I tried to do this on my new system (64 bit gentoo remember): 

ld: i386 architecture of input file `mandelbrot.o' is incompatible with i386:x86-64 output
ld: warning: cannot find entry symbol _start; defaulting to 00000000004002b0

This was somewhat annoying, as it was around 4 AM. I knew what the problem was, just couldn't find the solution for ages. I tried -m32 on the LDFLAGS, but that didn't help. It turned out to be -m elf_i386.

Next thing I tried was running it on freebsd 7 (which is running inside virtualbox). Both of the compilers spat out executables (of course the ld flags needed yet another tweak). One segfaulted (the one which was using the C putchar/getchar). The other ran, but did nothing. Both the interpreters worked fine though.

So I fired up ald on the first one to try and find where it was segfaulting, which didn't turn up much of use.... it was somewhere deep inside libc. On a side note I found some quite interesting snippets of assembly in the libraries. Things like xchg eax,eax which is quite obviously spat out by the compiler, but I can't see the use of it.

The C++ one didn't work for a few reasons (actually surprised it didn't cause some sort of nasty behaviour or a protection fault)

  1. Linux has different system call numbers to freebsd.

    FreeBSD is actual UNIX, Linux is not. There are clearly going to be differences in how things work.

    Large chunks of Linux (at least to begin with) were written without access to the POSIX documentation, if you read Linus' book "Just for fun" he mentions that it did occur to him afterwards that he should have kept the identifiers the same to maintain binary compatibility. Same with errno.

  2. Linux has a different calling convention to freebsd.

    While linux passes it's arguments to int 0x80 in registers, freebsd uses the stack (supposedly to save pushing/popping values, but it has to do that on the other side anyway, and with a chip like x86 you have to spill to the stack all the time anyway)

The (temporary) solution to this is to brand the resultant executable as a 'Linux' one, and run it through the compatibility layer:

$ brandelf -t Linux name-of-executable

At which point it ran fine. Retargetabiliy is on my list of things for the next version (these were written over the course of a week back in 2007 for a bit of fun.) I have since read a lot more about the ELF format, and have a pretty good idea how to construct an executable. Ideally I'd like to have it work across different processor families as well, even to the likes of VAXen (now I've got one running in emulation, but that's another story)

Wed, 03 Sep 2008 21:18:32 +0100

added requisite management

so I have added management of what I shall call 'requisites' to the CMS. This means it will keep a list of all the files required for the site, and make sure they're included when it is copied to the server. (or at least give an error if one is missing).

This was actually really difficult because of a one-line function. 

 REQUISITE_FILE="assets.txt" 
 function add_requisite { 
 	echo "$1" >> "$REQUISITE_FILE" 
 }

It was only adding the html files and style.css. Can you see why? I spent ages trying to dig to the bottom of this. Shows what happens when you've been up for days.

The problem is because of the use of pushd/popd in the maker script. It was sending the correct strings to the file, just the wrong file. sure enough a find -name assets.txt showed one in each of the subdirectories. Man I felt an idiot, I was almost on the brink of asking for help. :o

The solution was to take a copy of `pwd` when the script was run, and prepend that to the path. Now it all works fine. ;)

Also I added timestamps and anchors to the blog posts. I'll get around to doing an index soon enough.

Finally, I made the display of download/paper/readme optional, so you don't get "sorry not availiable" instead of a link. Should make it more relevant and less cluttered.

Wed, 03 Sep 2008 17:52:10 +0100

So I've upgraded the link handling system....

Firstly, there are three options on how to display the links in the sidebar. TEXT just prints the name of it, and omits the image, IMAGE does the opposite, and BOTH will do both (like you'll see if you look across now).

Additionally I went and grabbed the favicons from the various sites, and resized them with ImageMagick [sic] so they are all pretty uniform now. In the maker script you can now define either the width or height of images (or both) and it'll fix them to that size, however I think having them all the same size to begin with is a better approach.

Finally, there is a links page, which adds a short description to each of them. I'll get around to making the sidebar just show a couple (say the top 5).

Wed, 03 Sep 2008 12:41:17 +0100

Now this site validates as XHTML 1.0 transitional

Valid XHTML 1.0 Transitional

go on, give it a go ;)

The main change was that I had to make most of the blog posts and project descriptions use the _raw suffix so that they don't go through the paragraphing function in maker.

I had to do this for anything that uses <pre> or list tags, as they are not allowed inside <p>. At some point I will get around to writing a proper HTML sanitizer (I dare say a relatively powerful one could be constructed with sed), as currently I've still got to go through everything and insert the digraphs by hand.

The nice thing was there weren't any major troubles. Just minor presentation errors which were easily fixed.

Anyway, enough. I've had to type this twice now because I accidentally quit without saving the file :/.

Wed, 03 Sep 2008 01:00:51 +0100

Javas: simple java build system

This is a script I wrote a while back, but it comes in useful whenever I'm dealing with any non-trivial java project that doesn't warrant using Ant. 

#!/bin/bash
# (C) Richard Whitty 2008
# A few bash functions which will probably come in useful to java programmers.
# finds files with the extention provided.
# $1 the extenstion to look for
# if we are to be verbose
function find_by_extenstion {
        find  -name \*.$1
}
# gets a list of all class files
function get_class {
        find_by_extenstion class
}
# gets a list of all class files
function get_src {
        find_by_extenstion java
}
# removes class files
function clean {
        get_class | while read
        do
                rm $OPTIONS "$REPLY"
        done
}
# checks if a source file is due for a rebuild.
function needs_rebuild {
        local srcfile="$1"
        local binfile="${srcfile:0:${#srcfile}-4}class"

        if [[ "$srcfile" -nt "$binfile" ]]
        then
                return 1
        else
                return 0
        fi
}
function get_sources_to_be_updated {
        get_src|while read srcfile
        do
                needs_rebuild "$srcfile"
                if (( $? == 1 ))
                then
                        echo $srcfile
                fi
        done
}
# only does the ones that have been updated
function rebuild {
        get_sources_to_be_updated | while read
        do
                echo "javac $REPLY"
                javac "$REPLY" || break;
        done
}
# builds the sources starting from the current directory.
function build {
        get_src | while read
        do
                echo "javac $REPLY"
                javac "$REPLY" || break;
        done
}
function tbuild {
        time build
}
# prints a handy word/line/char count for the source files.
function wordcount {
        if [[ $VERBOSE == 0 ]]
        then
                find -name \*.java -print0 |wc -l --files0-from=-
        else
                find -name \*.java -print0 |wc --files0-from=-
        fi
}
VERBOSE=0
for arg in $@
do
        if [[ "$arg" ==  "-v" ]]
        then
                OPTIONS=" -v "
                VERBOSE="1"
        else
                $arg
        fi
done
Tue, 02 Sep 2008 20:10:51 +0100

If you're going to ask for help

Quite often I get people contact me over various channels asking for help. You know who you are. Here are a few pointers for you:

  1. Either know what your problem is, or be willing to give me a shell Trying to either debug something or talk somebody through a solution to a problem is bad enough. When they can't adequately explain the problem, and don't get what you're saying it just becomes annoying.
  2. If you get advice and don't take it, don't come moaning when it doesn't work Quite often a conversation will go like this: 
    someone: so I've got this problem yadda yadda yadda
me: ok, so you need to do x, y, z.
someone: ok, I'll do x and z, but I'll also do B instead of y.
me: I'm telling you, this is what you need to do. Don't come crying to me when it doesn't work.
a while passes, normally by which point it's stupid-o-clock in the morning
someone: ok, so I did this and it doesn't work.
me: (pastes a copy of the previous conversation)
someone: hmm I'm so dumb I can't read simple instructions
(keeps bugging me for hours)
me: ok, fine give me a shell
(log in, fix the problem in like 5 minutes)
someone: hey that's really cool, how did you do that?
me: same way I told you to do it 6 hours ago. Next time there'll be an invoice coming at you!
    What these people don't seem to realise is that this sort of carry on, when it extends to ~7am, makes me less inclined to help you next time, and chances are you'll end up on my ignore list for a few weeks or until I remember to take you off.
  3. Read the $%^$% manual

    I can't stress this enough. if you come to me with a problem involving a pile of technologies I've never heard of, you are going to have to provide at least a basic overview of how it's meant to work. It's quite annoying having to dig through documentation for some product I'm not using, to help someone with something that I'm not being paid for. How do you think I figure these things out. (hint, man pages rock, google rocks - if you ignore the glaring privacy issues).

Tue, 02 Sep 2008 19:20:31 +0100

64 bit portability in general

So I've been thinking for a while about 64 bit code, and now I've got a 64 bit system (on gentoo of course) I can start trying to move all my code over.

To be honest, most of the things that will cause problems I'd already taken care of, because I'm not as sloppy as I once was. There are still a number of things to avoid:

Tue, 02 Sep 2008 17:37:19 +0100

Porting IVAN to 64-bit Linux

The other day I decided to bite the bullet and port IVAN to 64-bit. Now there were a couple of troubles with this, which I will go through:
  1. The reference counting mechanism. All the shared arrays and strings were allocated including a reference count at the beginning of the array, then storing the pointer as value+4. Strings were allocated like: 
    char *newString = 4 + new char[length + 5];
REFS(newString)=0;
    which isn't good. After some digging (ok, grep -n REFS */*|grep \#define we find REFS defined as: 
    #define REFS(ptr) reinterpret_cast<ulong*>(ptr)[-1]
    So we see the 4 is for the reference count value, and the +5 is the reference count plus the null byte on the string. As long is no longer 32 bits, this causes major corruption. A simple fix would just be to tell REFS to use an int instead (the reference count is only ever accessed through that macro) but in my opinion this wouldn't really solve the problem. These all got replaced with: 
    char *newString = sizeof(ulong) + new char[length + sizeof(ulong)+1];
    Then those parts worked. Similar changes were needed for the arrays, but you get the idea.
  2. Once all these had been fixed, there was a problem with the fonts, where each character drawn would draw a lot more than was needed horizontally. This was due to the original developers using unsigned longs assuming they were also 32 bit. As they are not on 64 bit (long is same size as a pointer) then this was doubling the amount that was being drawn. However this was quite hard to pin down. The game was playable at this point, just most of the text was illegible. Here is the offending code: 
        culong* EndPtr = FontPtr +5;
    (culong is a typedef for const unsigned long) I converted the code block to use integers instead: 
      for(; SrcLine != EndLine; ++SrcLine, ++SrcMaskLine, ++DestLine)
  {
    cuint* FontPtr = reinterpret_cast<cuint*>(*SrcLine + B.Src.X);
	cuint* EndPtr = FontPtr + (20/sizeof(uint));
 // this also works with longs, but may as well keep to the original plan.
    cuint* MaskPtr = reinterpret_cast<cuint*>(*SrcMaskLine + B.Src.X);
    uint* DestPtr = reinterpret_cast<uint*>(*DestLine + B.Dest.X);
    for(; FontPtr != EndPtr; ++DestPtr, ++MaskPtr, ++FontPtr)
      *DestPtr = *DestPtr & *MaskPtr | *FontPtr;
  }
    Now we have fonts that are legible, and the system doesn't segfault everywhere. Valgrind still throws up a couple of errors but I'll leave that for another day.
Tue, 02 Sep 2008 17:34:11 +0100

prodigal first post

This is the first blog post ever to be published using this CMS. You should feel damned privelidged that you get to see it.

And paragraph splitting works as well ;)

Or does it?