The RawSushi

Good site for AMD and 64 bit CPU news

http://sharikou.blogspot.com/

I came across this blog last week, and I must say it's the best aggregator of AMD and 64 bit CPU news I've ever come across. He adds his fair share of biased commentary but it's very forgivable coz they're really funny and if you're a discerning reader, you'll know what to take with a pinch of salt anyway.

Hardware RAID vs Software RAID on Linux
In the past, I've had many people tell me how Software RAID actually outperforms Hardware RAID, costs less and therefore Hardware RAID isn't really worth the money. There's also plenty of literature out there saying pretty much the same thing.

I've had more than my fair share of opportunities setting up servers and have tried the various options, and I must say my conclusion is that I'd take Hardware RAID over Software RAID anytime.

The thing about Software RAID is that there are simply too many "catches" and it doesn't quite work out-of-the-box as it should. Special care and consideration needs to be exercised or you may setup a Software RAID that may not even be easily recoverable in the event of a disk failure! The surprising thing is, this isn't really covered very much. Most of the literature out there is focused on performance and portability rather than this very important aspect of RAID--recovery.

Setting up a Software RAID first requires you to be very clear about your needs and priorities, and knowing how far you can go with what you have. For example:
1. Is uptime critical than data integrity, or is data integrity more important?
2. Do you have hot-swappable drives?
3. Do you have the skills to setup your RAID to cover all failure scenarios, and to perform the recovery in any kind of failure scenario?

The easiest failure recovery scenario is when both your OS and data partitions are on RAID 1, and when you have hot-swappable drives. You can swop in a new drive without shutting down the OS and then immediate reconstruct by using the "mdadm" program. If it's the primary drive that failed, just reinstall the bootloader and everything should be fine.

However, most people who go for software RAID are on budget machines with IDE HDDs, which do not support hot-swap. This is where it gets really tricky. Assuming again both OS and data partitions are on RAID 1. If it's your secondary drive that failed, your primary will still be able to boot after you shutdown to replace the drive and you can perform your reconstruction after it boots up. However, it it were your secondary, recovery will be really, really hard as the OS no longer boots, and I haven't even managed to figure out how to recover from this failure. I believe it can be done but it's just too hard and I don't expect mere mortals to be able to figure this out on his own.

The only way out I know is to have the OS partition outside of the RAID 1. However, if it's not on RAID 1, that means that the server will fail if the disk that the OS on fails. This means that to promote ease of recoverability of data, server uptime is compromised. If the OS partition is on RAID 1, then recovery can be very hard depending on which disk fails, but uptime is maintained even in the event of a disk failure.

Ultimate, software RAID works best when you have hot-swappable drives. But this really begs the question: If you can afford a server with hot-swappable drives, why not just go for Hardware RAID? It's much cleaner and simpler, to the OS it's just a single HDD, and recovery is straightforward regardless which disk failed.

Using Yum instead of up2date

For some reason I do not understand, Yum is so much faster than up2date. up2date can take forever just to resolve the dependancies whereas Yum is really straightforward and updates 300 packages within 2 hours if you've got a fast line. up2date could take just as long, if not longer just to sort out the dependencies.

As at Fedora Core 3, Yum pretty much worked out of the box. I had some difficulties with earlier versions but Yum is real easy now.

"yum check-update" will give you a list of packages to update

"yum update" will update the entire OS

or you can selectively update by specifying package or package wildcards. e.g.

"yum update xorg*"

Remotely Running X Window Programs

Through my life with Linux, I never really had much luck with remote X sessions. I think pretty much the easiest and safest way is to do it via SSH. No additional ports to open on the firewall, and works out of the box. Well, at least for Fedora Core 3 and 4.

To do it, go into X Windows on your client machine, just ssh into the box you wish with the option "-X" or "-Y" e.g.

ssh -X 192.168.2.2

And then start any GUI program from the command line e.g.

firefox &

or

xterm &

And the program runs within a window on your desktop.

Fedora Core 4
I just upgraded my PC to use Fedora Core 4 and really the most significant improvement for me was Open Office 2 beta. Open Office 2 offers so much better compatibility with MS Office, especially almost perfect compatibility when drawing diagrams, which was never possible before.

However, FC 4 did have its fair share of issues. The most significant problem I had was that Java 1.4.2 could not access the network at all. I would get the following exception:

java.net.SocketException: Invalid argument or cannot assign requested address

Upgrading to Java 1.5 solved the problem. Though a workaround, it can never be seriously considered as a production option since many production software out there are still being developed on Java 1.4.

The good news, however, is that this turned out to be a IPV6 related issue. According to a poster in this forum thread, the problem will be solved when IPV6 is disabled on the kernel by adding:

'alias net-pf-10 off'

to /etc/modprobe.conf, and rebooting.

That solved my problem

XPath Engines for XOM

XOM is one of the many XML Object Models out there today. It's Open Source (LGPL) and is developed with "correctness, simplicity, and performance" in mind.

XOM is the XML Object Model used for Memoranda.

XPath is "XPath is a syntax for defining parts of an XML document". More specifically, it can be used to query for parts of an XML document. With hierarchal tasks, the querying and searching needs in Memoranda became a lot more complex and a solution such as XPath has to be used.

Previously, the Memoranda Tasklist stored all tasks as a child object "task" under the "tasklist" element. Obtaining the task element needed was really just a case of iterating through all the "task" elements and finding the task that's needed.

However, with hierarchal tasks, iterating now became far more difficult as "task" elements can now be nested inside other "task" elements (i.e. nesting a child task within a parent task)

Hence, the need for XPath. As far as I know, there are current two XPath engines that support XOM (i.e. when you do an XPath query, it returns XOM Nodes). They are Jaxen and Nux. Jaxen's XOM support is still in Beta and none of the code is currently available in a release (at the time of writing at least). In order to check it out, I had to connect to the CVS. On the other hand, XOM support for NUX was available as a release. However, what tipped the balance in the favor of Nux was that it supports XPath 2.0 (which is still in draft but supports more complex queries).

An example of how Nux is used in Memoranda is when we need to retrieve a task from the tasklist based on the task's ID, regardless of where it is in the task tree (i.e. top level or embedded within a parent task or many levels or parent tasks).

private Element getTaskElement(String id) {
Element el = (Element) XQueryUtil.xquery(_doc, "//task[@id='" + id + "']").get(0);
return el;
}

The path "//task[@id='" + id + "']" requests for Nux to return all "task" elements with a particular ID, no matter where it is in the document. It returns a XOM Node which can be cast into a XOM Element.

With XPath support, more complex queries can be done and this could open the door to many more cool features in Memoranda in the future,

Memoranda Development Update

As the team has decided to make a 1.0 release soon, hierarchal tasks will not be in this version. Instead, it will be in the 1.1 version. Of course, those of you who want the latest features can always checkout the latest code from the CVS, or wait for the next development snapshot.

Sorry, I can't really give any dates to any of the above because I don't work on the releases. I just write the code :P

In J2EE, sometimes portability is just not worth the cost

One of the key bragging points about J2EE is how applications can be portable across the various J2EE application servers. And there's been a lot of emphasis on writing "portable code" and avoiding application server specific features. But there're times where you really need some features really badly and the J2EE spec is simply inadequate and slow in delivering some features, while your app server tempts you with a proprietary implementation that works! So what do you do?

Well I never really had to face this question until recently, when I found myself trudging waist deep in mud with CMP Entity Beans. It came to a point where we had to implement querying of Aggregates such as "COUNT / SUM /AVG" in the Entity Beans, and this meant using EJB Select Queries rather than Finder Queries. Under Finders, Selects take double the amount of code and XDoclet annotations to get working because Select methods are private the bean class and cannot be exposed to the remote caller. Instead, it has to be wrapped around another method with exposes it. It just got to a point where it got so painful I decided that the solution was to switch a JBoss solution -- Dynamic QL.

Dynamic QL allows you to create just one set of methods to service ALL Finder and Select queries. In the past, every single Finder query needs to be defined in the Entity Bean via XDoclet annotations and XDoclet has to be run everytime after a Finder was added, and when a Select method is needed, well, it's quite a lot of work.

Using Dynamic QL means that XDoclet needs only to be run once, as only ONE Dynamic Select method needs to be implemented. The Query can be for an Object or just a single field (note that multiple column selects are not available though). The actual query to be run is instead passed into the Entity Bean at runtime and executed. By doing so, development time is cut significantly.

Furthermore, since portability has already been violated, why not just go ahead and use JBoss QL for the query anyway? It has more features than EJB QL, and allows more queries to be made truly database independant.

By using Dynamic QL and JBoss QL, I have cut development time and writing database independant code has become so much easier. There is a price to pay for app server portability and sometimes it's not worth it, especially when you're not expecting to switch app servers at any point in time anyway.

Multi-level tasks definitely coming your way soon in Memoranda

A basic implementation of multi-level tasks for Memoranda has been completed. It is now committed to the CVS and it's being tested.

This feature has been asked for by quite a number of users in the mailing list and it got worked on after Michael Bernadsky came up with a relatively simple way to do it.

A summary of the implemented features:
1. The agenda page was changed to allow recursive expansion and
closing of parent class nodes. Click the (+) and (-) beside the parent
task

2. To add a subtask, select and right click on the parent task in the
task panel and select "Add sub task"

3. The task panel will only show top level tasks by default from now on

4. To drill down to the sub-tasks, select and right click on the
parent task in the task panel and select "List sub tasks"

5. To back-out of a task and go up one level, right click anywhere on
the task table and click "Parent task"

6. In terms of data validation, very little is being done. One of the
very few examples is that you cannot delete a parent task while it has
sub-tasks. In terms of dates, effort and completion, there is no
validation at all. This is DELIBERATE. I want to keep the system
flexible so that people can enter whatever values they want.

7. However, for those who want to keep their project numbers tight all
the time, select the parent task, right click "calculate task data"
and you can do the following:
a. make all the start/end dates consistent. i.e. if sub-task start
date is earlier than parents, the parent's start date will be moved
forward
b. make all the "effort" fields consistent. Parent task's effort
field will be taken from a total of the sub-task's "effort" fields
c. Calculate all parent task's task completion. This is weighted
by the effort of each task. If no effort is filled in, the task is
assumed to have 1hr of effort.

Watch out for it in the next release!

Multi-level tasks for Memoranda

The power of the Open Source development model is in the way it taps the ideas and efforts of many people all around the world. What originally started as a "wishlist" feature became a feature that we can start implementing immediately, thanks to users who care enough to think about how a problem can be solved with the minimum effort.

Multi-level tasks is going to be such a cool feature because it's such a true reflection of how we work. Within big projects can always be broken down into sub-projects which in turn have their own sub-tasks.

With my hectic schedule, I don't know how long it'll take me to do this if I'm gonna be doing this on my own, but with the Open Source model, well.... I know I won't be on my own...

Remote Administration of Windows Servers from Linux!

I never thought this would be possible but I just found out today that with the "rdesktop" program, you can actually remotely administer a windows application by bringing the windows desktop onto X Windows!

No special configuration needed, just "rdesktop server_host" and viola, the windows desktop is on your X Windows.

Well, you learn something new everyday!