Wednesday, June 22, 2005
Hardware RAID vs Software RAID on Linux
In the past, I've had many people tell me how Software RAID actually outperforms Hardware RAID, costs less and therefore Hardware RAID isn't really worth the money. There's also plenty of literature out there saying pretty much the same thing.
I've had more than my fair share of opportunities setting up servers and have tried the various options, and I must say my conclusion is that I'd take Hardware RAID over Software RAID anytime.
The thing about Software RAID is that there are simply too many "catches" and it doesn't quite work out-of-the-box as it should. Special care and consideration needs to be exercised or you may setup a Software RAID that may not even be easily recoverable in the event of a disk failure! The surprising thing is, this isn't really covered very much. Most of the literature out there is focused on performance and portability rather than this very important aspect of RAID--recovery.
Setting up a Software RAID first requires you to be very clear about your needs and priorities, and knowing how far you can go with what you have. For example:
1. Is uptime critical than data integrity, or is data integrity more important?
2. Do you have hot-swappable drives?
3. Do you have the skills to setup your RAID to cover all failure scenarios, and to perform the recovery in any kind of failure scenario?
The easiest failure recovery scenario is when both your OS and data partitions are on RAID 1, and when you have hot-swappable drives. You can swop in a new drive without shutting down the OS and then immediate reconstruct by using the "mdadm" program. If it's the primary drive that failed, just reinstall the bootloader and everything should be fine.
However, most people who go for software RAID are on budget machines with IDE HDDs, which do not support hot-swap. This is where it gets really tricky. Assuming again both OS and data partitions are on RAID 1. If it's your secondary drive that failed, your primary will still be able to boot after you shutdown to replace the drive and you can perform your reconstruction after it boots up. However, it it were your secondary, recovery will be really, really hard as the OS no longer boots, and I haven't even managed to figure out how to recover from this failure. I believe it can be done but it's just too hard and I don't expect mere mortals to be able to figure this out on his own.
The only way out I know is to have the OS partition outside of the RAID 1. However, if it's not on RAID 1, that means that the server will fail if the disk that the OS on fails. This means that to promote ease of recoverability of data, server uptime is compromised. If the OS partition is on RAID 1, then recovery can be very hard depending on which disk fails, but uptime is maintained even in the event of a disk failure.
Ultimate, software RAID works best when you have hot-swappable drives. But this really begs the question: If you can afford a server with hot-swappable drives, why not just go for Hardware RAID? It's much cleaner and simpler, to the OS it's just a single HDD, and recovery is straightforward regardless which disk failed.
In the past, I've had many people tell me how Software RAID actually outperforms Hardware RAID, costs less and therefore Hardware RAID isn't really worth the money. There's also plenty of literature out there saying pretty much the same thing.
I've had more than my fair share of opportunities setting up servers and have tried the various options, and I must say my conclusion is that I'd take Hardware RAID over Software RAID anytime.
The thing about Software RAID is that there are simply too many "catches" and it doesn't quite work out-of-the-box as it should. Special care and consideration needs to be exercised or you may setup a Software RAID that may not even be easily recoverable in the event of a disk failure! The surprising thing is, this isn't really covered very much. Most of the literature out there is focused on performance and portability rather than this very important aspect of RAID--recovery.
Setting up a Software RAID first requires you to be very clear about your needs and priorities, and knowing how far you can go with what you have. For example:
1. Is uptime critical than data integrity, or is data integrity more important?
2. Do you have hot-swappable drives?
3. Do you have the skills to setup your RAID to cover all failure scenarios, and to perform the recovery in any kind of failure scenario?
The easiest failure recovery scenario is when both your OS and data partitions are on RAID 1, and when you have hot-swappable drives. You can swop in a new drive without shutting down the OS and then immediate reconstruct by using the "mdadm" program. If it's the primary drive that failed, just reinstall the bootloader and everything should be fine.
However, most people who go for software RAID are on budget machines with IDE HDDs, which do not support hot-swap. This is where it gets really tricky. Assuming again both OS and data partitions are on RAID 1. If it's your secondary drive that failed, your primary will still be able to boot after you shutdown to replace the drive and you can perform your reconstruction after it boots up. However, it it were your secondary, recovery will be really, really hard as the OS no longer boots, and I haven't even managed to figure out how to recover from this failure. I believe it can be done but it's just too hard and I don't expect mere mortals to be able to figure this out on his own.
The only way out I know is to have the OS partition outside of the RAID 1. However, if it's not on RAID 1, that means that the server will fail if the disk that the OS on fails. This means that to promote ease of recoverability of data, server uptime is compromised. If the OS partition is on RAID 1, then recovery can be very hard depending on which disk fails, but uptime is maintained even in the event of a disk failure.
Ultimate, software RAID works best when you have hot-swappable drives. But this really begs the question: If you can afford a server with hot-swappable drives, why not just go for Hardware RAID? It's much cleaner and simpler, to the OS it's just a single HDD, and recovery is straightforward regardless which disk failed.
Tuesday, June 21, 2005
Using Yum instead of up2date
For some reason I do not understand, Yum is so much faster than up2date. up2date can take forever just to resolve the dependancies whereas Yum is really straightforward and updates 300 packages within 2 hours if you've got a fast line. up2date could take just as long, if not longer just to sort out the dependencies.
As at Fedora Core 3, Yum pretty much worked out of the box. I had some difficulties with earlier versions but Yum is real easy now.
"yum check-update" will give you a list of packages to update
"yum update" will update the entire OS
or you can selectively update by specifying package or package wildcards. e.g.
"yum update xorg*"
For some reason I do not understand, Yum is so much faster than up2date. up2date can take forever just to resolve the dependancies whereas Yum is really straightforward and updates 300 packages within 2 hours if you've got a fast line. up2date could take just as long, if not longer just to sort out the dependencies.
As at Fedora Core 3, Yum pretty much worked out of the box. I had some difficulties with earlier versions but Yum is real easy now.
"yum check-update" will give you a list of packages to update
"yum update" will update the entire OS
or you can selectively update by specifying package or package wildcards. e.g.
"yum update xorg*"
Remotely Running X Window Programs
Through my life with Linux, I never really had much luck with remote X sessions. I think pretty much the easiest and safest way is to do it via SSH. No additional ports to open on the firewall, and works out of the box. Well, at least for Fedora Core 3 and 4.
To do it, go into X Windows on your client machine, just ssh into the box you wish with the option "-X" or "-Y" e.g.
ssh -X 192.168.2.2
And then start any GUI program from the command line e.g.
firefox &
or
xterm &
And the program runs within a window on your desktop.
Through my life with Linux, I never really had much luck with remote X sessions. I think pretty much the easiest and safest way is to do it via SSH. No additional ports to open on the firewall, and works out of the box. Well, at least for Fedora Core 3 and 4.
To do it, go into X Windows on your client machine, just ssh into the box you wish with the option "-X" or "-Y" e.g.
ssh -X 192.168.2.2
And then start any GUI program from the command line e.g.
firefox &
or
xterm &
And the program runs within a window on your desktop.
Fedora Core 4
I just upgraded my PC to use Fedora Core 4 and really the most significant improvement for me was Open Office 2 beta. Open Office 2 offers so much better compatibility with MS Office, especially almost perfect compatibility when drawing diagrams, which was never possible before.
However, FC 4 did have its fair share of issues. The most significant problem I had was that Java 1.4.2 could not access the network at all. I would get the following exception:
java.net.SocketException: Invalid argument or cannot assign requested address
Upgrading to Java 1.5 solved the problem. Though a workaround, it can never be seriously considered as a production option since many production software out there are still being developed on Java 1.4.
The good news, however, is that this turned out to be a IPV6 related issue. According to a poster in this forum thread, the problem will be solved when IPV6 is disabled on the kernel by adding:
'alias net-pf-10 off'
to /etc/modprobe.conf, and rebooting.
That solved my problem
I just upgraded my PC to use Fedora Core 4 and really the most significant improvement for me was Open Office 2 beta. Open Office 2 offers so much better compatibility with MS Office, especially almost perfect compatibility when drawing diagrams, which was never possible before.
However, FC 4 did have its fair share of issues. The most significant problem I had was that Java 1.4.2 could not access the network at all. I would get the following exception:
java.net.SocketException: Invalid argument or cannot assign requested address
Upgrading to Java 1.5 solved the problem. Though a workaround, it can never be seriously considered as a production option since many production software out there are still being developed on Java 1.4.
The good news, however, is that this turned out to be a IPV6 related issue. According to a poster in this forum thread, the problem will be solved when IPV6 is disabled on the kernel by adding:
'alias net-pf-10 off'
to /etc/modprobe.conf, and rebooting.
That solved my problem
