Max HDD size / Hot swap SATA in IBM System x3550 server

[Norway Pad]

Not a workstation, but still an IBM, so I'll try:

I got a hold of a outdated IBM System x3550 Type 7978 rack server from my wife's work, and we decided to bring it home and install Server 2012 on it and use it to practice and play around with. I installed the OS last night, and it runs just fine, but the standard two 3,5" drives it comes with are pretty small, 160GB. I set the ServeRAID up as RAID 1, so it gives me only 160GB. I would like to replace these drives with Western Digital Red series drives, but I have two questions in this respect that I hope someone here might be able to help with.

How big total drive capacity can this unit handle? I have downloaded the manual and looked everywhere, but the maximum capacity isn't stated anywhere. The goal would be at least 2 x 2GB drives, maybe even 2 x 3GB drives if the unit can handle that.

The unit has the Hot Swap trays, and the manual says it supports "3.5-inch hot-swap Serial ATA (SATA) hard disk drives". So is "hot-swap SATA" a different interface than regular SATA? Or is it just a regular SATA drive with electronics that handles being unplugged and replugged while the machine interface is powered on? And in that case: Can I use regular SATA drives here as long as I avoid hot-swapping them? In a home environment there won't be a need for that.

I have checked the drives that it came with, and I don't see anything specified on them. They are Maxtor SATA drives with a IBM FRU label.

Thanks in advance!

[bit_twiddler]

You could start by looking here:

http://ps-2.kev009.com/pccbbs/pc_server ... 1dxmst.pdf

Hot swap sata is a motherboard feature - any sata drive should work.

[Norway Pad]

Thanks for the go on the hot swap part. It was my assumption too that it would work that way.

For the size, this is the manual I downloaded too. But neither the specifications part, nor the installation or configuration chapters say anything about maximum supported drive size.

Edit: After searching more, I found this link: http://www-03.ibm.com/systems/info/x86s ... W7626.html which states that a "IBM Server 1TB 7200 SATA 3.5" HS HDD" is compatible. So at least 1TB seems to work. I would ideally like at least 2TB, but we'll see.

[bit_twiddler]

Starting on page 301 or so this document:

http://www.redbooks.ibm.com/redpapers/pdfs/usxref-w.pdf

describes the IBM x3550 type 7977, which is probably pretty
similar to your unit. It mentions that the 7977 uses an Adaptec AIC-9580W
disk controller. If you can verify that your machine has the same
controller, then (depending on other factors, such as your BIOS version),
you may be able to determine from the controller specs
what capacities of drives it will support.

Just for background, there is this thread, which may be accurate:

http://www.tomshardware.com/forum/24617 ... size-limit

but stupid people do stupid things and other factors such as whatever dimwits
and their management specified/wrote your machine's BIOS may have
limited the machine to 1TB or so per disk.

Have you looked on the IBM site for BIOS updates? You might find that
they updated their BIOS periodically to modify maximum disk capacity.

[bit_twiddler]

And, here is a link to what appears to me to be the BIOS for the machine:

http://www-947.ibm.com/support/entry/po ... GR-5081712

I don't recommend flashing the BIOS unless you really need to, and if you do,
you would want to take some extra precautions (such as putting the machine
on a UPS, etc.)

But, it is interesting reading. I don't see anything which would change the
maximum disk size.

[Norway Pad]

Thanks for some good clues to follow up on. The datasheet for the unit that's closest to my unit's manufacturing date, CPU etc, is here listed as supporting 1TB with two 500GB disks. I hope it's more, and the other info I found might indicate that is at least supports 1TB drives. This link: http://www-03.ibm.com/systems/info/x86s ... W7626.html states that a "IBM Server 1TB 7200 SATA 3.5" HS HDD" is compatible.

As you also say, the BIOS change log doesn't specify any changes related to HDD size. So I think I will follow you advice on not flashing it unless I need to.

When I come to think about it, I did buy some 3.5" 2TB drives for another project some years ago, and think they still might be around here somewhere. If I find them, will install them just to see what happens, and I will keep you guys posted as to what I find out.

[bit_twiddler]

If you try your 2TB drives, you might start by installing the OS in a
small (< 200GB or so) partition just to get it to boot, or on another
drive. I'm not familiar with Windows Server, but some operating systems
have a limit for the partition which they boot from which is smaller
than the limit which they set for other partitions which are added
later on the same or another drive.

[erik]

How big total drive capacity can this unit handle? I have downloaded the manual and looked everywhere, but the maximum capacity isn't stated anywhere.

unless it's running a UEFI BIOS, the limit will be 2TB per drive.   this is the largest size a legacy BIOS can handle.   i've yet to see IBM write an artificial HDD size limit into their servers.

The goal would be at least 2 x 2GB drives, maybe even 2 x 3GB drives if the unit can handle that.

you could certainly run 2GB drives but i'd suggest 2TB.

[bit_twiddler]

There is this tool which purports to support your system.

http://www-947.ibm.com/support/entry/po ... d=tool-asu

But, I don't know how to use it, not being an IBM server user (ever!)
It's probably worth grabbing, anyway, just so that you have it
in case they take it down from their web site.

I always thought that the 2TB limit was one of the MBR, and that you could
get around it by setting up a GPT partition table.

I'm using 2 Linux systems which are hooked up to drives which are larger than 2TB,
and I never boot in UEFI mode - so there may be some interdependence between
Windows of various flavors and the BIOS.

Here's some info from a Dell forum page on the issue, there probably is a comparable
IBM forum thread, but I don't know how to find it. Anyway, the issues are probably
pretty similar, and the poster discusses a lot of them:

http://en.community.dell.com/techcenter ... and-beyond

[Norway Pad]

Thanks for the input. I grabbed the tool just in case.

I know that MBR has a limitation at 2TB, which I assumed could be bypassed by simply partitioning >2TB drives in smaller partitions. But if I need to use GPT instead of MBR and without the UEFI and so on, I assume that requires some tweaking that's a little beyond my skill set. So I would say that if I could use two 2TB drives here, I would be satisfied. If storage space ever becomes an issue, I can just buy an external disk cabinet for it.

I will dig out the two 2TB drives form where they are installed over the next days, and see how that works.

[Norway Pad]

My two 2TB WD20EARS drives were luckily still sitting in the old Compaq box. So I installed them in the IBM server, and the verdict is that it doesn't work. The ServeRAID Array Manager simply didn't see them. The Disk Utility part of ServeRAID did actually see them, listed with the correct name and everything, but that's all. I even did a low level formatting, which took all night and all day, but they were still invisible to the Array Manager. I tried to run the Windows installer too, but no drives visible there either.

So I will go ahead and buy the 1TB WD Red drives instead, as 1TB seems to verified as compatible. Not the capacity I originally hoped for, but cheaper and still sufficient space to play around with some virtual machines in Hyper-V. A separate NAS box will anyway be a better choice for file storage than the big and noisy server.

Thanks for all input!

[erik]

interesting.

how old is this server?   the x3550 appears to have multiple generations.   i gave up trying to find the original between all the noise.

[bit_twiddler]

2TB WD20EARS are not a good option for a RAID because they probably
use an extended recovery algorithm to recover bad reads. For RAID, you
need what WD calls "Time-Limited Error Recovery" which gives up recovery
of a bad sector after a short time. Consumer drives like WD Green drives
are typically dropped by the RAID controller because they are busy
trying to recover from a bad read for too long a period of time, and
the RAID controller marks them as bad drives.

You probably didn't get to that point, though. If you just formatted them
as not being part of a RAID, then Windows is not seeing them for
some other reason. But, the controller would have caused you
grief with them at some point, anyway.

[Norway Pad]

how old is this server?   the x3550 appears to have multiple generations.   i gave up trying to find the original between all the noise.

The date of manufacturing printed on the label is 2007 - 07 - 02. The closest match I found in the Redbook, ( http://www.redbooks.ibm.com/redpapers/pdfs/usxref-w.pdf ), with the correct CPU and RAM is 7978-A2y-A2x, with Xeon 5335 2.0GHz CPU, available date Feb 2007.

2TB WD20EARS are not a good option for a RAID because they probably use an extended recovery algorithm to recover bad reads. For RAID, you need what WD calls "Time-Limited Error Recovery" which gives up recovery of a bad sector after a short time. Consumer drives like WD Green drives are typically dropped by the RAID controller because they are busy trying to recover from a bad read for too long a period of time, and the RAID controller marks them as bad drives.

I remember reading about this back when I bought these drives, yes. This was in 2011, before the WD Red series came out. I never had any problems with the Green drives in RAID, but I never used them much, as the box I installed them in ended up not being used for what I intended. So they have been spinning for a total of maybe 50 hours. Anyway, even if the 2TB WD Green drives had worked in the x3550, my plan was still to buy the 2TB WD Red drives for it, as they are more suited for RAID and 24/7 use. Thanks for confirming this.

[erik]

The date of manufacturing printed on the label is 2007 - 07 - 02. The closest match I found in the Redbook, ( http://www.redbooks.ibm.com/redpapers/pdfs/usxref-w.pdf ), with the correct CPU and RAM is 7978-A2y-A2x, with Xeon 5335 2.0GHz CPU, available date Feb 2007.

ahh... ok.   that explains a lot.   i thought this system was much newer.   you very well could be limited to 1GB depending on the ServeRAID controller.   there might be a firmware update to handle 2TB drives, or the controller doesn't like your particular drives for some reason. edit: see my post below.

bit_twiddler has a great point about TLER so you might consider WD RED or an equivalent drive if using in RAID.   if only JBOD, any drive will do.

search redbooks for your controller chipset and see if there's a firmware update.   it's worth looking.

[Norway Pad]

Yes, in computer terms it's an oold system. It was decommissioned last year, but it runs Server 2012 just fine, so it's still good to play around with. And I am in the process of gaining knowledge about Hyper-V for certain new tasks in my own job, so it was perfect.

I simply pulled the RAID card out and looked, and it's an Adaptec ATB-205 32MB card. But I don't know if this card is the culprit, or if something else is. The controller itself apparently is an Adaptec AIC-9580W. I also found out that the RAID card and the drive backplane / connectors are actually SAS. But SAS is apparently compatible with SATA, so that should be no issue.

A quick search revealed no firmware updates, but I will look more. And if I have to stick with 1TB drives, I could sacrifice redundancy and run them as JBOD, and still get my 2TB of space. There will be a solution no matter what.

[erik]

found it.   the x3550 supports 2TB SATA drives per the datasheet.   something else must be the issue.

x3550 datasheet: http://www-07.ibm.com/systems/includes/ ... 32USEN.pdf

the backplane connectors will be SAS and as you said will work just fine with SATA drives.   you can't mix SATA and SAS in the same array but should be able to mix separate SAS and SATA arrays in this system.   however, some older controllers don't support simultaneous SAS and SATA arrays so you'll need to research that if you have both drive types.

assuming you don't have any SAS drives in the mix, i'm curious why your 2TB drives aren't working correctly.

[Norway Pad]

Thanks! I looked a lot, but never found this one.

Unless this is a newer version, it actually says here that it supports 2TB drives. I will at least have to try the 2TB drives one more time, and check all configuration settings. Even though the WD Green drives are not suitable for RAID, I don't see why the controller wouldn't see them.

[bit_twiddler]

That data sheet shows maximum internal storage, but I don't believe that it
says how big each drive can be.

My memory is a little fuzzy about that era, but I seem to recall that a lot of
servers supported different size SAS and SATA drives - SATA generally
being a bigger limit. I'd be surprised if your machine didn't support 2TB
drives in hardware.

You might try formatting the drives on another system, the partition tables
on your drives may conform to a standard which is too new for the BIOS
in your machine to support. Or, perhaps you could download a program
to do it on Server 2012.

Also, is there a way to tell what version BIOS you have? Generally,
you can get into BIOS setup by hitting a function key (F1 or F2 or ...?)
Workstations and Servers generally have custom BIOSs, so you
may actually have to explicitly turn on each SATA port (or tell it that
you have a SATA drive) in order for the machine to recognize that
a drive is even there.

[erik]

That data sheet shows maximum internal storage, but I don't believe that it
says how big each drive can be.

IBM always words their specs as the largest single component size.   compare HDDs to other components such as memory and you'll see that they're intending 2TB to be the single drive limit.   the upper ceiling on a RAID array of any size may also be 2TB, but if you can create a 2TB RAID array then you can have a single 2TB disk as well.   the controller shouldn't discriminate between RAID and JBOD size provided it doesn't exceed the maximum size supported.

[Norway Pad]

Ok, a quick summary of facts is that there are two 160GB SATA drives installed now, on which I installed and run Server 2012 just fine. What happened when I tested the 2TB drives, is that the ServeRAID Array Manager didn't see them, while the Disk Utility part of ServeRAID did see them, and allowed me to low level format them. After which they still weren't seen by the Array Manager, nor the Windows installer.

So the basic communication between the drives and the disk controller seems to be up and running, and when I have the Array Manager scan for drives, the disk activity LEDs are flashing. As the RAID controller is a add-on card, separate from the disk controller, could it be that this card is the culprit here?

I will try the 2TB drives again tonight, and do a little more testing. Check all settings, one drive at a time, etc. I will keep you guys posted.

[Norway Pad]

The 2TB drives were tested again, separately and together, but no change. I also checked all the settings I found, but found nothing that's apparently wrong.

On POST I get this output when the RAID controller starts. It recognizes the drives, but it doesn't see them as logical drives: (On this boot I only had one drive inserted)

Code: Select all

Adaptec RAID BIOS V5.2-0 [Build 11835]
(c) 1998-2006 Adaptec, Inc. All Rights Reserved

<<< Press <Ctrl><A> for IBM ServeRAID Configuration Utility! >>>

Booting the Controller Kernel.....Controller started

Controller #00: IBM ServeRAID 8k-l at PCI Bus:02, Dev:00, Func:00
Waiting for Controller to Start....Controller Started
Controller monitor V5.2-0[11835], Controller kernel v5.2-0[11835]
Controller POST operation successful
Controller Memory Size: 32 MB
Controller Serial Number: 310001
Controller WWN: 5005076B01542340

Physical Devices Listing:
-------------------------
Dev# Box# Slot# Vendor   Product Info    Rev#    Speed    Size   PFA Error
---------------------------------------------------------------------------
 0    0     0    WDC     WD20EARS-00MVMB51.0AB51 3.0Gb/s   1.81 TB  N
 1    0    --    IBM     SAS SES-2 DEVIC1.10     3.0Gb/s            N

No Logical Drives Found

No Int 13 Drives to support

The only thing I see as an option, if I chose to pursue this further, is to attempt a BIOS update. My unit has the original BIOS version 1.05 from 04/04/07, while on this site: http://www-947.ibm.com/support/entry/po ... 78#5051463 I see BIOS version 1.17. I assume I do the DOS one, and not the Windows one. The DOS one describes being run from a floppy. The server has no floppy, but I think it supports booting from USB, so a USB floppy is maybe an option.

I also found an update for the ServeRAID 8K, http://www-947.ibm.com/support/entry/po ... nd=5000008, but I don't see anything in the changelog that indicates any changes with regards to drive size.

I will spend a little while considering if messing with this is worth the risk. The BIOS is probably ok to do, but any hints to a proper procedure here (Floppy / CD / from Windows) would be appreciated.

[bit_twiddler]

I would do the BIOS last, unless you see something in the changelog which
would indicate that it would help you solve your current problem.

There may be an explicit step that you have to take to map
a physical drive into a logical drive. For example, if you were
going to make a RAID array, you would have to map 2 or more
drives into an array, which would become 1 "logical" drive.

You may have to do the same thing, even if you want to map
one physical drive into a logical drive. So, you may be missing
a step here.

Does your Array Manager run from the BIOS, or is it something that
you get into after booting up Windows?

[Norway Pad]

The ServeRAID Array Manager runs from the POST / pre-boot environment, not from within an OS. It's invoked by pressing Ctrl+A, as indicated on the screen output I quoted above. Remember, I don't even have any OS installed on the 2TB drives.

Today I decided that the last verification I wanted to do was to verify that other, smaller sized drives would work, and not just the IBM FRU'ed drives it came with. Unfortunately I have no spare 3.5" SATA drives, but I found two Seagate 320GB 2.5" drives. So I placed these on pieces of cardboard, managed to slide them in and position them in the bays, and got them plugged in. When I powered the server on, the Array Manager recognized the drives immediately, so I initialized them and set them up in RAID 1. To remove any doubt, I also ran the Windows installer until the select disk window appeared, and the created RAID array was recognized as a valid single disk to install Windows on. So I am more and more certain that I'm not missing anything with the 2TB drives, but that the controller simply can't handle the size. The x3550 units that are months or years newer than mine might do, but this one doesn't.

After pondering for a while I decided that I will just leave this here without messing with the BIOS and the firmware, and just go with 1TB drives. A small NAS box is a better choice for a home file sharing device, which will also allow us to use 4-6TB drives. So if the IBM is used for Hyper-V training alone, 1TB capacity will be sufficient. To verify that 1TB drives will work, we decided to let our son buy a 1TB drive for his desktop, and have me borrow it first to verify it in the server. If it works, I will buy two 1TB WD Red drives.

But thank you for the input the two of you have given so far, even if this didn't work out for me. Much appreciated!

[bit_twiddler]

This could be a solution, your model is listed.

http://www-947.ibm.com/support/entry/po ... GR-5083942

It appears to require a reflash of the ServeRAID on your motherboard or controller card.

Personally, I would just download the file to keep it around in case you decide
that you want to go that route.

I follow the following rules when flashing anything on a motherboard (although
this could actually be a separate card).

1. I don't flash anything on hardware that I can't write off. But if, for example, you
found another one of these servers next to the dumpster during your lunch
hour, that would qualify. Or, you could just use the "five finger discount."
Heck, accounting probably wouldn't even notice their entire accounts
receivable database disappearing until the first of the month.

2. Hook up a UPS with enough run time for your system to cover the entire procedure.
(not sure what that is in your case, but > 30 minutes would probably be enough).

3. Change the CMOS button battery for a new one, and test.

4. Pick up another motherboard if you can, because once you rescue another one
of these from the dumpster, you will have another treasured object that you won't
want to lose.

5. Order pizza before you begin the procedure, and be sure to watch this video:
https://www.youtube.com/watch?v=y4UhOeYYXAA

[Norway Pad]

Kudos to you for finding this. I did a lot of searching, but probably used the wrong search terms.

I would think this is the explanation right here, which confirms my theory that the newer x3550s with the new firmware do support 2TB, while older ones, like mine, don't. I am not sure if I will do this update, as the potential benefit doesn't justify the risk of bricking it. There really aren't too many of these in any dumpster, so my option would be a mobo from eBay. The RAID controller is in fact a separate card, wile the BIOS isn't. But I grabbed the files anyway, as you suggested. I grabbed both the Windows and DOS ones, as I see the DOS route as the preferred one if I can get it to boot from a USB floppy.

Your advice on the UPS, is that in case of a power outage?

And for the movie, I guess we figured out why the "never under 3000 rpm in a forward gear" didn't work out. A Ferrari for NY's rush traffic perhaps wasn't the best choice.

[axur-delmeria]

Since the RAID controller is a separate card, the firmware is in a flash chip on that card and not on the motherboard.
If you manage to brick it, then only the card needs to be replaced-- though I admit I don't have any first-hand experience with flashing I/O cards.

Your advice on the UPS, is that in case of a power outage?

Yes. A sudden power outage is one of the worst things that can happen while flashing a BIOS.

[bit_twiddler]

Fortunately, there are a lot of dumpsters out there.

If you search for

ibm x3550 7978 motherboard

on ebay,
you'll find a MB with a buy-it-now price of $26.59 or best offer, so the only real
cost would be shipping to Norway.

The listing expires in a few hours, you may have to wait until it is relisted
to see it.

The listing has a part number, do you have a HMM for your server?

[Norway Pad]

I noticed that one too, it was cheapest one I could find. Shipping and customs adds another $47 to it, but there really is no used computer stuff like this to find here in Norway. So shipping/customs is just an additional fee I have to deal with no matter what I'm buying. I have the HMM, but didn't match the parts number, so I'm not sure if it's an exact match.

All in all it probably wouldn't be too big of a disaster if I bricked it. But I still don't think the hassle of figuring out how the update is done and the potential risk of bricking is worth the benefit of being able to use 2TB drives instead of 1TB ones. After sitting down and asking myself why I really needed the additional 1TB of storage, I realized I have no good reasons. As mentioned above, even 2TB isn't sufficient for a full scale file storage for us. Instead I see that the RAM might be the real bottleneck for my training purpose, so the $100 I save on buying 1TB drives instead of 2TB ones will probably be used for upgrading the amount of RAM.

[bit_twiddler]

If you wind up like me you'll have 2-3 workstations/servers and a bunch of laptops.
In my case, it's for my professional uses, but, heck, hardware is cheap and
getting cheaper every day.

In Heaven, hardware gets cheaper every year, whereas in Hell, it's the opposite.
(Unless you happen to live in Hell, Norway, in which case the cost of hardware
probably stays flat...)