More comprehensive guide to HDD cloning

[R.G.]

I'm writing this not because I HAVE a more comprehensive guide. But having spent two fullish days reading this forum, the IBM/Lenovo information, and several Holy Books looking for The Way.

I am not a novice. I've been using PCs and Thinkpads since before there were PCs and Thinkpads, just never ran into needing to do this.

With as many questions on this as are on this forum, and others, I think a better guide to HDD cloning than exists in the FAQ is needed. The number of variables is much larger than the "How to clone a HDD" FAQ lists.

1. Tools
This is covered OK-ish. I've used Ghost for years, but not recently. I've downloaded Acronis. I have not used the other tools, but I'm sure I could if I wanted to spend the afternoon per tool learning them. I have made images successfully with both Ghost and Acronis TI-11. Neither one will clone the hidden desktop area successfully onto my new disk, in my setup, at least yet.

Oddly, both produce bootable C: partitions which work first time. Just no HPA.

2. Cloning setup
I have what I consider the first step the novice would use to clone - a new HDD on a USB port in an external enclosure. There are no hardware issues with the new HDD on the new drive on USB. It formats, partitions, etc. just fine and is seen and manipulated by disk management. It even clones the C: partition perfectly with both Ghost and Acronis.

So far I have not been able to recreate the predesktop area with any combination of variables with a cloning program.

Then there is the issue of cloning internal to external (which is the obvious way for a cloning novice) and cloning external to internal, which is said to be magic. The magic method does not transfer a working predesktop area. Yes, I have done all flavors of security settings on the HPA to normal, disabled, and secure. All produce the exact same result - no transferred HPA. Or at least none that is recognized.

3. Partitioning
What pre-partitioning is needed, if any? I have tried it several ways, including no partitioning at all, prepartitioning into pre- and main, changing partitions on cloning, etc. And what formatting is needed per partition? Does the pre-desktop area need to be formatted FAT32? Can it be NTFS? Does it need formatted before cloning at all?
4. Recovery, FWRESTOR, etc.
If the predesktop HPA area cannot be cloned by a cloning, must it be rebuilt with recovery CDs?
If it is done with recovery CDs, can it be done on a disk with an already-cloned C: partition? Or must it be done on a fresh, (formatted? partitioned?) disk drive?

I realize that it is possible that there is one single missing bit of info that is located on one of the hundreds of posts on "HELPHELPHOWDOICLONEMYDISK?" posts here, but I seem to have missed it. And I suspect that I'm not the only one, given the existance of those posts.

I will happily volunteer my time collating the information and presenting it for review by experts with more experience than I have in this particular niche. But I think a better guide to this very common problem is needed.

Or, you can just tell me I'm too dense and HERE is the answer, I guess.

[R.G.]

One bit of info I've come up with - making the "hidden partition" visible makes it visible, all right - as unallocated space. I believe this is why the cloning programs do not copy it. Making it invisible does exactly that - make it invisible for all purposes.

[GomJabbar]

Whether you can clone the Predesktop Area or HPA depends upon a couple of variables.

I cloned the hard drive in my T42 successfully, and it would boot both Windows XP and "Rescue and Recovery". I used the EZ-GIG cloning software that comes with the Ultrabay Slim 2nd hard drive adapter. This software comes on a floppy disk, so you need a USB external floppy drive that the ThinkPad will boot from - such as the one sold by IBM (which I have). My T42 came with the Predesktop Area. I left the security setting on Normal when I did my cloning. See the last paragraph in the following post of mine.
http://forum.thinkpads.com/viewtopic.ph ... 195#169195

Earlier ThinkPads, (including some T42's I believe) came with HPA instead. To restore the HPA you need the software tools FWBACKUP and FWRESTORE. In addition, I believe you need a hard drive that supports this technology. It may be hard to find a modern hard drive that supports this (just speculating).

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

http://www.thinkwiki.org/wiki/Talk:Hidd ... ected_Area

[R.G.]

OK, latest theory.

HDD sector copier should do it, as long as I can force it to preserve the partitioning I want to wind up with. This is only a problem where the service area shows up as "unallocated" as it always does.

So in Windows, I make three partitions, one the size of the old system boot partition, one the size of the "unallocated" area that holds the HPA, and another that is the rest of it. I format the "rest of it" partition, then scrap the other two partitions. This leaves me a HDD with just the right size unallocated space and with the sectors unallocated in the right places on the disk, based on simple allocation order.

It's formatting now.

[GomJabbar]

If you have the Product Recovery Discs (CD's) you can restore the factory contents with those, and that will restore the Recovery Partition as well as Windows and all the factory software. You can then clone your existing drive to the Windows partition and still have Rescue and Recovery.

AFAIK, if you have the HPA of the 2004 era ThinkPads, you can only restore the HPA to a drive that supports that technology and only by using FWBACKUP and FWRESTORE.

[carbon_unit]

EZ-GIG is an OEM modified Acronis product. It is very similat to Acronis True Image just like the latest Maxtor Maxblast and Seagate Seatools. They all do the same thing.
When cloning the predesktop area are you un-hiding it in the BIOS?
The factory cd's will install the predesktop area and install windows for you. All properly partitioned. Then you could clone your old windows install over if you want but I always find a new install preferable to an old one. It just runs better.

[R.G.]

Thanks for dropping in and helping.

If you have the Product Recovery Discs (CD's) you can restore the factory contents with those, and that will restore the Recovery Partition as well as Windows and all the factory software. You can then clone your existing drive to the Windows partition and still have Rescue and Recovery.

Yep. I know that. This is an R40e, and needless to say, I don't have the product recovery disks. So, one bit of info that needs to go into the "improved hdd cloning procedure" is whether or not, yes or no, this can be done without recovery discs. In fact, I suspect that there are two paths that need to be documented - either (a) you have product recovery discs and then do thus and such, or (b) you do not, and must then abase yourself by doing this following incantation...

But it oughta be clear somewhere. Right now it is not, so far as I can find on this forum, Lenovos, and other places.

AFAIK, if you have the HPA of the 2004 era ThinkPads, you can only restore the HPA to a drive that supports that technology and only by using FWBACKUP and FWRESTORE.

The data that needs written down are then
- for Thinpad models X and Y, it is not possible to do this except with HDDs supporting esto y esta, and then only with FWBACKUP and FWRESTORE
- for other Thinkpad models L, M, and N, you may restore by this other process.

I have actually been copying down fragments of this and other pages in the attempt to do that but I'm not familiar enough with the process, not having gotten it to work even once, to advise people.

[purely as an aside, what I'm working on is an R40e; but I'd much rather solve the general problem and help someone else out too.]

EZ-GIG is an OEM modified Acronis product. It is very similat to Acronis True Image just like the latest Maxtor Maxblast and Seagate Seatools. They all do the same thing.

A couple of places I read that EZ-GIG can see the HPA as a partition instead of unallocated, which seems to be what is gumming up most of the other stuff. However, I have not yet been able to download and make EZgig work; it seems to be a for-fee thing and I haven't yet gotten to solving the problem by rubbing $100 bills on the thinkpad covers yet..

When cloning the predesktop area are you un-hiding it in the BIOS?

Yes. And no. And also trying "secure", all possible combinations including the recommended "disabled". I can un-hide it just fine. That makes the disk appear larger by the amount of the "unallocated" space, which amounts to the HPA. In addition, the HPA software is unreachable when the HPA is unhidden, works again when it's back to "normal". Both say, yes, there is a functioning HPA there, but it is not reachable by copying.

I've used Ghost, Acronis TI-11, EaseUS Disk Copy 2, a few minor segment copiers. While it is possible that they do the copy, and the TI-11 results in a booting windows system, there is no way to get to the functions of the HPA, indicating to me that it is either not there or not connected up properly.

A fair number of conflicting stories appear in places on the net, including fervent testimonials that "just put it in place and clone it works", "You have to use "AS-IS"", "You can manually assign sizes", "Program X will copy up to as big as the start disk and quit", and on and on and on. I think in all cases that the testator got lucky and wrote down what they thought they did, because I could not recreate any of them, including the normal fifteen or so reboots while dinking with the parameters.

The factory cd's will install the predesktop area and install windows for you. All properly partitioned. Then you could clone your old windows install over if you want but I always find a new install preferable to an old one. It just runs better.

Yep, I like fresh installs too. But like I said, I don't have product disks. Further, the consensus here over several hundred posts is evenly divided between "I can't make it work." and "No problem, just do X and you're upgraded."

So I'd kinda like to find out - (a) is it really possible (probably; lots of smoke around) and (b) by what path or paths, and how does one traverse that maze?.

I can tell you for sure that the procedure in the FAQ on cloning HDDs, while it results in a booting windows system with Acronis, does NOT produce a working HPA when followed not only to the letter, but followed in ways that represent several plausible misunderstandings of the letter. So at the very least it is above my head; and I think at worst it's inadequate or misleading [there, that's my frustration talking].

I'm willing to put in some time writing the right stuff down. I just have to find it. And I appreciate any of the right stuff you can provide for me.

[GomJabbar]

I had another link (from IBM) that is no longer available (AFAIK), that explained the differences between the differerent Predesktop Areas used by IBM. There is a partial quote in the following post.
http://forum.thinkpads.com/viewtopic.ph ... 712#161712

Here is something that explains the HPA and how to clone your drive.
Predesktop Area white paper - Access IBM

The following thread does not cover cloning per se, but it is enlightening.
Format HDD with IBM Hidden Partition

HTH

[R.G.]

Thank you. Yes, it does help.

I have some writing to do, even if it's only a skeleton for more knowing people to hang flesh onto.

[RealBlackStuff]

What I don't understand is your obsession with this HPA/PDA.
You need your HD working with the OS that is on it.
Image that when it is fresh and to your liking.
If things go south, restore your image.
It's way faster and you save the (wasted) HPA space.

It is very commendable to want to create THE manual, but ask around how many people are really using/requiring this HPA/PDA?

[R.G.]

What I don't understand is your obsession with this HPA/PDA.

It's raging intellectual curiousity.

The attitude which says: "I don't care HOW it works! It works, get on with it!" is perfectly valid. And I respect it. However, in this case the problem managed to trip over the threshold inside me that made me want to know.

As I noted, I've already gotten a bootable HDD, and if that was all there was to it, I'd quit. But I want to know.

Thanks for the advice, but just getting it to run is not what I'm after.

It's much like the feeling that "I'm going to finish that $#&*(#$ crossword puzzle!", and it could even save someone else some time trying to do things that aren't possible.

You don't need it, but maybe someone else will.

[R.G.]

OK, here's where I am so far.
=====================================

A More Comprehensive Guide To Cloning Thinkpad Hard Drives

Cloning Thinkpad hard drives is difficult because in doing so, you are put into the position of the blind men feeling of an elephant. There are many parts to a large issue, and even if you have successfully “felt” of your portion, your knowledge is not complete enough to advise the next guy who has the problem.

As of September 2008 are the following issues to consider in cloning your Thinkpad HDD:
1. Pre-HPA, pre-Rescue-and-Recovery (R&R) styles using a factory recovery disk.
2. HPA style recovery
3. Rescue and Recovery (R&R) recovery
4. R&R 4.0 recovery
5. PATA/IDE and SATA disks and controllers

CUT TO THE CHASE:
==============
For Pre-HPA, disk cloner programs work in a straightforward manner.

For HPA machines, you must (a) have a HDD which participates in the BEER/PARTIES architecture AND either (b) use the Factory Product Recovery Disks, OR (c) manually relocate the HPA after cloning. Otherwise, your only option is to (d) decide you can get rid of the HPA and do some other means of recovery after a failure. There are BIOS manipulations that will be necessary for all of these.

For R&R machines, you can use a drive cloning program. For a very few R&R machines using a SATA-to-PATA bridge (see table) you must also have a HDD which can be flashed to eliminate the “Error 2010” problem.

BACKUP DATA/REFERENCE
====================

Thinkpad models as regards HDD cloning:

(long table of thinkpad models and which has what)

Retrieved from "http://www.thinkwiki.org/wiki/ThinkPad_History"
a. http://www.thinkwiki.org/wiki/Rescue_and_Recovery
b. http://www.thinkwiki.org/wiki/Hidden_Protected_Area
c. http://www.thinkwiki.org/wiki/Problem_w ... hard_disks

More info on HDD disaster recovery for Thinkpads.
===================================

The earliest recovery system was one or more CDs which could restore the HDD content to the factory-shipped condition. For these systems, simple disk cloning programs work.
The earliest R&R versions (2.n?) were used on models T23 and T30. This used the R&R programs, but in a visible disk partition. Simple disk cloning works.

HPA is hidden from hardware (!) by BIOS commands to the HDD firmware, which then reports a smaller hard disk size by listing fewer cylinders. The access to the HPA is controlled by the BIOS setting of the “Pre Desktop Area” in Security settings. When the BIOS is set to “Disabled”, the extra cylinders are visible to other programs for copying.

However, this is a complex issue. First you must have a HDD that supports the BEER and PARTIES drive access architecture. Without that, the disk itself will not play well with the BIOS calls to the disk to change its hardware visible size.

The layout of data on the HDD has the Windows/boot partition first on the disk. The HPA is not at the front of the disk, but at the last few cylinders. When the BIOS calls make this area visible, it appears as unallocated space to any partition program I’ve yet found. So partition-copiers, even bit-for-bit partition copiers, will refuse to do anything with it because it’s apparently not in a partition.

Because of this, if you bit-clone, sector-clone, etc., the HDD, you get the Windows partition first, the HPA in the cylinders at the end of the Windows partition, and nothing out at the end of the HDD. Even if you have a BEER/PARTIES compatible disk, the end of the disk where the changeable size exists is not written with the necessary data for the HPA to function correction in the Thinkpad.

I can hypothesize that one could low level format a disk, then fill all sectors with some marker byte; following that, one could exact-clone the resumably-smaller space from a previous drive. The final fillip is to then use a hex disk examiner to find the last-written sector of the Windows partition and then use a low level sector reader/writer to copy an image of the HPA data into the HPA area at the end of the HDD. Then you could adjust the Windows partition size, or better yet make the really-unallocated area in the middle be your data area, separating your fungible data from the Windows partition. Yes, that’s a lot of work, and uses a workflow that doesn’t exist yet.

You could also simply format the new HDD, then use the Product Recovery Disks from IBM/Lenovo for your model to place the correct Windows partition/HPA on the disk. I do no know whether this will actually work or not; it could be that the programming on the PRDs doesn’t put the HPA at the end of the disk, but only somewhere where it thinks they should go. In any case, the PRDs appear to be the only non-guru practical way to clone an HPA-era HDD and get the HPA in the right place on the HDD.

Note that the recovery disks which the HPA programs create if you choose “make recovery disks” on your Thinkpad with HPA will NOT be able to create a new HDD image on a larger disk which will work properly. Only the factory Product Recovery Disks will work for this.

Your only other option is to just bag trying to get the HPA copied over and use some other backup and recovery scheme, forever tossing the HPA recovery stuff out.

R&R uses a hidden partition, but it does not use the hardware-hidden techniques of the HPA. This gets very confusing because IBM literature uses “Pre-Desktop Area” as the single name for both HPA and non-HPA versions. HPA exists only for certain specific models.

Early R&R, versions 3.n, uses a FAT32 file system partition; later (R&R 4.n) versions reported on Z61 use NTFS partitions. Disk cloning programs will successfully copy these disks.

The caveats mentioned in other places about not booting your system with both the original HDD and the clone on it to avoid Windows labeling the clone partition with a drive letter apply.

Another issue to be careful of is PATA versus SATA. Early Thinkpads were PATA (IDE or ATA). Later ones have gone to SATA. However, in the middle, there were Thinkpads which used PATA HDDs, but had a SATA controller on the motherboard, which ran the PATA HDD by means of a SATA-to-PATA bridge chip. For these models only, the HDD must have the correct firmware inside the HDD, or you will get the “Error 2010” on trying to boot a clone. You must either use an IBM/Lenovo HDD or flash update the OEM HDD you try to put into these machines to avoid this and get a bootable drive.

[carbon_unit]

Maybe that is part of the reason they moved away from the HPA?

[R.G.]

Maybe that is part of the reason they moved away from the HPA?

Maybe. The drive certainly has to support this little song and dance of hiding its own storage. Maybe not enough drive makers supported the funny interaction, or maybe SATA just didn't.

It's also possible they just wandered away, not remembering what was so all-fired neat about it. IBM is an odd social setup. People who do well and stamp their thumbprint on things get promoted out of usefuness and the next person in puts their different thumbprint on the next one. It's also a social structure that makes Klingon society look like a gracious formal ball.

It was long ago apparent to IBM insiders that it was easier to get ahead by assasinating the guy above you and to one side than it was to just turn out exceptionally good products. There are remarkably smart, motivated people there - using their powers for good, evil, and hybrids.

[josh999]

If you DO have the recovery disks, and use them first, and then you restore the "C" Drive with an Acronis backup, I believe there will still be one issue:

You likely have an Updated Version of R&R on "C" (than the one on the Recovery Dosks), which will now be a "mismatch" between the version on the Hidden Partition and whatever drivers/files for R&R are on the "C" Drive.

Can you then just Update R&R, or will it cause issues (or will the update have problems running/refuse to run altogether), because an older version is on the Hidden Partition, and a newer (and probably the same Version you could run an update with) is on the "C" Drive?

[R.G.]

Thank you Josh - this is exactly the kind of considerations I was hoping to flush out.

Before I started doing this research here and elsewhere, I knew exactly zero about Thinkpad disks. The writeup is where I got in three days, so the real answer to your question is the hard thing for me to say... I don't know.

I need to be clear about the question before I could even hazard a guess. First, Pre Desktop Areas contained in the hardware-hidden HPA may not use the "Repair and Recovery" software. I don't remember that from any of the reading I did. I do remember the PDA/HPA versions using the HPA for hidden backups of ... stuff(?) from C:. So restoring to factory original and then copying over C: would necessarily lose the stuff. It is not clear to me if the PDA/HPA version had its programs updated.

The R&R versions using the non-hardware-hidden recovery partition absolutely have had multiple versions of R&R software released. But these versions will copy fine with a cloning program (I think... the reading here and elsewhere supports that) and so do not need factory product recovery disks to recover. In this variation, yes, you are correct that the factory recovery disks will back-level your R&R partition as well as putting a clean C: partition there. If you then clone C: partition across, any expectations which happen to be hidden in the C: partition on the level of R&R will be mis matched.

However, it is not clear to me that any expectations/information about the level of R&R are stored in the C: partition. In the R&R version, the recovery partition is bootable for recovery itself.

I ... think that the right way to handle non-HPA R&R is to clone the disk. If you unnecessarily use the Factory Recovery Disks to rewrite new R&R partition and OS partition, then clone over your old OS partition, you are open to mismatch issues. And so my approach would be what my doctor recommended - don't do that, you'll hurt yourself.

But as I've noted before, I would like someone who knows to fill in the gaps here. I think I've produced some good elaboration on the disk cloning issues, but it's by no means complete - just more comprehensive than "just clone it".

[GomJabbar]

Oh, you want to know about R&R!

Troubleshooting - Rescue and Recovery

Symptom

Any machine that has IBM Rescue and Recovery with Rapid Restore Ultra 1.0 or IBM Rescue and Recovery 2.0 factory preloaded, then installs Rescue and Recovery 3.0, will lose the ability to have Rescue and Recovery installed on the machine after a Restore to Factory contents action is selected when booting from the computer's hard drive.

Rescue and Recovery 3.0 updates the contents of the recovery environment stored on the computer's hard drive. Restoring to Factory contents does not alter the recovery environment and could result in an inconsistent configuration. Therefore, the ability for the factory preload to install IBM Rescue and Recovery with Rapid Restore Ultra 1.0 or IBM Rescue and Recovery 2.0 after Rescue and Recovery 3.0 has been installed was purposely disabled.

Solution

It is recommended that you create Recovery CDs prior to installing any software. If this is done, Restoring to Factory contents using the Recovery CDs will restore both the recovery environment as well as the preloaded version of Rescue and Recovery into Windows. If the Recovery CDs were not created before installing Rescue and Recovery 3.0, before doing a Restore to Factory Contents you may wish to copy the Rescue and Recovery 3.0 installation files off to external media so you will not have to download from the Web again.
CMVC Defect Number: 282239

And don't forget Client Security Solution!

Troubleshooting - Client Security Solution

[GomJabbar]

Oh, I just found a link to the real gem I wanted to show you. The link below downloads the pdf file: IBM Rescue and Recovery Deployment GuideVersion 2.0.

ftp://ftp.software.ibm.com/pc/pccbbs/th ... r20mst.pdf

[dalmolin]

I am delighted that someone has volunteered to write an understandable guide to cloning etc. Thinkpad hard disks. I recently bought a 250 Mb drive to upgrade my 80 Gb drive that came with the T42p I have (it's a great computer!!). I also want to dual boot XP/Linux.

My first goal was to clone the existing disk but give the C: drive about 100Gb to work with and second goal to use the rest for Linux and the backup pre-desk
top. I am stuck on first the first goal. I have done what I am proposing to do many times over the past 10 years .... that's when I started using Linux too. But I am totally stumped as to how to do this so far.

I have tried Acronis....but did not disable the pre-desktop in the bios and all it did was hang after 30%. I have looked into other tools, there is a tool called Fiesta which is open source, and some claim to have success with it.

Don't know if others have tried this from the Lenovo site, I find it ambiguous:

If you need to Ghost an entire hard drive, including a hidden partition, use the following method:

1. Enter BIOS, go to Security, and set the predesktop area to "disabled" to make the hidden partition visible on the system with the source drive.
2. Run Symantec Ghost with the -IB switch when you create the image (example: C:\> GHOST -IB)
3. After the image is created, make the predesktop area visible on the target system by disabling it.
4. Restore the image created in step 2. No special switches are needed for this step (for NetVista only).
5. Reset the predesktop area to "normal", the system may be booted normally to the operating system or to the predesktop area by pressing the Access IBM key during POST.

[R.G.]

Thanks, guys! I'll read and learn.

The "deploying R&R" is particularly interesting because it clearly (as clearly as IBM does anything) defines what the disk structures are. They match mostly with my concepts from earlier reading.

I recently bought a 250 Mb drive to upgrade my 80 Gb drive that came with the T42p I have. I also want to dual boot XP/Linux.

The T42p is listed in the table of models I didn't put in the posting. Unfortunately, its a bit ambiguous, as it was listed with both R&R and an HPA. I think from my first scan that it does have an HPA. As nearly as I can tell, you can clone the OS partition, but you can not clone the HPA and have it emerge usable without using Factory Recovery Disks on the newly formatted replacement disk. If I"m right about that, the simplest thing to do is to clone the OS partition and count on never using R&R, but instead keeping recovery by some other means, like disk images, or partitioning into OS and data partitions and backing up the data partition. I'm in the same boat, so I will have dug to the bottom of this soon.

I have tried Acronis....but did not disable the pre-desktop in the bios and all it did was hang after 30%.

I successfully cloned my 20G drive onto a 60G drive with the Acronis TI-11 trial, producing a bootable disk, but without HPA. This must be done from within Windows for the trial, and I cloned directly to the new drive in a USB enclosure.

I have looked into other tools, there is a tool called Fiesta which is open source, and some claim to have success with it.

I have not tied Fiesta yet. I have not branched into Linux, despite years of using AIX.

If you need to Ghost an entire hard drive, including a hidden partition, use the following method:

I believe this means on a 1:1 size basis, not for enlarging the drive.

1. Enter BIOS, go to Security, and set the predesktop area to "disabled" to make the hidden partition visible on the system with the source drive.

As my run-down explains, this is what tells the BIOS to tell the hard file to let software see the extra cylinders. If you don't do this, the HPA is invisible and can't be copied by anything.

2. Run Symantec Ghost with the -IB switch when you create the image (example: C:\> GHOST -IB)

This tells Ghost to copy even sectors that are not in formal partitions; as I found out, HPA areas are not in discernable partitions even when visible.

3. After the image is created, make the predesktop area visible on the target system by disabling it.

That is, tell the BIOS on the target/destination system which will receive the image to un-hardware-hide its final few cylinders so the image can be written into them. Again, I think the idea was to enable you to put the image into a same-size hard file.

4. Restore the image created in step 2. No special switches are needed for this step (for NetVista only).

No issues. Ghost writes out its image sector by sector to the new disk.

Note that this process implies that you are removing the image from somewhere else and putting it onto the hard drive in the machine where it will be used. This fits with the idea that the machine hardware may access HDD geometry in a peculiar manner, and that the target machine BIOS is managing putting the HPA into place.

5. Reset the predesktop area to "normal", the system may be booted normally to the operating system or to the predesktop area by pressing the Access IBM key during POST.

And once you have it there, the BIOS can manage it like usual, so you need to change the BIOS setting back to "normal" to hardware hide the HPA.

Again I think this only works as stated on a same-size hard file. Upverting never seems to have crossed their minds.

[Ken Fox]

I'm only using EZ GigII these days (a version of Acronis that I got free with a USB to IDE/SATA adapter), for both backup and for cloning.

I went through a stretch when I was having very bad luck at cloning disks, with a success rate of maybe 30% at best. I have since modified my approach and my success rate is almost 100% lately.

Here's what I'm doing and it won't be possible for everyone to do this, but if you have the proper equipment it is easy to do. First of all, I never clone onto a disk that has something on it (e.g. files or partitions or prior installations of operating systems). In order to get a completely "blank" disk, you either need to have just purchased one and done nothing with it, or you need to delete the partitions that are on it already. The easiest way to do this is to get a bootable linux utility such as Qparted (I assume there are quite a few of these and they are free to download). With the disk to be cloned onto (e.g. the recipient) in the machine in the regular hard disk bay, boot up the CD you have made with the linux utility either in the ultrabay optical drive or in a freestanding USB optical drive, and after the program boots up delete all the partitions on the recipient disk so it is unpartitioned, unformatted, and completely blank. One other advantage of doing this is that it completely eliminates the possibility of cloning in the WRONG direction, something you might do if both the source and recipient disks have operating systems installed on them.

Only after doing this, proceed with the cloning. You have to be careful with Acronis to select cloning the entire source disk, choosing to clone the MBR and all partitions (which includes the service partition if you want it). I always use the "custom" option, and generally choose either "As Is" or "Custom" for partition sizes depending on whether I am going to a smaller disk, a same sized disk, or a larger disk. I do not want to shrink or expand the service partition, e.g. I want it the exact same size on the new disk regardless of the size comparison of the old to the new disk. Sometimes you will be able to use "AS Is" which is simpler, but sometimes you will be forced to use custom sizing, but I never use proportional as I don't want to either shrink the free space in the service partition nor to increase it. The main NTFS partition for Windows is what will change in this case, which is what you would generally want.

I prefer to clone from the source disk in an ultrabay (either in a machine with an ultrabay such as a T6x, or in an Ultrabase as for a tablet or X machine). Obviously you will need to buy hardware in order to do this if you don't have it so in that case you may need to make do with a USB box for the source disk.

Doing this as I am now doing this, onto a totally blank recipient (new) disk, I'm having hugely greater success than I used to have when I simply tried to clone over what was already on the disk at the time of the cloning. An added benefit is that I never worry about accidentally cloning in the wrong direction, which makes the cloning process nearly idiot proof.

[R.G.]

Ken,
That's good info.

I've had 100% success (that is, all four times ) cloning the source disk from the working drive slot to a target in a USB enclosure with Acronis TI-11 trial. It asks if it can wipe the target clean and I tell it yes.

I'm guessing that you're using a model of machine without the HPA (that is, the service partition is visible to normal disk utilities) since you don't mention messing with the BIOS switches for the Pre Desktop Area. In that case, yes, I'd think that a straightforward clone would work fine, even with the service area, which is just another partition in that case.

Good advice on making the target be blank - it sidesteps the issue of which disk is which neatly, like a matador performing a Veronica. And there are others who prefer to clone from an external disk into the main bay, as you suggest; that may help in funny-geometry cases.

What model/types have you done this on?

[dalmolin]

Interesting that you are using EZ Gig.... just discovered that Lenovo is selling a disk upgrade kit from Apricorn, which includes EZ Gig for both SATA and IDE drives.

A couple of questions:

1. Do you place the target drive in the normal drive bay and move the source to an external location to prevent the hidden directory from getting messed up somehow when you load the cloning software?

2. Do you need to "disable" the pre-desktop in the bios before removing the source drive?

[GomJabbar]

IIRC, some of the T42's have the HPA and some do not. The T42's I have owned (3) do not have the HPA, but have the Type 12 protected partition. I believe it was the early T42's that had the HPA (if memory serves).

The clones I made in my T42 were from the main hard drive bay (source) to the Ultraslim Bay (target). In my case, I left the Predesktop Area security set to Normal when I made the clones, and I did end up with a bootable Rescue and Recovery partition.

I decided never to upgrade Rescue and Recovery (2) to avoid the problem I quoted above in a earlier post.

Here is another tidbit for you from forum member Paul Pavlik.
Cloning T60 With Apricorn (Update)

EDIT: One other thing. For anyone reading this, be sure and remove the 2nd hard drive immediately after cloning. Booting up Windows when both hard drives having a copy of Windows (and are in the machine) can disable the second copy of Windows.

[R.G.]

IIRC, some of the T42's have the HPA and some do not.

Could you possibly find the reference to that? That's an important point for people - whether there is an HPA or not. That's the boundary.

The clones I made in my T42 were from the main hard drive bay (source) to the Ultraslim Bay (target). In my case, I left the Predesktop Area security set to Normal when I made the clones, and I did end up with a bootable Rescue and Recovery partition.

That makes sense - no HPA, no trouble cloning the recovery partition.

I decided never to upgrade Rescue and Recovery (2) to avoid the problem I quoted above in a earlier post.

Good point. That needs to get posted to a FAQ somewhere. It's not really a part of a HDD cloning guide, but it sure belongs in a recovery FAQ. Apparently if you don't make R&R disks immediately before changing your installation, you can't ever restore your setup completely by making R&R disks later.

EDIT: One other thing. For anyone reading this, be sure and remove the 2nd hard drive immediately after cloning. Booting up Windows when both hard drives having a copy of Windows (and are in the machine) can disable the second copy of Windows.

I nee to make that very prominent. Thanks!

[R.G.]

Here's the next update, copied from Word. I'll convert to HTML soon.
============================================
A More Comprehensive Guide To Cloning Thinkpad Hard Drives

Cloning Thinkpad hard drives is difficult because in doing so, you are put into the position of the blind men feeling of an elephant. There are many parts to a large issue, and even if you have successfully “felt” of your portion, your knowledge is not complete enough to advise the next guy who has the problem.

To do a successful job, you must know what is inside your model/type of Thinkpad. As of September 2008 Thinkpads fall into the following buckets:
1. Pre-HPA machines which shipped with a factory recovery disk.
2. HPA style recovery, which eliminated the factory recovery disk with the machine.
3. Post-HPA machines, which used no factory recovery disks, but had a visible FAT-32 service partition.
4. Post-HPA machines, which use an NTFS service partition
5. PATA/IDE vs SATA disks and controllers
6. The release number of your Rescue and Recovery software.

You must also think about what you consider “Success” in cloning a HDD. The central issue is whether you are satisfied with a HDD that has the OS, programs and data all copied, but forever gets rid of the manufacturer-provided recovery process, or whether you want to stay with the IBM-designed R&R architecture.

Quick Answers: Here’s what to do depending on your category:
1. For Pre-HPA machines, disk cloner programs like Ghost, Acronis True Image and others work in a straightforward manner. There is no hidden partition, and a clone is a clone.
2. For HPA machines, you must decide whether you want to keep the HPA as a rescue means. You must set the Pre-Desktop Area security to “Disabled” in the BIOS for either of these.
a. If you want to keep the R&R abilities, you must:
i. first have a target HDD which participates in the BEER/PARTIES architecture
ii. EITHER clone the entire disk and then manually relocate the HPA to the last HDD cylinders after cloning the OS parition
iii. OR use the Factory Product Recovery Disk set to first set up the target disk with the HPA area, hide the HPA area in BIOS and then use a cloning program to move the OS partition.
b. If you decide you can get along without the IBM recovery assistance, Just ignore the HPA and clone your disk. Only the OS partition will be copied, but that’s OK because you will use some other recovery means.
3. For Post-HPA machines, you can use a drive-cloning program because the service partition is not hidden by hardware.
a. For a very few Post-HPA machines using a SATA-to-PATA bridge (see table) you must also have a HDD which can be flashed to eliminate the “Error 2010” problem.
b. Post-HPA machines come in two flavors, with FAT32 recovery partitions, and later NTFS recovery partitions. Your cloning software must be able to clone the one you have, and in the case of NTFS OS partition and FAT32 recovery partition, not get confused over the two types of partitions in a single clone.

In all of these cases, you must remove either the source HDD or the target HDD from the copying machine before rebooting if you use Windows. Windows “marks” partitions with their letter usage, and this may contaminate your target or source disk and make that disk not bootable.

As a final caveat: this is not properly a HDD cloning issue, but for Rescue and Recovery releases after 2.0, YOU MUST HAVE ALREADY MADE YOUR RECOVERY DISKS BEFORE INSTALLING ADDITIONAL SOFTWARE IN YOUR HDD SO YOU CAN SUCCESSFULLY USE FACTORY PRODUCT RECOVERY DISKS TO RECOVER THE INITIAL STATE. That is, you have to have already made them as nearly the first step you took with your new machine. IBM warned you – in small type on a single leaflet of paper tossed into the bottom of the box, no doubt.

Thinkpad models as regards HDD cloning:
ThinkPad model release year HDD Type
Pre-2003 Models (mostly) Pre-HPA with included Recovery disk
T23 and T30 Pre-HPA, R&Ra 2.0

2003
R40 Jan 2003 HPAb
T40, T40p Mar 2003 HPAb
X31 Mar 2003 HPAb
G40 Apr 2003 HPAb
R40e Oct 2003 HPAb
R50 Oct 2003 HPAb
T41 Oct 2003 HPAb
T41p Nov 2003 HPAb
R50p Nov 2003 HPAb
2004
X40 Feb 2004 HPAb
R50e Apr 2004 HPAb
R51 Apr 2004 HPAb
T42, May 2004 Early ones have HPA, later ones Post HPA
T42p May 2004 HPAb
G41 Oct 2004 ?
2005
T43 Feb 2005 Post HPA, R&Ra 3.0; SATA to PATA, Error 2010c listed as having HPA
R52 Mar 2005 Post HPA, R&Ra 3.0; SATA to PATA, Error 2010clisten under having HPA
T43p Apr 2005 R&Ra 3.0; SATA to PATA, Error 2010c listed as having HPA
X32 Apr 2005 HPAb
X41 Apr 2005 Post HPA, R&Ra 3.0; SATA to PATA, Error 2010c
X41 Tablet June 2005 Post HPA, R&Ra 3.0; SATA to PATA, Error 2010c
Z60m Sep 2005 Post HPA, R&Ra 3.0; SATA
Z60t Sep 2005 Post HPA, R&Ra 3.0; SATA
R51e Sep 2005 R&Ra 3.0
2006
T60 Jan 2006 Post HPA, R&Ra 3.0; SATA
X60 Jan 2006 Post HPA, R&Ra 3.0; SATA
X60s Jan 2006 Post HPA, R&Ra 3.0; SATA
T60p Feb 2006 Post HPA, R&Ra 3.0; SATA
R60 May 2006 Post HPA, R&Ra 3.0; SATA
R60e May 2006 Post HPA, R&Ra 3.0; SATA
Z61e May 2006 Post HPA, R&Ra 3.0; SATA
Z61m May 2006 Post HPA, R&Ra 3.0; SATA
Z61t May 2006 Post HPA, R&Ra 3.0; SATA
Z61p July 2006 Post HPA, R&Ra 3.0; SATA
X60 Tablet Nov 2006 Post HPA, R&Ra 3.0; SATA
T61[1] May 2007 Post HPA, R&Ra 4.0; SATA
R61 Post HPA, R&Ra 4.0; SATA
X61 Post HPA, R&Ra 3.0; SATA
X61s HPAb
X61 tablet HPAb

Retrieved from "http://www.thinkwiki.org/wiki/ThinkPad_History"
a. http://www.thinkwiki.org/wiki/Rescue_and_Recovery
b. http://www.thinkwiki.org/wiki/Hidden_Protected_Area
c. http://www.thinkwiki.org/wiki/Problem_w ... hard_disks
Background for HDD disaster recovery for Thinkpads.
The earliest recovery system was one or more CDs which could restore the HDD content to the factory-shipped condition. For these systems, simple disk cloning programs work.
The earliest R&R versions (2.n?) were used on models T23 and T30. This used the R&R programs, but in a visible disk partition and did not ship with factory provided recovery disks. Simple disk cloning works.
HPA is hidden from hardware (!) by BIOS commands to the HDD firmware, which then reports a smaller hard disk size by listing fewer cylinders. The access to the HPA is controlled by the BIOS setting of the “Pre Desktop Area” in Security settings. When the BIOS is set to “Disabled”, the extra cylinders are visible to other programs for copying.
However, this is a complex issue. First you must have a HDD that supports the BEER and PARTIES drive access architecture. Without that, the disk itself will not play well with the BIOS calls to the disk to change its hardware visible size.
The layout of data on the HDD has the Windows/boot partition first on the disk. The HPA is not at the front of the disk, but at the last few cylinders. When the BIOS calls make this area visible, it appears as unallocated space to any partition program I’ve yet found. So partition-copiers, even bit-for-bit partition copiers, will refuse to do anything with it because it’s apparently not in a partition.
Because of this, if you bit-clone, sector-clone, etc., the HDD, you get the Windows partition first, the HPA in the cylinders at the end of the Windows partition, and nothing out at the end of the HDD. Even if you have a BEER/PARTIES compatible disk, the end of the disk where the changeable size exists is not written with the necessary data for the HPA to function correction in the Thinkpad.
I can hypothesize that one could low level format a disk, then fill all sectors with some marker byte; following that, one could exact-clone the resumably-smaller space from a previous drive. The final fillip is to then use a hex disk examiner to find the last-written sector of the Windows partition and then use a low level sector reader/writer to copy an image of the HPA data into the HPA area at the end of the HDD. Then you could adjust the Windows partition size, or better yet make the really-unallocated area in the middle be your data area, separating your fungible data from the Windows partition. Yes, that’s a lot of work, and uses a workflow that doesn’t exist yet.
You could also simply format the new HDD, then use the Product Recovery Disks from IBM/Lenovo for your model to place the correct Windows partition/HPA on the disk. I do not know whether this will actually work; it could be that the programming on the PRDs doesn’t put the HPA at the end of the disk, but only somewhere where it thinks they should go. In any case, the PRDs appear to be the only non-guru practical way to clone an HPA-era HDD and get the HPA in the right place on the HDD.
Note that the recovery disks which the HPA programs create if you choose “make recovery disks” on your Thinkpad with HPA will NOT be able to create a new HDD image on a larger disk which will work properly. Only the factory Product Recovery Disks will work for this.
Post-HPA R&R uses a hidden partition, but it does not use the hardware-hidden techniques of the HPA. Instead, a special driver hides the recovery partition from the OS. This gets very confusing because IBM literature uses “Pre-Desktop Area” as the single name for both HPA and non-HPA versions. HPA exists only for certain specific models. It has been reported that the T42 had early models with HPA and later models without HPA.
Early R&R, versions 3.n, uses a FAT32 file system partition; later (R&R 4.n) versions reported on Z61 use NTFS partitions. Disk cloning programs will successfully copy these disks.
The caveats mentioned in other places about not booting your system with both the original HDD and the clone on it to avoid Windows labeling the clone partition with a drive letter apply.
Another issue to be careful of is PATA versus SATA. Early Thinkpads were PATA (IDE or ATA). Later ones have gone to SATA. However, in the middle, there were Thinkpads which used PATA HDDs, but had a SATA controller on the motherboard, which ran the PATA HDD by means of a SATA-to-PATA bridge chip. For these models only, the HDD must have the correct firmware inside the HDD, or you will get the “Error 2010” on trying to boot a clone. You must either use an IBM/Lenovo HDD or flash update the OEM HDD you try to put into these machines to avoid this and get a bootable drive.

[Ken Fox]

What model/types have you done this on?

I have never worried one way or the other about the bios settings regarding the service partition (e.g. hidden or not; I don't change them, they are what they are in the bios' default configuration).

Machines I have done this on and do this on fairly regularly:

X60 (two of them)
X60T (two of them)
T60 (5 of them!)
X32 (but not very recently)

I clone and or image the whole disks, not just one partition or another.

[GomJabbar]

IIRC, some of the T42's have the HPA and some do not.

Could you possibly find the reference to that? That's an important point for people - whether there is an HPA or not. That's the boundary.

I don't recall ever reading that except in the forum posts. Here are a couple of such posts. There is some other good information in those threads as well.
http://forum.thinkpads.com/viewtopic.ph ... 212#323212
http://forum.thinkpads.com/viewtopic.ph ... 599#326599

In the Deployment Guide it says:

IBM computers that are announced in the first quarter 2004 and that come with the IBM Rescue and Recovery environment preinstalled will feature this configuration. The Rescue and Recovery environment resides entirely in a type 12 partition, not in the virtual partition as with the previous scenarios. In addition to the Rescue and Recovery environment, the factory recovery and system diagnostics will also reside in the type 12 partition. The Rapid Restore Ultra 4.0 backups, however do not reside in the type 12 partition.

You would have to check tawbook to see exactly when the T42 was initially released. I do not have a copy of tawbook or tabook on the hard drive I am using at the moment.

Here is some more info.
Making recovery cds more than once?

[underclocker]

Wow. That's a lot to read and I'm not sure I caught it all. However, I will say that I've built a lot of T40 & T41 machines and for every one I had to use the FWRESTOR utility. I've never been able to clone one successfully.

But I've made it easier for myself with the following steps;

1) Buy a 4GB (minimum) USB memory stick
2) Make it bootable with any free utility (I use HP's)
3) Boot the T40/T41 from the USB memory stick
4) Use FWBACKUP to create the HPA restore files on the USB memory stick (remember to limit the file size to below 2GB or it will fail - that is a DOS limitation - I use 1GB to be safe, it creates four files).
5) You're good to go.

Now you can boot a T40/T41 with a fresh hard drive from the USB memory stick and run FWRESTOR to create the HPA image. Then you can run the factory restore or use your favorite utility to clone your old drive - just the non-HPA partitions to the new drive. Tip: copy your utility to the same USB memory stick so you have all your HPA drive swap/upgrade tools together.

That's it. HPA restores aren't fast, but it works every time.

Note: With T42's and later, Ghost with the IB switch works fine. However, you usually need to reboot once or twice before the F11 key function works. If you have issues, there is also a fix mbr diskette from Lenovo that helps, too. Much easier than HPA restores!

P.S. R.G. is right, most people only know enough to get others in trouble! You have to know and remember quite a lot to avoid wasting a lot of time and encountering signifcant frustration.

[underclocker]

i. first have a target HDD which participates in the BEER/PARTIES architecture

This is my absolute favorite piece of information that almost no one knows or even understands. Not only is it amusing to read or ponder, but it's freakin' true!

In all my years of hard drive swapping, cloning and HPA restoring, I only came across one drive that didn't support BEER/PARTIES and I couldn't believe it until I Googled it. I do not remember the model.

Let's just say I was ready for a BEER or PARTY after that waste of time!

Seriously, BEER/PARTIES will be a Final Jeopardy answer when they finally have Jeopardy for Geeks.