preventing GPU BGA issue

[ryan1212]

I got a new motherboard from IBM a few months ago, and I am wondering what I can do to keep from ever having that issue again.

I know not to lift it by one hand, and I have heard of people transplanting the motherboards into r51 chassis which were tougher.

Has anyone found a way to reinforce the t40 chassis to make it more rigid?

[sojourner]

Has anyone found a way to reinforce the t40 chassis to make it more rigid?

I've wondered if there is a way to do this myself but have not had time to address the issue. Have thought about this, putting a bead of epoxy around the GPU board cementing it to the main board. The hope is, the epoxy bond would make the area a little more rigid, plus help prevent the GPU from separating due to heat expansion of solder joints (if that in fact occurs).

Talked with a business customer (just this week) which does a lot of epoxying; they are going to send me some epoxy, a syringe and needle to lay a bead. One other thing, my personal belief is that it's flexing of the mobo which is causing most of the problems. Apparently you are thinking the same way considering your desire for a more rigid chassis. I noticed even palm pressure on the palm rest causes the base of these units to flex, in the WRONG area (under the GPU)! So I've gone ahead and put just the right size self-stick 'feet' under the TP where the GPU is. It helps prevent some flexing!

This T41 is a very nice laptop and I don't want to lose it to GPU failure, so perhaps working together we can come up with some solutions to an awful and GROWING problem! I'm planning to try an epoxy bead and have already implimented installing additional 'feet'. Don't KNOW if that's 'the solution' but you may want to consider the same.

[poshgeordie]

You're not the first to think about epoxying the chip down.

Jamiphar who's highly recommended on these forums for graphics rework, does this as part of the repair process:

http://forum.thinkpads.com/viewtopic.php?t=48458

[rkawakami]

Here's a couple of technical articles that explains the use of epoxy corner bonds:

http://ap.pennnet.com/display_article/1 ... flip-chip/
http://www.circuitnet.com/articles/article_42329.shtml
http://www.globalsmt.net/documents/Tech ... nowski.pdf
http://www.circuitmedic.com/features/519.shtml

The last link talks about using spacers during re-flow in order to prevent the solder balls from flattening.

[ajkula66]

Something that must be also considered are constant heating/cooling cycles on the GPU. A long fan will take away some of the pressure in this respect, as will keeping the laptop on as much as possible and not turning it off unless absolutely necessary. Running TFC also helps.

And, yes, James does use epoxy and it works wonders, at least on the machines he had repaired for me.

My $.02 only...

[poshgeordie]

Very interesting articles rkawakami, especially the last one about using spacers.

Whilst reflowing T4x graphics, I've had a couple, where there is what can only be described as a solder explosion from underneath the chip. This is accompanied by a lump of solder oozing out between the two graphics memory chips through a couple of plated through holes.

From reading this article, I guess the chip is sitting down on the BGA's and the solder expanding out from where ever it can.

What I might try is to make some tiny spacers and insert them under the corners of the chip to see if it prevents this in future.

[sjthinkpader]

For spacers, I suggest a bead of RTV about midway on each edge of the BGA.

I concur with George as well that thermal cycles are likely the worst enemy of solder joints breaking and not bending. From my own T41's GPU going south, it seems they have a life span of about 2500 cycles.

If you are keeping the machine on for long periods, set the LCD backlight to turn off after a short period to prolong the life of the CCF backlight.

[poshgeordie]

If thermal cycles are the culprit, I do wonder why this only affects T4x's. After all, construction-wise the graphics chips are the same across the TP range.

When checking whether a T4x has a potential graphics problem I very gently flex the front of the unit; usually if a unit has a graphics fault, the very tiniest amount of flexing will cause display flickering.

Very rarely indeed can I replicate a graphics fault by pressing down on the graphics chip.

[ajkula66]

Heating/cooling cycles are a known culprit for GPU failures on A31p units...

[visionviper]

It's the heating/cooling cycles that weaken the solder connections and then the flexing that breaks them. That is on reason why the later T4x series laptops have less GPU problems - they do not flex as much.

[poshgeordie]

Thanks both of you for putting me right - appreciated

[sjthinkpader]

Different material has different thermal expansion coefficient (TEC). Package designer use material with matching TEC so that they expand and contract together. Sometimes they are layered so that intermediate layer material become the buffer between layers with material that have differing TEC.

When a package is poorly designed, the TEC of layered material may be mismatched and the expansion and contraction rate exceed what the solder balls can stand. Then they break. These are typically the corner balls. This is because corner balls are furthest apart and material contraction and expansion will be the largest. In the early BGA designs, sometime designers place dummy balls in the corner that carry no signal. Their sole purpose is to take the stress in thermal cycles.

When a solder ball fractures, they are still in contact. When you bend the board, they they may lose contact.

I doubt that ATI MCM (multi-chip-module) GPUs are more reliable in R5x. I think T4x may experience more on/off cycles due to they portability. R5x may be used more as a desktop replacement and experience less on/off cycles.

[dr_st]

I would also assume that 15" T4x models and R5x models would be less prone, due to some extra thickness of the chassis compared to a 14" T4x, but I am not sure about that.

[fasterbybike]

I would also assume that 15" T4x models and R5x models would be less prone, due to some extra thickness of the chassis compared to a 14" T4x, but I am not sure about that.

14" T4x and 15" T4x are the same thickness.

15" ARE less rigid (typing this on the 14" T40p with the 15" T42P next to it !)

Not sure about R5x models though as I don't have one.

[ajkula66]

Well, here's how I see it:

a) 15" units (both T4x and R5x) are definitely less rigid. However, they have better ventilation and are used as desktop replacements more often.

b) New soldering technique applied on T43/p machines (and on some very late T42/p units) has taken away a huge portion of the problem, given that these ThinkPads are far less likely to fail than their older siblings.

Rigidity that T6x series have brought along seem to eliminate the problem. The other side of coin is that they're nowhere as pretty as T4x as far as I'm concerned...

[dr_st]

a) 15" units (both T4x and R5x) are definitely less rigid. However, they have better ventilation and are used as desktop replacements more often.

Since the base of the 15" is the same thickness as the 14" as you say, it is obvious that they are less rigid, but on the other hand better ventilation might make the heat-cool cycles less extreme. Overall, would you say that the 15" are more, less or equally prone to the GPU problem?

b) New soldering technique applied on T43/p machines (and on some very late T42/p units) has taken away a huge portion of the problem, given that these ThinkPads are far less likely to fail than their older siblings.

If you have a T42/p, is there a way to check whether it is part of that later badge?

Rigidity that T6x series have brought along seem to eliminate the problem. The other side of coin is that they're nowhere as pretty as T4x as far as I'm concerned...

The rollcage makes the difference in the T6x. Personally I also find the T60 nice looking. The 14" is one of the best looking laptops I've ever seen, if not the best, and I definitely like its looks more than the 14" T4x series, and the 15" looks very similar to the 15" T4x. The widescreen T6x and the T61 is another story, unfortunately.

[ajkula66]

dr_st wrote:

Overall, would you say that the 15" are more, less or equally prone to the GPU problem?

A very tough call. All R5x units that failed on me were 15", namely R50p. And my experience with T4x machines is the exact opposite-failures have occurred mainly on 14" ones with Radeon 7500. So no, I don't have a definite answer...but would take my chances with a 15", for the sheer respect of an IPS LCD...if it has to be fixed, this alone makes it worth it IMHO...

dr_st wrote:

If you have a T42/p, is there a way to check whether it is part of that later badge?

Only by opening it up and looking at solder joints themselves. If I'm not mistaken, the colour around newer ones should be red. Date on the machine may give you a partial clue as well...anything produced after spring of 2005 should be considered safer, because that's about the time T43 was rolled out...

Hope this helps.

[beeblebrox]

It's the heating/cooling cycles that weaken the solder connections and then the flexing that breaks them. That is on reason why the later T4x series laptops have less GPU problems - they do not flex as much.

Not correct. We have a couple of T40's that are mounted in a suitcase board together with a small portable printer, all portable for travelling sales men. The notebooks can not be moved, just their lid opened and the presentation to the customer given.

About 60% of the whole bunch had experienced the flexing problem, which actually is no flexing but de-bonding of the Radeon chip.

It's all the thermal expansion and weak solder joints!
Of course, bending the board by carrying the notebook with one hand furthers the degradation of the solder joints much more.

For example, my own beloved T40p developed after 4 years a strange error this spring: 32MB instead of 64MB video ram. After another 3 months I got the pixelated video errors, AFTER I used Thinkpad-Fan-Control and forgot to set it correctly. The Radeon reached 85°C in a closed docking station. Immediately I let it cool down and then I rebooted. The Radeon was GONE. Boot screen a pixel mess. Pushing on the chip did not help.
Last weekend I used a professional heat reflower and within 3 minutes I had a fully functional T40p again, with full 64MB video ram.
So, my advice: keep the Radeon cool, use the long fan and a have someone with a good heat gun, just in case.

[sparta.rising]

Can anyone post a picture of the epoxy beed applied to the GPU?

[ryan1212]

I am kind of curious, the motherboard I received from IBM as a replacement , was it a refurb or new motherboard?

In either case would the chip have been soldered using the newer technique less prone to failure??????

It also is a different model number than the original board,
I had a 2373-75U, they sent me a 2374-PU2. Not sure what the difference is, if any.

Edit: I just looked up the serial number of the board they sent me, the warranty expired 2007-06-30, so the replacement they sent is likely a refurbished board.... Does it have the new soldering technique?

[ajkula66]

All T40/41 boards have old type of soldering applied, and yours fits in that group.

"Refurbished board" can mean a variety of things, but IBM planars were rarely repaired per se, the item in question probably came from a damaged unit that was returned under warranty or from lease.

[dr_st]

The T43 appeared in shops in March 2005. My 15" T42 has been manufactured in April 2005, so I guess it is a close call to know which batch it belongs to. Now that I think about it, it is quite possible that the new soldering technique has been applied to my laptop, since It also has the LG IPS screen, which was officially introduced only in the T43 line, and is featured only on some late T42s.

This could explain why I haven't had any problems with the GPU (Radeon 9600), even though I am far from being gentle with the laptop, and ran it through quite a bit on/off cycles. Then again, even though I am not gentle, I try my best not to flex the chassis, and in the past year the laptop has indeed served mostly as a desktop replacement, docked 95% of the time, and rarely turned off.

I may just open it up and take a look as the joints, for curiosity and some peace of mind. I plan to give this laptop to my sister soon, and I would hate for it to die on me or on her.

[beeblebrox]

The T43 appeared in shops in March 2005. My 15" T42 has been manufactured in April 2005, so I guess it is a close call to know which batch it belongs to. Now that I think about it, it is quite possible that the new soldering technique has been applied to my laptop, since It also has the LG IPS screen, which was officially introduced only in the T43 line, and is featured only on some late T42s.

This could explain why I haven't had any problems with the GPU (Radeon 9600), even though I am far from being gentle with the laptop, and ran it through quite a bit on/off cycles. Then again, even though I am not gentle, I try my best not to flex the chassis, and in the past year the laptop has indeed served mostly as a desktop replacement, docked 95% of the time, and rarely turned off.

I may just open it up and take a look as the joints, for curiosity and some peace of mind. I plan to give this laptop to my sister soon, and I would hate for it to die on me or on her.

If you read my post above, you'll see that there is nothing you can do. If the chip heats up the chip carrier expands and there is tension on the solder points. It all depends whether IBM has changed the carrier or the solder mixture which might be stronger and less brittle than as with the T40 series.
As I said, we have a bunch of T40 that were never moved, and still they got the GPU failure (a professional heat gun solved that problem easily, however).
I assume, that with the introduction of ultra-small chip carrieres like the one form Intel with the Macbook Air, the thermal expansion problems might be solved for a long time to go.

[poshgeordie]

Just opened up a T41 and 2 T42's and found the graphics and both bridge chips had red epoxy dots applied as in this photo.
I guess they'd had faulty graphics in the past which had been repaired in some way.

Currently they all show graphics failures, and reflowing has not sorted them out.
Therefore, assuming that they did work before they were bonded, then the bonding has not prevented the problem from occurring again.

[sojourner]

Currently they all show graphics failures, and reflowing has not sorted them out.
Therefore, assuming that they did work before they were bonded, then the bonding has not prevented the problem from occurring again.

Just a guess; I believe the 'red dots' are acting as:

1. spacers (between the MOBO and GPU substrates)
2. position restraints (so they don't move once solder is fluid)

I do not believe what you are looking at is bonding; these dots do not look sufficient to to bond as a good bead around the circumference would. PERHAPS a good bonding after they were originally reflowed would have prevented this GPU failure recurrence!

BTW, nice pic! Also, found a guy that does reballing for $50 ... don't know anything about the company, whether legit or not etc, but perhaps it could be good source for reball (which may fix the issues with your three MOBOS which reflowing does not).

[poshgeordie]

I think you're right there - such a pity that the owner went to all the effort of applying them for the reflow process, and then doesn't complete the job!

$50 - that's very interesting. Over in the UK, I hear that there are a couple of companies who do it for around 2 -3 pence (approx 4 - 6 cents US) per ball.
A T4x graphics chip has 696 balls so that's £24.53 (around $46) including taxes (at 17.5%).
On the other hand an new chip is £25 from Ebay costs £25 ($46), so I'm struggling to work out what to do.
I don't just like throwing out perfectly good chips for the want of getting them reballed; on the other hand I can buy a new one which will be ready to reflow for the same price!
However because we're getting so many chips which need to be reballed, we're looking into the costs (and skills) required to do reballing ourselves.

[poshgeordie]

BTW, nice pic! Also, found a guy that does reballing for $50 ... don't know anything about the company, whether legit or not etc, but perhaps it could be good source for reball (which may fix the issues with your three MOBOS which reflowing does not).

Thanks for the picture comment - just a cheap digital camera on a tripod.

Superior Reball has exactly the same wording as the Ebay site. Does anyone know anything about them?

<EDIT> scrolled to the bottom of the Ebay entry to see Superior's name - Doh!!

It does seem that both reballing AND reflowing for $50 is very cheap indeed.

[sjthinkpader]

A chip epoxy example can be seen in all the recent intel CPU flip-chip-BGA/PGAs. These have metal bumping (instead of balls) sometimes numbering a thousand. The bumps are applied either by plating or sputtering process. Epoxy is then applied after attachment to the plastic BGA/PGA assembly. Years ago, ceramic PGAs were used but phased out due to speed issues in favor of Plastic/copper BGA/PGAs.

The forces experienced by the corner balls in a temperature cycle may be quite large if the BGA package substrate and the MB (usually FR-4) Temperature Expansion Coefficient (TEC) different is large.

Normally, silicon TEC is small and FR-4 is large. The ATI GPUs are MCM where there are a lot of silicon coverage (3 chips) so the MCM itself may have a small effective TEC as compared to the MB.

http://scitation.aip.org/getabs/servlet ... s&gifs=yes

Many solutions were developed such as FR-5 (lower TEC), FR-4/ceramic/FR-4 sandwich circuit boards (expensive), elastomer buffer layer, etc to combat this TEC miss-match induced solder joint failure issue.

[joester]

Can anyone post a picture of the epoxy beed applied to the GPU?

Read this article to learn all about it.


Joe

[benvanderjagt]

I have been doing a lot of board-level laptop work (in the thousands), and I've done BGA reflows and reballs (maybe a hundred or so). I'm starting to come to the conclusion, just from the rates of failures that come to me, that the red epoxy is the cause of most of the failures. Perhaps it acts like water in cement in the winter time, but the stronger (and more) epoxy used, the more frequent the failure. Has anyone else seen this trend? I know I'm going against the grain, but I've looked at my crates of bad boards and unscientifically found a strong correlation between the red stuff and dead chips.

For what it's worth, I usually remove the stuff with heat, and when I do, it does change shape, so it sure seems to have the right properties to cause chips to gradually pop up.