Zinc Whiskers – What You Should Know Now!

BACKGROUND

Metal whiskers have long been a known problem for electronics. For the last 10 years, the article Are Zinc Whiskers Growing in Your Computer Room? has been used as a guide to investigate and remediate Zinc Whiskers from data centers around the globe. Unfortunately, despite significant effort, Zinc Whiskers haven’t gone away.   As an industry it was generally considered that the only serious source of Zinc Whiskers in computer facilities was certain types of access floor panels. Facilities without these specific panel types were assumed ineligible for whisker contamination.   How wrong was that assumption! Zinc Whiskers have been found on a variety of metal components within all types of facilities, including: steel building studs; electrical conduit; suspended ceiling T-bar grid and hanger wires; and of course access floor panels, pedestals, pedestal heads, and stringers. This may be surprising, but it’s not really news.

PART I – Zinc Whisker Susceptibility   The real news is Zinc Whiskers are being discovered every day on cabinets, racks, and the servers and computers themselves. That’s right! Zinc Whiskers may be growing on and in computer hardware.   How is this possible? It’s simple really. Computer systems are a combination of electronic circuit cards mounted and contained within metal boxes and enclosures. The metal of choice is steel because it is conductive, strong, and reasonably inexpensive. The steel is often plated to prevent oxidation or rust. Zinc is still the plating material of choice because it’s relatively inexpensive, is conductive, and yields a good finish appearance. Many computer enclosures are zinc plated; so are rack rails, cabinet shelf supports, and other structural elements.   If Zinc Whiskers are everywhere, aren’t they noticeable? Remember that Zinc Whiskers are thinner than a human hair and roughly 0.5 – 5.0mm long. You have to be looking for them to find them. Look for them growing en masse. Seeing a single whisker is like looking for the proverbial needle in a haystack.    Zinc Whisker contamination should be considered whenever there are abnormally high failure rates – both catastrophic and less severe soft failures. The failure rate may peak within 72 hours of performing invasive maintenance work in or around the equipment.   Many factors determine the probability of Zinc Whisker failures. These include but are not limited to:

  • Age of the source material and therefore the general length of the whiskers.
  • Susceptibility to mechanical actions such as scraping, scuffing and vibration, that can cause whiskers to release from the host surface and migrate freely.
  • Susceptibility of equipment to whisker failures.

Many users wrongly conclude only power supplies are susceptible to whisker related failures. This is likely because power supply failure tends to occur with a dramatically loud ‘pop’ and cause a system outage. Unfortunately, power supplies are not the only exposed electronics in a computer system. There are a myriad of integrated circuits (chips), leads, circuit traces, and other components. To be sure, parts of all the items on this list may be concealed by plastic or solder mask and generally unexposed.   

But not everything is protected, and these uncovered leads are just as susceptible as the power supply. Zinc whisker bridges and shorts of exposed circuitry still have the potential to wreak havoc on a system. What happens if leads on the memory bus are intermittently shorted during the critical setup and latch portion of the clock cycle? Perhaps data will be corrupted. Perhaps the corruption will be detected and corrected by error correction algorithms. Perhaps the affected data is really an instruction for the processor. What if the processor tries to load and execute this corrupted instruction? Will the system failover or hang? Any engineer will agree that finding and fixing intermittent failures is one of the hardest things to do. ”If you can’t see it, you can’t fix it.” Whisker related failures fit into this category.

Many system anomalies are not logged or tracked. If a reset clears the situation, the problem is quickly dismissed as annoying but non-critical. Often, these on-the-floor fixes don’t get the visibility of management. Ask an IT manager if equipment needs to be reset and they’ll say, “…no, why do you ask?” Ask an operator if equipment needs to be reset and they’ll answer, “…of course, all the time, why do you ask?”

So, if Zinc Whiskers are everywhere and affecting equipment, how come it is not common knowledge? Most users get their information from personal experience or from trusted sources. If personal experiences are not memorable, it’s human nature to discount and discard them. If resetting a stuck machine is no more memorable than filling a coffee cup, it isn’t remembered. A power supply popping is unusual and memorable. Clicking the button is not.

In the IT world, trusted resources are typically associates and vendors. Neither one is talking because neither one has an incentive to talk. Users don’t admit they have Zinc Whisker problems because of fear of condemnation and repercussions from vendors. Users are supposed to honor their equipment contracts by maintaining suitable computing environments. Zinc Whisker contamination does not contribute to a suitable environment. Likewise, vendors aren’t talking for fear of liability. Vendors are supposed to honor explicit and implied warranties that the equipment they produce and sell is free from defects. If the very equipment is vulnerable and or producing the whiskers, there is a legitimate fear of legal liability. 

The result of all this silence is customer ignorance about a very serious topic.

PART II – The Reach of Zinc Whiskers and What to Do Next  

How bad is it?   Evidence suggests that Zinc Whiskers may affect one or more components in 50% or more of the racks and cabinets in any given environment. Historically, manufacturers only tested equipment when problems were suspected.  Users only tested when the manufacturers weren’t providing answers. Recently, large users have been willing to sponsor broader, facility-wide testing. Unfortunately, for the reasons indicated above, the specific results of these tests remain confidential. Suffice it to say, Zinc Whiskers affect or have affected virtually all vendors.  

If whiskers abound, why aren’t there more problems?

The evidence suggests that Zinc Whiskers tend to remain reasonably well connected to the host surface. Until they reach a certain length, Zinc Whiskers will remain connected until they are liberated by mechanical means such as rubbing and scraping. After they reach a certain length, not only is liberation possible from direct mechanical means but also from more passive means such as vibration or airflow. Once dislodged, Zinc Whiskers are free to migrate within the environment.

Zinc Whisker failures need not be catastrophic. Bit errors, soft failures, and other anomalies may be attributed to Zinc Whiskers.

What is the cure for Zinc Whiskers?

Generally, the accepted cure for Zinc Whiskers is to remove and replace the root source material with an uncontaminated version. It is not reasonable to replace every contaminated piece of equipment, either from a logistics or financial perspective. That doesn’t mean the problem should be ignored. . Zinc Whiskers will continue to grow. As they become longer, they become potentially more harmful. 

Users can’t stop using their equipment nor can they stop meeting the needs of the business through hardware migrations, moves and rearrangements. Users who want to proactively address the issue should develop a plan for managing the issue through staff training, vendor management, and equipment and facility handling procedures. 

There are many suitable healthcare analogies that can realistically be applied: don’t stop working with a sick patient; don’t ignore the patient’s condition. Rather, take proactive steps to help the patient while preventing the patient from infecting or sickening others.

Part III – Addressing Zinc Whisker Contamination

The following recommendations are based on a logical argument that to do nothing is neither proactive nor rational in the long term. Something must be done. Outlined below is one possible approach to dealing with broad Zinc Whisker contamination.

Users should require:  

  • All persons who enter the site will be informed of the presence of Zinc Whiskers and be required to sign a nondisclosure agreement. Violators of the NDA may jeopardize their employment or vendor status. 
  • All staff and visitors who have any business touching any equipment in the room must be trained and tested on Zinc Whisker Awareness
  • All staff and visitors who have any business working on any equipment in the room must be trained and tested on Zinc Whisker Management
  • Upon passing the Zinc Whisker management training, all staff and visitors will be required to sign the Zinc Whisker conduct pledge. This pledge will compel staff and visitors to treat Zinc Whiskers seriously and to take no action that would aggravate the problem. Their actions will reflect the best interests of the user and reliable computing. 
  • All cabinets will be examined for Zinc Whiskers. The results of the examination will be posted on the front and rear door of the cabinet. 
  • Identified Zinc Whiskers in or on cabinets will be so indicated with colored adhesive markers. The markers will serve to alert staff and visitors where the contamination is most significant. 
  • Staff and visitors will be expected, by virtue of their training and agreement with the pledge, to work around the contaminated areas to the best of their ability.

  Users will: 

  • Require, by way of purchase agreement, all new equipment to be free of Zinc Whiskers for a period of 36 months. 
  • Work with all vendors to help understand the problem and develop solutions for future designs. 
  • Seek to replace (either by purchase or through vendor agreement) any equipment that is expected to be on site longer than 18 months. 
  • Seek to monitor and manage any equipment that is expected to be retired or replaced in less than 18 months. 
  • Establish a monitoring program for failures. 
  • Establish test sites with regular sampling to monitor conditions in the room(s). 
  • Establish a regular cleaning program for the facility. 
  • Establish a cleaning program for inside racks. 
  • Continue with the investigative process to locate and eliminate any additional root sources. All cabinets in the data center should be inspected and tested, as needed, to determine where additional sources exist. 
  • Planning should begin immediately to undertake a thorough investigation, tracking, and remediation program. The program should include:
    • Identification of sources.
    • Management of the sources.
    • Removal of the root sources, as possible.
    • Cleaning of the data center to remediate and mitigate the potential impact.

CONCLUSION   Zinc Whiskers are more prevalent than previously considered and acknowledged. At the same time, we can live with Zinc Whiskers and enjoy reasonably reliable operations. But it is important to acknowledge and manage the condition – not ignore it. Living with a chronic contagious disease provides a useful operational model. Once a surface becomes a Zinc Whisker source, it will always be a Zinc Whisker source. Left undisturbed, reliable operation may continue. Unless interaction with that surface is required, the Zinc Whisker status of that material need not be disclosed. However, if interaction with the Zinc Whisker source is required, then service staff should be informed and trained to take appropriate precautions to prevent an unnecessary release of Zinc Whiskers and possible equipment damage or broader facility contamination.

Basic Hand Signals In Crane Operation

Cranes are commonly used in the construction of towers and industry, and in manufacturing heavy equipments. Cranes ranges from small site crane to big cranes and deck cranes that lift heavy equipments. Basically, they are temporary structures in construction. They are either fixed on the ground or hoarded on a purpose-built vehicle. Cranes come in different types such as jib, gantry, ship and deck, bridge or overhead, boom, tower, and mobile or truck.

Before operating the crane, operators should carefully read and understand the operation manual from the crane manufacturer. Further, they must always note any instructions given by a reliable instructor or operator. It is also crucial for the crane operator to understand the consequences of careless operation of cranes. They must be instructed of the proper use, prohibition and the safety rules and regulation during the operation.

It is always the responsibility of the owner to make their personnel aware of all federal rules and codes so as to preclude violations along with their penalties. Employers must also make certain that their operators are properly trained and are equipped with the know-how. To be safe in the operation of crane, it requires skill and exercise of great care and ideal foresight, alertness and concentration. Also strict adherence to proven safety rules and practices is necessary.

The personnel who handle the operation of cranes in an area must utilize hand signals, if necessary, as their means of communication. Here are the most commonly used hand signals during crane lifting operation:

1. HOIST. Raise the forearm vertically and extend the right arm straight out with forefinger pointing up. Then, move hand in small horizontal circle.

2. LOWER. Forefinger pointing down and extend right arm downward then move hand in small horizontal circle.

3. STOP. Extend right arm down with wrist bent, palm down and open.

4 SWING. Right arm away from body, point with finger in direction of swing of boom.

5. RAISE BOOM. Fingers closed and thumb pointing upward while extending the right arm straight out.

6. LOWER BOOM. Fingers closed and thumb pointing downward while extending the right arm straight out.

7. BRIDGE TRAVEL. Extend the right arm forward, hand open and slightly raised and make pushing motion in direction of travel.

8. TROLLEY TRAVEL. Thumb pointing in direction of motion with palm up and fingers closed, jerk hand horizontally.

9. EMERGENCY STOP. Extend right arm, palm down and move hand rapidly left and right.

10. MULTIPLE TROLLEYS. For block marked 1. hold up one finger, and two fingers for block marked 2. Regular signals come next.

11. RAISE BOOM and LOWER LOAD. Right arm extended and thumb pointing up. Flex fingers in and out as long as load movement is needed.

12. LOWER BOOM and RAISE LOAD. Right arm extended and thumb pointing down. Flex fingers pointing in and out as long as load movement is needed.

13. DOG EVERYTHING. Hold hands in front of the body.

14. MOVE SLOWLY. One hand gives any motion signal while the other hand motionless in front of hand giving the motion signal.

15. MAGNET IS DISCONNECTED. Spread both hands.

When using these hand signals be sure that you and the crane operator are familiar with these signals. A wrong signal could cause a serious injury or worst – death.

Always stay alert when you are working in construction near any crane. If possible, avoid working under a moving load and stay clear of the counter balance. Always use your safety devices and helmet to avoid injuries. Safety is always the top priority of all workers and the crane operator.

Portable Generators – Safe Use

The demand for portable generators has increased substantially in recent years. There are myriad reasons for this increase. Emergency portable generators can have significant benefits to individuals and communities, helping to save lives, and lessening the hardships caused by natural disasters and lengthy power outages. Consumers should, however, be aware of the dangers associated with improper use of electric generators. We will highlight a few of these in the following paragraphs.

Portable Generators Produce Poisonous Carbon Monoxide

Carbon monoxide is an odorless and colorless gas discharged in generator exhaust. Inhalation of carbon monoxide is often lethal, and a number of deaths occur each year as a result of consumer generator use.

In 2004, the Consumer Products Safety Commission (CPSC) studied deaths from generator use following four major hurricanes that struck land in the state of Florida. Powering air conditioners and other appliances during nighttime hours was the primary factor identified in generator-related deaths in the CPSC Florida study, and in each of the cited cases, improper location of the portable generator became key to the tragic outcome. In 2000, two children swimming behind a family houseboat on Utah’s Lake Powell drowned after losing consciousness when a portable generator beneath a swim deck produced dangerous fumes. Once again, poorly planned placement of a consumer-use generator was cited as the primary cause of the tragedy.

Because of many similar incidents, the Consumer Products Safety Commission promulgated in December, 2006 that all new generators sold after March of 2007 be affixed with labels setting forth technical and performance data, in addition to the following warning:

“Using a generator indoors CAN KILL YOU IN MINUTES. Generator exhaust contains carbon monoxide. This is a poison you cannot see or smell. NEVER use (generator) inside a home or garage, EVEN IF doors and windows are open. ONLY use outside and far away from windows, doors, and vents.”

The CDC reported that a small portable generator will produce the carbon-monoxide level of six idling cars, a reality that surprises many consumers. Carbon-monoxide levels can be compounded with generator use because the gas is heavy and tends to linger, making it difficult to expunge from an infected area. This means that generators are never safe to use indoors, including inside of open garages, and that during operation they should be located as far from residential units or buildings as possible. In particular, operation near windows, screen doors, vents, and air conditioning ducts should be avoided. Operators should also note wind direction, and locate generators so that prevailing air currents carry fumes away from nearby buildings or residences.

Though all portable generators produce carbon monoxide, certain models create less CO emissions than others. For example, generators equipped with overhead valve (OHV) engines, a standard design in modern models, produce less carbon monoxide emissions than models sporting older side-valve, pushrod engines. Any consumer who intends to use a portable generator in locations with restricted airflow should seek a model creating the fewest emissions possible.

Portable Generators must be Dry and Free from Debris

Safe emergency portable generator use requires planning. Since portable generators are often used in inclement weather, or during the night when visibility is restricted, understanding how and where to use them in advance is critical. It is best to operate generators only in open areas, and, whenever possible, in locations where the generators will be protected from falling debris like leaves and twigs, and from blowing rain, sleet, or snow. A portable generator should never be wet during operation. An operator should likewise never be standing in water or on damp ground when he or she starts a portable generator. Portable generators should always be grounded according to manufacturers’ recommendations. Methods of grounding vary by generator model, but in general will require that a generator be connected to a fixed metal object (for instance, a cold water pipe – spigots for hoses or sprinklers on the outside of the house can be effective choices for generator use).

If heavy debris falls onto the engine of a portable generator, covering it or restricting air flow, the unit can overheat, producing dangerous levels of heat that in extreme instances can even ignite debris. If possible, an operator should dislodge undesired debris from a generator using pressurized air rather than scooping the debris out by hand.

We recommend placing a portable generator atop a concrete pad which rests at least 15 feet from all residences or buildings. Ideally a small roof should cover the pad, leaving a minimum of three feet of clearance on all sides of the generator for ventilation purposes. Other suggested generator locations are beneath a canopy, or inside an open or well-ventilated shed or carport.

Portable Generators must be Properly Connected

A portable generator should never be plugged directly into a residential electrical system (i.e., a wall outlet). Appliances should either be plugged straight into a generator outlet, or into a generator-ready extension cord (often referred to on packaging as “generator cords”). Since portable generators will be placed outdoors, operators need to be sure that any extension cord employed has been manufacturer-rated for outdoor use, and that it carries the Underwriters Laboratories endorsement for the maximum wattage produced by the generator model. Whenever possible, operators should also avoid routing extension cords in a manner that would create tripping hazards or covering extension cords with combustible carpets or padding which can cause heat build-up and perhaps even fire. Particular care must be taken when using an extension cord in wet conditions. If an extension cord is hot to the touch, it has been overloaded and must be either replaced or its load reduced. Operators should periodically inspect all extension cords for frays, cuts, cracks, exposed wiring, and plug damage, and replace any which have been compromised. An operator should always power on a portable generator before connecting a load to it (this is true whether appliances are to be connected directly or via an extension cord), and when connecting appliances, an operator should first connect the highest-wattage ones.

Rather than plugging appliances into a portable generator directly or via an extension cord, a user may wish to employ an electric transfer switch (which should be installed by a licensed electrician or somebody familiar with building codes in the operator’s area). A transfer switch serves as a bridge between the generator and main circuit-breaker panel of a building or residence. It allows a portable generator to send power directly and safely into a home electrical system. The National Electrical Code (700-6) provides that transfer equipment must be designed and installed to prevent inadvertent interconnection of normal and emergency power sources. In other words, an electric transfer switch needs to stop a potentially-lethal problem known as back feed — electricity being created by a home generator that enters outside power lines where it poses an electrocution risk to unsuspecting power-company workers. In an attempt to ensure that transfer equipment is installed according to code, some local government agencies require that the installer obtain a permit prior to installation. A qualified electrician will know when a permit is needed, and how to go about obtaining one. One type of transfer switch, referred to as a double-pole, double throw model, won’t engage unless outside utility power has been safely disconnected. Models of this kind are an excellent way to ensure that rules of safety are observed.

Operators need to keep something else in mind: Because utility workers are completely vulnerable when working on downed lines, lines that they believe are without power, many municipalities have criminalized the reckless use of portable and home generators. Violators can be subject to harsh fines, and even incarceration if convicted. Improper or reckless employment of portable or home generators can also void homeowners’ insurance in the event of property damage or personal injury. Given the number of drawbacks, engaging a licensed electrician to install transfer equipment is a sound investment.

Operators should take care not to overload a portable generator. In addition to its running watts, all generators have a maximum or surge-watts rating, reserve power which is intended to start appliance motors, and is not available for more than a few seconds at a time. During normal use, appliances connected to a generator should not consume above 80% of the generator’s maximum running watts. This reduces the chance of unintended damage or overheating. The watts used by an appliance will often be listed on a data plate attached to its back or underside. If a data plate cannot be located, a wattage meter which is inserted between an appliance and wall outlet is a good way to determine its exact wattage demands. A list of the average watts used by many common household appliances is also available on our website. Lists like ours are not intended to be comprehensive; however, such lists give some indication of the number and size of appliances a portable or home generator model ought to power safely.

Other Safety Tips for Portable Generators

Portable generators use 12 to 18 gallons of gasoline per day; for this reason, an extended blackout will require substantial fuel reserves. Gasoline, diesel, and other fuels burned by portable generators are highly combustible, and storing substantial amounts of them can be hazardous if done carelessly. We recommend that gasoline be stored in a container meeting American National Standards Institute (ANSI) requirements, as well as any applicable state requirements (the state of California, for example, has more stringent requirements known as “CARB”). ANSI coordinates the development and implementation of voluntary safety standards by United States manufacturers. An ANSI-compliant container will always be prominently marked as such.

It is advisable to use a stabilizer, such as STA-BIL, in gasoline or diesel which will be stored for emergency purposes. The shelf-life of pump gas is roughly six months, and for diesel, roughly a year. Stabilizers can frequently double a fuel’s effective shelf-life. Some stabilizers can endow gasoline with a life span of up to two years, and diesel considerably longer. T-REX recommends that, ideally, gasoline supplies be used and replaced annually. It is good practice to use fuel stored for emergency use in other gas-powered equipment — like chain saws and lawn mowers, ATVs or motorcycles. This guarantees that fuel supplies are replenished, and that gasoline is always ready for use when needed. Having a few extra quarts of oil, air and fuel filters, and spark plugs is also advisable.

There should never be open flames near a portable generator during operation. Flames, including lighted cigarettes, should also be kept away from a generator fuel supply. In particular a candle should never be used to provide illumination when preparing a generator for operation or powering it on.

A portable generator should not be refueled while running. If possible, refueling should occur after the generator engine has been allowed to cool, to prevent inadvertent fire should fuel make contact with a hot surface. To avoid refueling spills, it is always wise to store alongside a generator a funnel that can be used for pouring fuel and a rag to clean up spills. It is also advisable to have a flashlight and fire extinguisher within reach.

Keeping a portable generator in good working condition is important to ensure that it operates efficiently and safely when needed. Most manufacturers recommend that portable or home generators be started and allowed to run for five minutes every three months to insure that they remain ready for use. During down time, fuel tanks should be kept full (topped off). This prevents condensation from accumulating inside the tank, and diluting fuel. Water-saturated fuel can produce starting problems, and the engine may sputter during operation. Unfortunately, few people who purchase portable generators for emergency power needs heed this simple advice. Ignoring it can mean that a generator will not operate when it is most needed. We also advise generator owners to have their units serviced annually by a qualified technician.

It is a sound practice to keep small children away from portable generators. Even if a generator is being operated in a safe and well-ventilated area, its engine parts can become burning hot during normal use.

Finally, certain portable generator models are safer than others. Emergency generators are not intended for daily use, but are built to provide continuous occasional power in an emergency, or during a natural disaster or blackout. Inexpensive models can be more susceptible than top brands to heat failure, and heat failure in extreme instances can start fires. There are many inexpensive generators on the market — while inexpensive does not mean bad or unreliable, lower-priced brands may not last as long, or be as safe, as their higher-priced counterparts.

Summary

Here is a summary checklist of safety tips for portable-generator operators:

oOperators should always read the operating instructions or user’s manual before starting or using a portable generator. For best results and ultimate safety, an operator should adhere to these instructions. Generator owners should never remove warning stickers or safety devices from their models.

oA user should be familiar with the sound of his or her portable generator during normal operations. This allows for quick and easy identification of a problem during use; if a problem is detected, operators must immediately power down the unit.

oA portable generator should be used only outdoors and as far away as possible from open windows, air conditioners, vents or air-conditioning ducts.

oUsers should check wind direction to ensure that during operation, exhaust emissions are being blown away from buildings and residences.

oThere should be plenty of ventilation on all sides of a portable generator.

oA portable generator should be operated only in clean, dry locations. A permanent or temporary shelter may need to be built to house a portable generator during its operation. An operator should never touch any generator unless his or her hands are dry.

oA portable generator should be properly grounded before use to prevent inadvertent electric shocks.

oAn undersized or frayed extension cord should never be used with a portable generator. If an extension cord becomes hot to the touch during use, it has been overloaded and the operator needs to replace it immediately with a larger cord, or power down the generator. Operators should always route extension cords to minimize tripping hazards.

oA generator should never be connected to an existing residential wiring system unless a DPDT electric transfer switch is used to prevent back feed, and isolate generator power from external lines.

oThe main circuit breaker in a home or building should be turned off before any appliances are connected to a portable generator.

oOperators should ensure that generators are not overloaded by too many or too-large appliances; it is a good safe practice never to attach enough appliances to exceed 80% of a generator’s running watts.

oFuel for the portable generator should be stored in a safe ANSI-compliant container.

oThere should never be open flame near a portable generator during operation or fueling.

oA user should power down and, if possible, allow to cool, a portable generator prior to refueling.

oA generator should be serviced at least annually even if it has not seen use. Unless an owner is qualified, it is best to engage a specialist for generator service or repairs.

_______________________

This article is for informational purposes only. It is not intended to serve as a substitute for, or replacement of, the operating instructions or user’s manual of any generator, regulations or guidance promulgated by government agencies, or recommendations made by qualified electricians. Generators should be used only in accordance with manufacturers’ instructions.

This article was prepared by the management of T-REX Generators. T-REX Generators is a virtual warehouse mega-store specializing in portable generators and generator accessories. We carry portable generators for natural disasters and emergency preparedness, job sites, recreation, and home back-up power needs. By supplying superior generators from industry-leading manufacturers, and maintaining a low-cost structure, T-REX is able to assure consumers that every generator purchased from us is a top value. For more information, visit our website at http://www.trexgenerators.com.