Extended Generator Runtime      top of page
By Marc A. Soucy

As we prepared to publish the latest issue of designPlus, the Northeast prepared for a historic early season snow storm that caused long lasting and wide spread power outages to the region.  In fact, even as we go to press some residential and municipal customers are still without power more than a week after the storm.  Ironically, a designPlus article concerning extended generator runtime was already planned for based on wide spread power outages after Tropical Storm Irene this past August.  After the second utility outage event in nearly as many months, this topic is even more important to facility owners and operators.  Generators are very good at handling the short term outages, but it is the longer outages that will require some forethought and pre-planning in order to prevent a failure or to be prepared to react in the event a generator fails. 

Obviously, the first item to consider before the onset of any large storm would be to make sure the fuel storage tanks are full and the fuel has been polished and is ready to be introduced into the system.    Speak with the fuel supplier to understand their emergency plan and reconsider your onsite storage requirements accordingly.  During a prolong outage you will not be the only one requesting fuel and priority will be given to health and public safety facilities.  Also your vendors own supply lines may be adversely affected by their own power related issues. 

Other items to consider are replacing the belts, fluids, and filters prior to and during the extended runtime.  Even though these are small components to the entire generator system, they are the first to require maintenance and/or replacement during an extended outage.  Be proactive, replace all your belts and filters at recommended intervals and make sure you have extra stock on hand.  If you need to replace them during the outage, keep in mind that some of these parts can only be replaced after shutting the generator down, so develop a plan that would allow you to proactively service these items.  Depending on your business needs, this may build the case for a redundant generator so one could be serviced while the other remains online.  If redundancy is already in place, than a plan to proactively maintain wear components during outages should be developed.  In other less critical applications it may be possible to service the engine after hours or shed non-essential load to ‘free up’ a generator. 

A facility should evaluate if their generator redundancy or lack thereof coincides with the extended runtime objective. Sites typically specify the number of gallons to store on site to hit the runtime objective but may overlook the generator performance during the outage. Take for example a plan to operate for 48 to 72-hours continuously. This may not be a realistic expectation for a single generator. Redundancy in this case will help guard against unexpected failures and allow for planned maintenance during extended runtimes.

Know the routine maintenance schedules for your generators, as some of these might be exceeded during an extended runtime outage. These items should all be acknowledged and addressed beforehand. Typical items that could require attention at 250 hours would be topping off fluids, adjusting or replacing belts, taking oil samples, lubricate bearings, etc.

Proper training of the onsite staff is also something that needs to be addressed before emergency action is required. Knowledge of the equipment and location of any spare parts is essential to provide the needed attention quickly and efficiently. It’s important to note that an extended runtime is when the generator will experience the most severe operating conditions. The last two major storms and the resulting power outages taught us that a long term utility outage is a real possibility that we need to actively prepare and plan for.

top of page

Data center managers depend on Building Automation Systems (BAS) to control their chiller plant and to help maintain temperature and humidity on the raised floor. The building automation control industry continues to mature resulting in easier to use systems with a higher level of features, more capabilities, and better graphic interface that make them easier to use and understand. The value of this functionality has proven to be well worth the financial investment to design and build, particularly at the more complex facilities that operate a central plant filled with sensors and cooling machinery. Not long ago, chiller plants required a crew of full time operators to keep running, however now there is a greater reliance on Programmable Logic Controllers (PLC) to start and stop equipment, rotate operation for equal runtime, alarm upon failure, automatically optimize setpoints for peak efficiency and meter energy usage. As BAS controllers continue to improve functionality, operators take on more of hands off approach to running the plant. When this happens it is even more important to have a clear understanding of what your automation system is doing and why it reacts the way it does. It’s easy to set and forget after the initial start up is done by the BAS contractor and design engineer. When the knowledge base is lost after an experienced operator or building manager moves on to another position it is especially important to have a good documentation program in place. To help avoid these risks we have several recommendations in this issue of designPLUS to help you stay on top of your BAS and to bring new team members up to speed quicker after they join your team

Documentation

Start with the original construction documents or shop drawings that have the written control Sequence of Operation from when the BAS was initially installed.  The goal is to use it to build a written Master Sequence of Operations document that reflects the actual programming in place.   Frequently there are minor setpoint and timer modifications made between the original documents and the actual programming.  This occurs as a result of refinements made in the system after the original sequences are programmed.  Sometimes a programmer will fill in gaps that were not specifically documented.  For example a sequence may call for an additional chiller to come on line if the leaving water temperature from the cooling plant rises so many degrees above a particular setpoint “Start LAG chiller if leaving water temperature rises 3°F above setpoint”.  An improvement may include an additional conditional statement that may allow the chillers already on-line to gain control of temperature before another chiller is started prematurely.  The programmed addition may be  “……..temperature rises 3°F above setpoint for duration of 15 minutes”.  If this minor change is not documented in an As-Built Sequence of Operations then the operators may not be aware of the change and lose confidence in the BAS since it doesn’t match the documented description.   Additionally, undocumented program enhancements may not always be an improvement as they may be the result of a genuine misunderstanding of the original sequences.  A well intentioned programmer may also incorporate a small change based on experiences from other projects however these changes add to the mystery of what is really inside your BAS.  A robust commissioning process will flesh out hidden control code.  It is not always fair to assume the programmer knows how your system should operate if the original written sequences are not clear.  Communicating the design intent clearly is the responsibility of your design engineer.   Encourage communication between all parties involved with design, implementation and commissioning of the BAS.  Keep good notes during BAS commissioning to document changes for the As-Built Sequence of Operations.  Knowing what the BAS system is doing and why is key to building confidence in the system for the operators that are responsible for keeping your cooling plant online.

Throughout the life of the facility, there may be renovation or additions to the building that affect the BAS. This is where a Master Sequence of Operation comes into play.  It incorporates the As-Built Sequences from the original installation with all As-Built Sequences from subsequent projects into one cohesive document.     A Master Sequence of Operations should be broken down into sections for each subsystem written in a way that can bridge the understanding between a programmer who codes the BAS in machine language and plain English for the end user.  A well written sequence of operations will be clear enough to be understood by any reader and specific enough for the programmer to avoid the need to fill in the blanks.

Annual Testing

Depending on the system and the size of the facility, the initial BAS commissioning could be a significant task particularly when there are redundant controllers for critical operations.    This essential phase of the project might take weeks to get through at a mission critical data center.  The people involved usually include the BAS programmer, the design engineer, the commissioning agent, and hopefully the building operation technicians.  These people tend to come away from the experience knowing the systems inside and out and become a valuable resource for the building owner because of the history they carry and the consequential training they receive during testing.   Re-commissioning the facility is not usually as time consuming or intense as the original effort and can be broken up into smaller sections.   Even though certain tests cannot be repeated because the live load is critical, there is a great deal that can be accomplished during annual re-commissioning.   New members of the team that participate will gain understanding and confidence that the automation will react the way it is expected to.  Gaps in the written Master Sequence of Operations document get uncovered and filled in.  The emergency functionality described in words comes alive during testing and users who interface with the system daily often identify opportunities for improved operations.  The collective knowledge of the systems is expanded beyond the first few participants that originally commissioned the building.   Annual testing helps your staff know what’s in your BAS and have confidence it will perform as expected.

Training

Like all training, BAS training will reduce errors and improve uptime.  The first level of training should be about how to navigate the software and hardware that make up the BAS.  Many BAS manufacturers will offer generalized training for their particular user interface.  Rotate all staff members who use the BAS through this class to help them understand the capabilities of the system.  This is a safer approach then having them tinkering with pull down menus and search the help screens.  They’ll learn about tools that can help troubleshoot and understand your operation such as setting up trend graphs for specific areas of concern.

The second level of training should be specific to your facility.  The user interface screens are customized for the building and cooling plant they control.   Training should be created for each subsystem and broken down to describe inputs which may include sensors or user setpoints and the outputs that include component functions such as start/stop, valve, pump or fan modulation and alarming.  Subsystem interconnection should be described and the final results displayed on the user interface workstation in a cohesive usable format.  This training is best provided by your design engineer or a BAS representative with detailed knowledge about your configuration.   Training helps your staff know what’s in your BAS and how to use it

Qualified Personnel

Once the initial work of design and installation is complete, it is important to hire and retain qualified personnel in-house that are tasked with learning the details of your system to keep it running smoothly.  It is usually in your best interest to maintain good relations with consultants and contractors who did the original installation to assist in-house personnel as needed.    The manufacturer of the BAS Head End is most likely going to be a permanent part of your system for the life of the building because interoperability between different controllers from multiple vendors is an industry goal that has not been achieved yet.   Whether you like it or not, your initial selection of a control vendor for your BAS establishes a long term committed relationship with them and any discussion or follow through to divorce yourself from the vendor will likely cost you.  For this reason take care in choosing a BAS vendor and if at all possible maintain a favorable relationship with your provider.  A good BAS provider is a combination of strong local support and a solid product offering.   Consider your provider an extension of your team with specialty knowledge you draw on only when needed.  Utilize your consulting engineers to help keep your specific interests in mind when planning an expansion or BAS modification.    Insure that all parties involved know what’s in your BAS and how you plan to use it.

In closing, your BAS programming may have hidden surprises that surface at inopportune times. Good documentation of the sequences and training of qualified personnel will keep you firmly in control of your facility.

top of page

Initially, power for computer equipment in a rack was provided via “power strips”.  These were usually 15 or 20 amp rated units with multiple receptacles for plugging in the computer equipment.  As rack loads increased these “power strips” could not support the higher loads so rack mounted “power distribution units” (PDUs) were developed.  Depending on the manufacturer and optional features, PDUs included metering, remote monitoring and in some case remote control and load monitoring of each receptacle.  To support the high loads in the cabinets these PDUs were designed for higher current and multiple phases.  The National Electrical Code (NEC) has a provision that 15 and 20 amp receptacles must be protected by a 20 amp maximum circuit protective device – circuit breaker.  As the PDU designs proliferated, this NEC requirement resulted in many different circuit protection configurations for the groups of receptacles of a PDU assembly.

FEA has examined a number PDUs to determine how they were configured, their physical construction and to develop any application constraints.  These inspections were at the request of one client that was concerned with the proliferation of “rack mounted PDUs” that were being advocated by computer server manufacturers, cabinet manufacturers and “rack mounted PDU” manufacturers.  There are now dozens of units comprising multiple voltage, phases and current configurations.  With dual power corded equipment it is imperative that you thoroughly understand the specific “rack mounted PDU’s” rating and metering.  Below are a few examples of “rack mounted PDU’s” that we have examined.

All “rack mounted PDUs” were to be applied in cabinets that housed dual power cord computer equipment so this was the basis of the analysis.   The analysis evaluated the overall PDU assembly rating and the ratings of groups of plugs protected by the 15 or 20 amp circuit breakers internal to the PDU.  It is important to note that dual power cord equipment MUST be plugged into the exact same receptacle in both “rack mounted PDUs” otherwise it is impossible to insure that the branch circuit, circuit breaker will not be overloaded should one PDU fail. All the PDU’s will be applied on a 208 volt single or three phase system so all calculations are based on this utilization voltage.

Unit 1 Analysis

30A, 250v Single Phase – with branch circuit protection, receptacles and ammeters

This PDU includes a 30A 250V NEMA single phase twist-lock plug.  Internal to the PDU this is further sub-divided into two 15A 2P branch circuits protected by circuit breakers.  Each circuit breaker has an ammeter and feeds a group of receptacles. 

Analysis of the PDU per the NEC requirement determined the following:

• PDU assembly NEC rating – 24 amps @ 208 volts single phase – 5 kW.
• Internal 15A branch circuit NEC rating – 12 amps @ 208 volts – 2.5 kW
• PDU assembly rating equals PDU internal rating – 5.0 kW

Due to the potential load transfer of the dual power cord equipment the maximum load on a PDU branch should not exceed 2.5 kW so it is recommended that each ammeter be labeled:

DO NOT EXCEED
       6 AMPS

Unit 2 Analysis

30A 250v Three Phase – with branch circuit protection, receptacles and ammeters

The PDU includes a 30A 250V NEMA three phase twist-lock plug.  Internal to the PDU this is further sub-divided into three 20A 2P branch circuits protected by circuit breakers.  Each circuit breaker has an ammeter and feeds a group of receptacles.

Analysis of the PDU per the NEC requirements resulted in the following:

• PDU assembly NEC rating – 24 amps @ 208 volts three phase – 8.7 kW.
• Internal 20A branch circuit NEC rating – 16 amps @ 208 volts single phase  – 3.3 kW
• The combined NEC rating of the three internal branch circuits is 9.9 kW which is greater than the PDU assembly rating of 8.7 kW so the assembly rating is the limiting factor.

The maximum load on each internal branch circuit (group of receptacles) should not exceed 2.9 kW.  Since the PDU is feeding dual power cord equipment the load needs to be no greater than 50% of its rating so it is recommended that each ammeter be labeled:

DO NOT EXCEED
       7 AMPS

Unit 3 Analysis

30A, 250v Single Phase Tandem PDU – with branch circuit protection, receptacles and ammeters

Two identical 30A 250V single phase PDUs are housed in the same PDU enclosure.  One will be called “A” and the other “B”.  There is no physical separation between the “A” and “B” sections. 

The PDU includes two 30A 250V NEMA single phase twist-lock plug – one of the “A” circuit and one for the “B” circuit.  Each 30a circuit feeds two internal 20A 2P branch circuits protected by circuit breakers.  Each circuit breaker has an ammeter and feeds a group of receptacles.

The following NEC analysis is for each PDU plug:

• PDU assembly NEC rating – 24 amps @ 208 volts single phase – 5.0 kW per plug
• Internal branch circuit NEC rating – 16 amps @ 208 volts single phase  – 3.3 kW
• The combined NEC rating of the two internal branch circuits is 6.6 kW which is greater than the PDU assembly rating of 5.0 kW so the unit rating is limited to 5.0 kW.

The maximum load on each internal branch circuit (group of receptacles) should not exceed 2.5 kW.  Since the PDU is feeding dual power cord equipment the load needs to be no greater than 50% of its rating so it is recommended that each ammeter be labeled:

DO NOT EXCEED
       6 AMPS

This PDU was not recommended for the following reasons:

1. Both the “A” and “B” sources are housed in a common enclosure with no physical separation, although this PDU could be used as a 10kW “A” source only PDU. 
2. The assembly consists of large solder connections which are not deemed reliable enough for mission critical application.
3. The ammeter monitors the combined total amps of the two circuit breakers on either the “A” side or “B” side and as such there is no way of knowing if the individual 20 amp breakers are overloaded.
4. The ground wire is terminated to the PDU enclosure with the ground pin of the receptacles also terminated to the PDU enclosure.  There is no direct connection of the receptacle ground pin to the cord ground connector.

Conclusion
In all cases the rating of PDU’s 1, 2 and 3 is the NEC rating of the PDU’s NEMA plug.  Only PDU 1 has the capacity to allow the branch circuits to be loaded to their NEC rating.  Loading PDU 2 and 3 branch circuits to their NEC rating will result in tripping the 30 A breaker feeding the PDU even though the PDU branch circuit breakers will not be overloaded.  It is important to understand what amperage the PDU meter is measuring and adjust your load thresholds accordingly.  As we found with Unit 3 the meter would not provide accurate information to help guard against a internal branch circuit overload condition. 

top of page