[BNC-all] Update: Facility Notice

Wed Aug 2 13:06:27 EDT 2023

Dear BNC Users,

We have found some issues with the main static UPS system and those repairs are being completed.  At 2PM this afternoon we plan to switch to UPS power.  The transfer should be transparent, given no issues with the breakers.  Extra steps are being taken to manually operate process cooling water systems to maintain water pressure and flow to all critical research tools in the center.

Thanks,

Mark

From: BNC-all <bnc-all-bounces at ecn.purdue.edu> On Behalf Of Voorhis, Mark L.
Sent: Wednesday, August 2, 2023 10:48 AM
To: bnc-all at ecn.purdue.edu
Subject: Re: [BNC-all] Update: Facility Notice

Dear Birck Users,

The UPS field engineer continues to review the condition of the UPS and has not finished his evaluation of the UPS or the error logs.   We will not transfer power or test the 800 amp breaker @ 11 AM as originally planned and will be pushed to the afternoon.  We will communicate when updates and changes are available.  No power transfer will happen without prior feedback to the stakeholders connected to the UPS power distribution system so time is available to prepare for the testing.

Thank you,

Mark Voorhis
Building Manager
Purdue University
Birck Nanotechnology Center
Office: (765) 494-3036

From: BNC-all <bnc-all-bounces at ecn.purdue.edu<mailto:bnc-all-bounces at ecn.purdue.edu>> On Behalf Of Reger, Ronald K
Sent: Wednesday, August 2, 2023 8:56 AM
To: bnc-all at ecn.purdue.edu<mailto:bnc-all at ecn.purdue.edu>
Subject: [BNC-all] Update: Facility Notice

Dear Birck Users,

As of this morning we have determined that when a breaker tripped yesterday the back-up DC power supply did engage and provided power to the building after the trip.  Mark pulled a number of log sheets and data to determine this, and a meeting is scheduled with Purdue electrical engineering this morning to review the data and information in order to determine the root-cause of the breaker tripping.  At this time we are still on 'purchased power' without the back-up batteries and we need to get back to the normal configuration where the back-up batteries are hooked back into the system.  A possible scenario is this:

10:00 am             Meeting between Birck building management and Purdue electrical engineering to review data and observe electrical panels, and determine a recovery plan.
11:00 am             With the support of electrical engineering and trades switch the building back to the normal configuration with back-up batteries.

Note:  This switch back to normal configuration MAY cause a momentary disruption of power.  All equipment should be in a safe state prior to 11:00 in order to protect it in the event that power is momentarily disrupted.  Also, all hazardous gases will be shut off during this time so any runs using hazardous gases should be completed by 10:00 am.  The Toxic Gas Monitoring System will also be taken down at this time, and spurious alarms could occur.  Please follow the direction of the Fire Equipment Services officials regarding whether to evacuate the building or not.

We will provide updates as they become available, and thanks for your patience & support.

Ron

From: BNC-all <bnc-all-bounces at ecn.purdue.edu<mailto:bnc-all-bounces at ecn.purdue.edu>> On Behalf Of Voorhis, Mark L.
Sent: Tuesday, August 1, 2023 1:28 PM
To: bnc-all at ecn.purdue.edu<mailto:bnc-all at ecn.purdue.edu>
Subject: Re: [BNC-all] Facility Notice!!

Dear Birck Users,

I wanted to provide feedback to what happened this morning with the building wide static UPS power.  The static UPS power distribution system, supports are building control systems as well as critical research equipment in the center.  The main 800 amp breaker that is between the UPS output and building wide critical load tripped this morning.  As a result this completely shut down all critical research tools and building controls to critical services (e.g. process cooling water, chemical exhaust, cleanroom operations, and lab operations).  This assumes that the connected research tools in the cleanroom or laboratories and/or building control systems, cause the main 800 amp breaker to trip, or the breaker tripped itself, for some unknown reason.  After the 800 amp breaker tripped, we noticed the UPS unit had a high priority critical alarm and appeared to have failed.  With not knowing if the UPS, connected critical load, or the main 800A breaker failed, we decided to energize the critical load to the building on purchase power, bypassing the UPS system and suspect 800 amp breaker.  At this time we are on purchased power with NO UPS protection.

We have call the UPS vendor to schedule service for 10AM tomorrow morning to check the UPS system and switch the critical load back to UPS.  There is a chance that the 800A breaker is still the issue, but we have to test it.  If the breaker trips again, that becomes a more serious issue, that could require a full day shutdown for all UPS power to all research tools and building control shutdown involving the cleanroom and labs, to replace the breaker.   The breaker replacement will have to be planned and scheduled at a later date.

Please call me if you have questions or concerns and to coordinate with activities planned for tomorrow morning.

We are sorry for the interruption, this kind of breaker failure is very rare and has never happen in the last 20 years.

Best Regards,

Mark Voorhis
Building Manager
Purdue University
Birck Nanotechnology Center
Office: (765) 494-3036

From: BNC-all <bnc-all-bounces at ecn.purdue.edu<mailto:bnc-all-bounces at ecn.purdue.edu>> On Behalf Of Reger, Ronald K
Sent: Tuesday, August 1, 2023 10:42 AM
To: bnc-all at ecn.purdue.edu<mailto:bnc-all at ecn.purdue.edu>
Subject: [BNC-all] Facility Notice!!

Dear Birck Users,

Just now the building experienced a power issue and we have lost all exhaust.  Therefore work in fume hoods should be suspended immediately and no further work requiring fume hoods will be allowed until power is restored and all fume hood exhaust is stabilized.  The facility group is working to understand and solve this issue.

We are sorry for any inconvenience this causes, and we will notify you when work is safe to resume.

Ron Reger
Engineering Manager
Birck Nanotechnology Center | Room 2289 |
Office:  765.494.6667 | Email: rreger at purdue.edu<mailto:rreger at purdue.edu>
Wiki:  https://wiki.itap.purdue.edu/display/BNCWiki
iLabs:  https://purdue.ilabsolutions.com/homepage/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpurdue.ilabsolutions.com%2Fhomepage%2F&data=05%7C01%7Cbnc-all%40ecn.purdue.edu%7C3f3d65861da44ef16fca08db937acf1b%7C4130bd397c53419cb1e58758d6d63f21%7C0%7C0%7C638265927884752461%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RL2TVZK0SRrOF4S183RN%2B58SlalmPRAsmWUKgU%2BfAoA%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ECN/mailman/archives/bnc-all/attachments/20230802/b597c99b/attachment.htm>