A biopharmaceutical company committed to bringing to patients novel products for the treatment of rare and ultra-rare diseases, with a focus on serious, debilitating genetic diseases, was looking to expand. When they began building a new pharmaceutical research laboratory at their New England facility with an extremely condensed timeline, they reached out to Superior Controls to provide the qualified automated monitoring and alarm system (QMAS).
The new validated control system would monitor 78 relocated or new pieces of equipment. The critical monitoring and alarm system project completion date was just three months after the project kickoff, which was a very tight schedule for a customized system such as this.
The laboratory equipment to be controlled by the QMAS includes:
- Lab refrigerators
- Lab freezers
- Cryogenic freezers
- Water generation skid
- Glass washer
Superior Controls proposed a Rockwell Automation based solution. The off-the-shelf equipment was customized to provide the critical alarming, control, and reporting to the client’s specifications. The hardware provided a single control cabinet containing an Allen-Bradley ControlLogix PLC with ControlLogix I/O. The software platform included a FactoryTalk View SE server with local HMI clients, FactoryTalk Historian, FactoryTalk VantagePoint, FactoryTalk ViewPoint, and WIN-911.
Superior Controls designed a customized, validated QMAS to provide centralized monitoring and alarming for all 78 pieces of equipment, featuring local audio-visual beacons throughout the facility with an email notification system. With the new system, the client no longer has to be present in front of a machine to know it is in alarm. Those employees on site are notified by the beacons alert to check the equipment status. Employees both on- and off-site receive alarm notifications by email. The client created a detailed hierarchy of permissions and multiple notification lists that are triggered by which specific area has the equipment fault. At the facility, notifications are sent by room, and each area beacon covers a group of rooms.
In addition, the system provides data historian and reporting functions. It gathers data from all equipment and stores it for historical reference. To facilitate Cloud data storage, the system includes a hardware module as a local data collection and storage site before it is uploaded. The customer requested a minimum of 24 hours of local data storage, and the hardware module provided is able to capture more than 10 days of data, for redundancy.
Superior Controls built the entire control panel, tested it, and developed all software. Once integrated, the system was tested and implemented on site.
Pre-Validated Control Modules
Control modules developed in-house by Superior Controls based on past experience and projects to reduce development and test time. The functionality of each module has been tested and validated. Taking one day per control module to test, the five pre-validated control modules saved five days of testing.
A Tight Timeline
With the extremely short timeline, Superior Controls Project Manager Anthony Whitesell front loaded the project to stay ahead of schedule. “Right off the bat at the kickoff meeting, I asked for an instrument list. I requested a list of everything the customer wanted to connect to the QMAS: its name, description, and equipment number. I asked for that right up front because the first document we generate is called the I/O list – it’s the list of all the inputs and outputs of the system. We can’t proceed any further without that information.”
Also at the kickoff meeting, Whitesell outlined the functional specification and got initial customer buy-off to prevent any unwanted surprises later on. “Since I had all the stakeholders in the room, I tried to get the whole project defined and everybody pointed in the same direction right at the kickoff meeting. The client also oriented themselves to the success of this project. They were amazing – everybody pointed in the same direction and everybody stayed pointed toward the finish line.”
Part of the strategy to condense the schedule was to conduct the factory acceptance test (FAT) on the software functionality at Superior Controls. The control panel was tested at the fabricator, then the wiring was tested at the site with the control panel after it was installed.
Along with the tight timeframe, another project challenge was that the laboratory was being constructed concurrently with the development of the QMAS and delivery of the equipment. The laboratory was constructed on one floor of a five-story building and was gutted down to the concrete walls before building began. The space needed to be highly customized for the laboratory, with specialized equipment, air filtration suitable for biopharmaceuticals, and more.
At the project kickoff meeting, Whitesell noted, “When I first walked through the building, there were no walls, a ripped up concrete floor, and no ceiling. There were metal studs stacked up on pallets waiting to be installed.”
Working around and within the construction schedule required ongoing coordination. “We were working around framing crews, electrical crews, plumbing crews, and HVAC crews. There was a lot of coordination with the general contractor because my work had to conclude with their work, but our work typically comes after. We were working in parallel with people installing the walls that our connections had to be plugged into.”
For example, when the project kicked off, the room housing the control panel didn’t exist. “That was a unique challenge to coordinate with the general contractor and the entire construction crew to ensure that our system was installed and ready to be put to work when construction completed. The crew actually had to lift the control panel up to put the floor underneath it; that’s the level of construction that we were working in.”
“We worked very closely with the client’s management and skilled trades teams to coordinate schedules for the critical monitoring system. I attended a weekly trades meeting to facilitate the communication. It all went very well,” said Whitesell.
Site Acceptance Testing
The customer notified Superior that three pieces of laboratory equipment were indefinitely delayed due to the onset of the pandemic. Superior quickly pivoted to create a cascading site acceptance testing (SAT) schedule, allowing the partially complete system to be validated and functional until the final equipment arrives from overseas.
“We were able to break the system apart such that when we incorporate the utility equipment, it will not affect anything that we have already tested,” said Whitesell. “We used the modular design of the system with the notification to make a modular test. We tested the system with all pieces of equipment that arrived on time, such that they will not have to be retested when the remaining three machines arrive. We will be able to incorporate the new equipment without impacting any testing that we have already done.”
With the efforts of the teams at Superior Controls and the client, the system was constructed, programmed, installed on site and tested in according to the aggressive schedule. Had it not been for unforeseen delays in the delivery of three pieces of laboratory equipment due to the pandemic, Superior Controls would have completed the project according to the initial schedule.
Whitesell reflects, “Had it not been for the COVID-19 equipment delay, we would have finished the project on time, exactly three months from the project kickoff. The Superior Controls team worked very hard and the project went well. The customer is able to run the system even before the last three machines arrive, and they seem very happy with its performance.
With the QMAS in place, future additions to lab and storage equipment can be installed at the customer’s own pace.
Have a project to discuss? Let’s talk!