A cut in the 90 kV external power supply to a dam occurred at 12:30 a.m., causing the electric generator to automatically stop. The operator was warned of the outage by an alarm. Because of work being done on the dam, its lift gates were LOTO locked closed and the automatic float valve was isolated. The entire flow was discharged via the central valve. The two position encoders of the dam’s central valve, powered by the same power supply without backup, were no longer available and the PLC was no longer able to regulate the reservoir height. When the operator arrived at around 1:00 a.m., the height had exceeded the maximum water level by 15 cm and reached the load range of the backup PLC monitoring system (used to open the central valve and the micro-valves on the LOTO locked valves). The operator lowered the central valve to manual mode and restored the normal height of the reservoir at 1:30 a.m. The hydroelectric power plant was then restarted.
After the incident, the risk analysis for work was updated to indicate that the central valve should always be lowered at the end of each day. In addition, the hydraulic structures oversight body asked the operator to rethink its entire I&C system (systems requiring backup power supplies, common failure modes of encoders and other components, etc.).