I oversee a network of approximately 290 PanelView 7 Plus devices located throughout North America. These devices are PV Plus 7 Performance models that have been experiencing issues recently. Some of them display a pop-up window and freeze, requiring a reboot to resume normal operation. The problem tends to occur randomly, with intervals ranging from 30 minutes to 24 hours. It does not seem to be related to the machinery's workload or processes, as the devices are only connected to the PLC and not to any external networks. All 30 engineers responsible for these sites began reporting the issue on Tuesday, and by Wednesday, all sites had at least one machine affected. The problem seems to be escalating rapidly, and I am in need of assistance. Surprisingly, Rockwell suggested a firmware update as a solution. Various troubleshooting steps have been attempted, including reloading a .mer backup, firmware updates/downgrades, loading a previous runtime backup, adjusting FTLinx settings, and applying relevant patches from Rockwell's knowledge base. Despite these efforts, the issue persists. While replacing the PanelView units with new ones seems to resolve the issue, it is not a viable solution for all 290 devices. Error messages such as "Application RSLinNG.EXE encountered a serious error and must shut down" and "Failed to load isapi dll - \windows//analytics/saw.dll" appear on the HMI during freezing episodes. I have exhausted all possible solutions and now seek assistance from the community for a resolution.
It's quite strange to experience these issues. I have only a few PV+7 units, and luckily none of them have faced any malfunctions recently. Have any IT solutions been implemented across the organization? Perhaps a new probe or inventory software? If there is a specific unit consistently malfunctioning, consider setting up a mirror port to monitor the network traffic. Are all terminals displaying the same error messages, or are these errors coming from various terminals?
Ken, your idea might be onto something. It seems that despite our collective engineering expertise, the team may have overlooked the fact that IT installed drops for SCADA this week. I will investigate by monitoring traffic and potentially disconnecting some cables. If this is indeed the issue, I will return to thank you.
It was previously stated that the PV was solely linked to the PLC, but new information reveals that this is not accurate. It is important to verify the facts before implicating the PV manufacturer. In cases where IT is involved, which is common in many instances, responsibility is typically assigned based on the principle of "you touched it last, you own it."
I consistently point the blame at the PV manufacturer, as I never encountered these issues with Mitsubishi or Omron. In my first decade working in automation, I never paid attention to firmware. However, in the past five years at an all-AB company, I have had to navigate through various versions of Studio, FTV, and Studio View. It seems like every drive requires a different software like FTLinx or RSlinx. But enough about my frustration with Rockwell. The team is struggling to identify a pattern in the sites experiencing connection drops. Some sites have the errors, while others do not. If you have any insights or suggestions, please share them with us.
It seems unfair to blame Rockwell for the lack of equipment updates within your company, considering your teams were responsible for installing and using the various versions. While I do criticize Rockwell when necessary, it appears that the blame is being misplaced in this situation. Regarding the unusual problem at hand, it is quite peculiar. While I have not encountered a wide area failure like yours with 290 panelviews, I have worked on numerous similar projects throughout my career. If this were a widespread issue, it would likely be discussed extensively on different forums, yet your case seems to be an isolated incident. It appears to be a unique issue within your infrastructure.
It sounds like you've tried quite a number of troubleshooting methods with no success, which is really frustrating. One suggestion would be to closely check the error logs of the devices, specifically around the time the freeze happens. There might be a certain process or event that triggers it. Also, including relevant data like OS version, firmware version and any third-party software or drivers in use would be helpful. It's worth mentioning that although the devices are not connected to an external network, there could be a network issue within the PLC network itself, possibly bottlenecking somewhere. I would also recommend getting in touch with a Rockwell service engineer directly, if possible. I've found them to be incredibly useful for these deep-dive issues in my own experience. Depending on the age of the units, it's worth looking into if any warranty or extra support might be in place - Rockwell offers comprehensive support plans that can prove immensely beneficial in such cases. Despite the lack of visible connection with the machinery's workload, try isolating one machine and monitor its workload minutely; sometimes there could be a tiny fluctuation that can give a clue. I hope this helps and best of luck with getting it sorted!
This is indeed a complicated issue you're dealing with, and I understand your pain. An apparent "random" issue can always be a challenge. Considering that you've already made an exhaustive effort with software fixes and firmware changes, it's possible that the problem could be hardware related. Have you inspected the physical conditions these devices are operating in? Factors like heat, moisture, or heavy-duty electrical environment may cause such crashes. Also, check any underlying issues in the PLC communication loop which might be causing overloads. Worst case scenario, despite the logistical nightmare, might be hardware replacement, not necessarily with fresh units, but swapping units around to see if the faults move with them.
It sounds incredibly frustrating to be dealing with such a widespread issue across all those devices, especially since it started so suddenly. Given that you've already taken extensive troubleshooting measures without success, it might be worth reaching out directly to Rockwell's technical support for a more in-depth analysis. Sometimes there could be an undocumented firmware conflict or a known issue that their support team can identify. Additionally, consider engaging with other users in online forums or user groups—there might be someone who has faced a similar issue and found a workaround that they haven't published yet. Good luck, and I hope you find a resolution soon!
It sounds like you're facing quite a challenge with those PanelView devices. Have you considered reaching out directly to Rockwell's technical support for a deeper investigation into the specific error messages you're encountering? Sometimes, they might have more targeted insights based on current issues reported by other users. Additionally, it could be worth checking for any environmental factors or changes in the sites where the issues are occurring, as sometimes even small shifts can lead to unexpected behavior in devices. Hang in there, I'm sure the community will have some great ideas!
✅ Work Order Management
✅ Asset Tracking
✅ Preventive Maintenance
✅ Inspection Report
We have received your information. We will share Schedule Demo details on your Mail Id.
Answer: The freezing and error messages on the devices could be due to various factors, such as software glitches, compatibility issues, or firmware issues.
Answer: Some troubleshooting steps include reloading backups, updating firmware, adjusting communication settings, and applying relevant patches from Rockwell's knowledge base. However, if the issue persists, further investigation may be required.
Answer: Rockwell may have recommended a firmware update to address known issues, improve device stability, or provide bug fixes that could potentially resolve the freezing and error message problems.
Answer: While replacing the units with new ones may resolve the problem, it may not be a feasible solution for all 290 devices. Exploring alternative troubleshooting methods, seeking community assistance, or engaging with Rockwell support for further guidance could help find a resolution.
Join hundreds of satisfied customers who have transformed their maintenance processes.
Sign up today and start optimizing your workflow.