Today is a day for introspection that many businesses and IT departments will unfortunately overlook. The realm of IT should not interfere with operations technology (OT).
JeremyM questioned whether legal action is necessary to address Microsoft's persistent behavior of imposing updates. He suggested that the issue might be related to a Crowstrike problem and the necessary access for security applications to function properly. While criticizing Microsoft is common, it's important to recognize that the issue may not solely be attributed to them. Imagine the potential consequences if Microsoft restricted users from installing essential security tools like IDS/IPS or antivirus software, forcing them to rely solely on Defender.
Zensequitur pointed out that today serves as a moment for introspection, as many companies and IT departments may overlook the lessons to be learned. It is emphasized that IT should not interfere with operational technology (OT). Unfortunately, these lessons are often only learned through significant challenges, with the hope that future decisions will be made more wisely.
As I delved into the topic further, I discovered a workaround for a potential issue that hasn't impacted my company yet. This morning, I booted up my computer at my home office before heading to work, and all systems were back online via VPN. Could this problem be related to a reliance on cloud-based systems?
dwoodlock asked if the problem is related to cloud-based systems. Many of my coworkers have been impacted, but not me. They turned off their computers last night while I didn't. Make sure to check for any pending installations on your PC.
Cardosocea claims that the issue with running security applications may not solely lie with Microsoft, but also with Crowstrike. They express concern about the potential consequences if Microsoft were to restrict users from installing third-party security tools, such as IDS/IPS or antivirus software. This could lead to legal disputes and negative outcomes. The preference is for Microsoft to not enforce strict standards that limit user autonomy, disrupt workflow, override app defaults for promotional purposes, or bypass Windows firewall rules for automatic updates. These actions can be frustrating for users and have negative impacts on their experience with the operating system.
The current dilemma stems from a recent update rolled out by CrowdStrike. The delivery mechanism of these updates is what determines their presence on your device. As of now, I have been tackling this issue for over 24 hours, but progress is being made steadily.
JeremyM expressed his preference for Microsoft not imposing the "no opt-out option" standard, which disrupts workflow and resets app defaults to promote their browser. He also mentioned Windows firewall rules being bypassed for sneaky updates. While it's a valid point, the current issue at hand is different. Let's continue the discussion.
Global Tech Outage Disrupts Airlines, Banks, Healthcare, and Public Transit Services in the U.S.
A widespread technology outage is causing chaos in various industries, including airlines, banks, healthcare, and public transit systems. In the U.S., several flights were canceled on Friday morning, while public transport services also suffered disruptions. Stay updated on the latest news at www.yahoo.com.
Cardosocea bluntly remarked that after a year of being disregarded as the "idiot in the room," they feel vindicated as their warnings about a potential shutdown of chemical production sites in a Fortune 300 company have finally come to pass. It is clear that the importance of operational technology (OT) is often underestimated compared to information technology (IT).
Numerous businesses have opted to migrate their data servers and ERP systems to the Microsoft Azure cloud, streamlining operations through a user-friendly web interface. This transition was made in the hopes of achieving greater stability and reliability compared to traditional, on-site server setups. However, despite the initial benefits, there are instances where reliance on cloud-based solutions can lead to unexpected failures.
In an effort to enhance stability and reliability, numerous businesses have transitioned their databases and ERP systems to the Microsoft Azure cloud, abandoning local servers. This move towards cloud-based management through a web interface was believed to be a more efficient solution. However, despite these efforts, there is a misunderstanding among IT professionals that adding layers of complexity actually increases the risk of system failure, rather than minimizing it. This misconception has become evident when systems unexpectedly crash, reinforcing the fact that more complex setups do not always equate to improved reliability.
In recent years, many companies have transitioned from using local servers to hosting their databases and ERP systems on the Microsoft Azure cloud. This decision was made in hopes of achieving greater stability and reliability through a web interface. However, relying solely on cloud-based solutions can have its drawbacks, as seen when systems unexpectedly fail.
As a user of Visual Studio, I was given a complimentary year of Azure, but found it limiting to solely rely on cloud storage. I believe it is crucial to have offline backups of all important data, including CAD designs, PLC configurations, HMI projects, and material lists, stored on multiple hard drives.
I recall a past experience working in a company where critical HR and payroll information was solely reliant on a third-party server. Any downtime meant the inability to make vital changes such as timeclock entries, promotions, terminations, and new hires.
Just recently, news broke of numerous car dealerships experiencing a major shutdown due to a cloud-based dealership software system failure. This incident disrupted car sales, payroll processing, title and warranty procedures, parts procurement, and overall operations. It serves as a reminder of the importance of having a backup plan in place for critical business operations.
It appears that car dealerships are also dealing with a widespread technical issue. During a recent visit to have my truck serviced, I was unable to receive a printed receipt as their computer systems were offline. According to the staff, this was a nationwide problem affecting multiple dealerships.
- 09-07-2024
- Peter Nachtwey
According to Peter Nachtwey, car dealerships across the nation are facing a widespread technological issue, possibly due to universal malware. During a recent visit to get his truck serviced, he was unable to receive a receipt as the dealership's computers were down. The staff mentioned that this issue was affecting dealerships nationwide. Surprisingly, Delta, a competing dealership, seemed to be unaffected by the IT problem. It seems like being the unofficial IT guy is a common occurrence in situations like these.
Today, I enjoyed a complimentary ride on the city bus. Unfortunately, the contactless payment system was malfunctioning, possibly due to a disconnection from the central server.
Peter Nachtwey noted that even car dealerships are not immune to widespread technology issues. During a recent visit to get his truck serviced, he encountered a situation where the dealership's computers were down and they couldn't provide a receipt. The staff mentioned it was a nationwide problem. It makes one wonder if in the future cars will refuse to start until the oil change is confirmed internally for safety reasons, potentially leading to more visits to the dealership for service.
Arpus4KM mentioned that the vehicle will not start until the oil change is deemed "complete" through internal connection and confirmation. This serves as a reminder for manufacturers to take precautions, similar to seat belt interlocks, to ensure user safety and compliance.
Get ready for the chaos when the majority of stores and gas stations are unable to accept card payments. Many businesses lack a backup plan for manual transactions, and most individuals rarely carry cash. The concept of a "single point of failure" is alarming as we face uncertain times ahead. Prepare for disruptions in payment processing systems and be ready for potential challenges in the near future.
According to plvlce, it is hoped that lessons have been learned from seat belt interlocks. However, it seems that self-driving cars may soon be capable of repossessing themselves.
Ron Beaufort emphasized that a crisis may arise when the majority of stores and gas stations are unable to process card payments. Many businesses lack a backup system and people commonly rely on cards rather than cash. The concept of a "single point of failure" becomes relevant in such situations, indicating potential challenges ahead. An example of this occurred in my city recently when utility workers mistakenly damaged a single fiber optic line, disrupting 911 services for 2 days due to lack of redundancy.
Arpus4KM mentioned the potential for future challenges in automotive maintenance, where vehicles may require internal confirmation for completing tasks such as oil changes before allowing the car to start. This added safety measure could lead to more visits to dealerships for servicing, similar to how BMW charges for subscriptions for features like heated seats already built into the vehicle. In another example of lack of redundancy, an incident in the city resulted in the loss of 911 service for two days when a utility worker accidentally damaged a fiber optic line. This scenario of insufficient backup systems is common across various industries, with significant investments made in redundant systems for some components while neglecting others. The issue may not solely be due to cost considerations, but rather a broader oversight in system design and implementation.
During a project, I witnessed a situation where a construction worker accidentally unearthed a communication cable using a digger. After that incident, all excavation work was completed using water blast and a vacuum truck to prevent any further damage to cables. This proactive approach effectively eliminated any future instances of damaged cables during construction projects.
According to dmroeder, it seems that Delta was not affected by the issue. It appears that a significant investment has been made in our network infrastructure, including servers, switches, firewalls, and VPNs, to ensure security. Although I am not the IT person, I am retired but still stay informed about technology updates.
- 09-07-2024
- Peter Nachtwey
Arpus4KM raised an interesting point about the potential frustration of cars insisting on completing an oil change before starting again, potentially leading to a dealership visit for service. This issue is reminiscent of the concerns farmers have with John Deere tractors requiring a dealership visit for basic functionality.
Discover the root cause of the CrowdStrike IT outage through the insights of a former Microsoft engineer. Learn how a kernel mode driver played a crucial role in the incident explained by Dave. For more information on this topic, check out the book "On the Spectrum" at https://amzn.to/3XLJ8kY. Watch the full explanation on YouTube at https://youtu.be.
Is there anything preventing a frustrated Microsoft developer(s) from including a hard drive wipe on boot in the next Windows update? It frustrates me how challenging it is to disable updates from Microsoft. As a result, I prefer to operate everything on virtual machines that are not connected to the internet.
I believe the problem lies not in a Windows update, but in the ability of a certified driver to update external dependencies without requiring recertification. Additionally, I suspect that CrowdStrike has the capability to independently update definitions, as the certified driver failed to properly sanitize.
It is important to consider the vulnerability of software that has the ability to automatically update and has access to kernel or root system privileges, such as Windows and antivirus programs. In this instance, the issue was not with the Windows update, but the potential risk of a certified driver updating external dependencies without undergoing recertification. It is crucial to note that any software with the ability to auto-update and access sensitive systems is susceptible to malware attacks from within. While CrowdStrike may have the capability to update definitions independently, it is essential to remain vigilant for potential security breaches in the future.
That is a completely valid argument.
Robertmee raised a valid point about the vulnerability of software with auto-update capabilities that have access to kernel or root system privileges, such as Windows and antivirus programs, to malware attacks. While this time there wasn't an attack, the potential risk remains high. It is essential to understand that ignorance is not an excuse when it comes to cybersecurity. Quick downloads of anti-virus definitions are crucial, but failing to test them before implementation is irresponsible and risky. The recent incident involving Crowdstrike and its customers highlights a failure in meeting basic business continuity requirements. This issue is not uncommon in the IT industry, where risk assessments often take a back seat.