H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Third Tribulation: Forwarding Abnormality

Table of Content [Hide]

After a large number of internet cafe users were cut over to the S6500, the third tribulation struck! Nodes reported occasional service outages affecting one or several internet cafes connected under the S6500.

h3cs-first-general-purpose-mid-to-high-end-layer-3-switch-third-tribulation-forwarding-abnormality-1.jpg

Service was restored after shutting down and re-enabling the corresponding GE access port on the S6500 for the faulty internet cafe.

The support team promptly dispatched engineers to the faulty nodes. On-site engineers checked the Ethernet cable connectors of the affected GE ports, port mode configurations, and the grounding of the S6500 chassis, but found no obvious problems. The only suspicious point was that the GE ports were configured in auto-negotiation mode.

The GE ports of the S6500 were set to auto-negotiation because the connected customer-end devices in internet cafes included both 1000M and 100M ports, and internet cafes with 100M ports might upgrade to 1000M at any time. Therefore, all GE ports on the S6500 were configured to auto-negotiation.

h3cs-first-general-purpose-mid-to-high-end-layer-3-switch-third-tribulation-forwarding-abnormality-2.jpg

When FE/GE ports operate in auto-negotiation mode, compatibility issues with peer devices often cause link failures, frequent port Up/Down events, rate degradation, or packet loss. However, complete port lockup after a period of operation was rarely reported.

After thorough discussion with the customer, the support team decided to change all GE ports on problematic S6500 nodes to forced mode and keep records.

Nevertheless, reports of internet cafe service outages under S6500 continued to reach the trial support team. Each time, the team immediately sent engineers on-site to troubleshoot and collect fault data. For this specific issue, the S6500 R&D team also required on-site collection of debug information.

After analyzing fault logs and debug data from multiple nodes, the S6500 R&D team finally identified the root cause: abnormal Forwarding Information Base (FIB) table.

The FIB table abnormality stemmed from insufficient performance of the low-cost ASIC chip purchased under extreme cost-reduction measures. When a large volume of forwarding entries was written simultaneously, some entries could not be processed in time and were dropped.

After pinpointing the root cause of the FIB table abnormality, the S6500 R&D team released a corresponding software version, which added rate control and queuing mechanisms for writing forwarding entries into the FIB table.

The S6500 trial support team then upgraded the software version on all S6500 switches at the 112 trial nodes, overcoming the third tribulation.

h3cs-first-general-purpose-mid-to-high-end-layer-3-switch-third-tribulation-forwarding-abnormality-3.jpg

By Lanbras

Lanbras specializes in translating cutting-edge optical and Ethernet transmission technologies into clear, valuable insights that help our customers stay ahead in a fast-evolving digital world.

By turning complex technical concepts into practical, business-driven content, we aim to empower decision-makers with the knowledge they need to make confident, future-ready choices.

PREV: H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Second Tribulation: Specification Limitation

NEXT: H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Fourth Tribulation: Routing Abnormality

References

Lastest News & Blog about Lanbras

H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Fifth Tribulation: Excessively High CPU Utilization

Before the routing abnormality issue was fully resolved, the fifth tribulation struck! Following the recommendation of the S6500 R&D team, the routine inspection tasks of the S6500 support team we...

 May 09, 2026

Learn More

H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Fourth Tribulation: Routing Abnormality

As more and more internet cafe users and enterprise users were cut over and activated on the S6500, the fourth tribulation emerged! From time to time, nodes reported that internet cafe users connected...

 May 08, 2026

Learn More

H3C's First General-Purpose Mid-to-High-End Layer 3 Switch: A Breakthrough Story – Second Tribulation: Specification Limitation

In late 2004, H3C's newly developed S6500 general-purpose mid-to-high-end Layer 3 switch was delivered for trial deployment at 112 broadband MAN nodes of friendly user XX Telecom. As user services...

 May 06, 2026

Learn More