Teaming and VLANs in NetWare*

Overview

Teaming Requirements

Command Lines

Allocating Memory

Adding an Adapter into an Existing Team

Removing an Adapter for an Existing Team

Hot Plug/Hot Unplug an Adapter

Removing a Team

Receive Load Balancing (RLB)

Usage Notes

Troubleshooting


Overview

Driver name: iANS.LAN Location on CD-ROM: \NWSERVER

iANS.LAN is the Intel software for NetWare* 5.x and 6.x that provides a variety of Advanced Network Services (ANS). ANS include teaming and VLAN-based features. 

Teaming

Teaming includes:

See the Teaming Options page for complete description of these modes.

For detailed instructions on configuring teams in NetWare, see the Examples File.

VLANs (Virtual LAN)

See the VLAN page for complete description of VLANs.

General Configuration Notes:

Example for a Single 802.1Q VLAN Team:

Load ce100b slot=5 frame=ethernet_802.3 name=e83
Load ians vlanid=2 frame=ethernet_802.3 name=T1-VL2 team=1
Load ians vlanid=3 frame=ethernet_802.3 name=T1-VL3 team=1
Bind ians e83 team=1
Load ians commit mode=AFT team=1
Bind ipx T1-VL2 net=2
Bind ipx T1-VL3 net=3

In order to function properly, the adapters configured for IEEE VLANs must be connected to a "tagged" port (called a trunk port by Cisco) on an 802.1Q-capable switch.

For detailed instructions on configuring VLANs in NetWare, see the Examples File.


Teaming Requirements

Server Adapters Supporting Teaming:

Some advanced features are also adapter dependent as the table below depicts. 

Adapter

VLAN
(IEEE 802.1Q)

Teaming Mode
AFT ALB SFT FEC* GEC* AD
Intel PRO/100+ Server Adapter x x x x x   x
Intel PRO/100S and PRO/100+ Dual Port Server Adapters x x x x x   x
Intel PRO/100 and PRO/100S Server Adapters x x x x x   x
Intel PRO/1000 Gigabit Server Adapter   x x x   x x
Intel PRO/1000 F Server Adapter x x x x   x x
Intel PRO/1000 T Server Adapter x x x x x x x
Intel PRO/1000 XT Server Adapter x x x x x x x
Intel PRO/1000 XF Server Adapter x x x x   x x
Intel PRO/1000 MF Server Adapter x x x x   x x
Intel PRO/1000 MT Server Adapter x x x x x x x
Intel PRO/1000 MT Dual Port Server Adapter x x x x x x x
Intel PRO/1000 MF Dual Port Server Adapter x x x x   x x
PRO/1000 MT Quad Port Server Adapter x x x x x x x

AFT - Adapter Fault Tolerance
ALB - Adaptive Load Balancing
SFT - Switch Fault Tolerance
FEC - Fast EtherChannel (*used here to include Intel Link Aggregation and static 802.3ad)
GEC - Gigabit EtherChannel (*used here to include Intel Link Aggregation and static 802.3ad)
AD - IEEE 802.3ad teaming mode

IMPORTANT: Before creating a team, adding or removing team members, or changing advanced settings of a team member, make sure each team member has been configured similarly. Settings to check include VLANs and QoS Packet Tagging, Jumbo Frames, and the various offloads. Pay particular attention when using different adapter models or adapter versions, as adapter capabilities vary.
NOTES: Copper gigabit adapters may be teamed with Intel PRO/100 adapters at 10 or 100 Mbps or with adapters from other vendors.

SFT mode only allows for two adapters in the team.

The ANS driver (required for all ANS features) for Intel PRO/100, as well as PRO/1000 adapters is located at \NWSERVER\iANS.LAN.

Multi-Vendor Teaming

Multi-vendor teaming (MVT) is supported on NetWare 5.x and 6.x systems.

Only some vendors support LAN drivers that register NESL events. These drivers are required for MVT support.

MVT only supports mixed speed adapters in AFT, ALB, and SFT modes. MVT Load Balancing is provided at layer 3 for TCP, UDP (including multicast), and IP/Snap protocols.

Hot Plug is not supported for members of a multi-vendor team. Hot Add and Remove of non-Intel adapters may be done in FEC/GEC and 802.3ad modes only.

NOTES: ANS must be loaded prior to installing a non-Intel adapter for Multi-vendor teaming.

If a failover occurs or if the primary adapter is changed, the MAC address changes. Clients are notified by an ARP broadcast. When any team member receives an ARP request, the ANS software responds only with the MAC address for the current primary adapter.



Command Lines 

Basic Structure: 

  1. LOAD base driver (CE100.LAN or CE1000.LAN) with appropriate parameters, including frame size.

  2. LOAD iANS driver with appropriate parameters, including frame size.

  3. BIND iANS driver to the adapter name.

  4. Create the team: LOAD iANS COMMIT <mode>.

  5. BIND the protocol to the iANS driver using the name that was assigned when iANS was loaded.

If you are using only one frame type, a single team, and no VLAN, you can bind the protocol directly to the iANS driver without using an assigned name. Include a net=n number, especially in a script since the system requests the number if not given.

Example of Mixed Speed Fault Tolerance Team:

Load CE100b name=100Meg
Load CE1000 name=Gigabit
Load iANS
Bind iANS Gigabit Primary
Bind iANS 100Meg
Load iANS COMMIT MODE = AFT
Bind IPX iANS Net=2
Bind <Protocol> iANS <Address>

For more examples of specific load commands for AFT, ALB or FEC see the adapter teaming Examples page.


Allocating Memory

When using teams or multiple VLANs, the server's default packet buffers probably need to be increased. To do this, add the following lines to the STARTUP.NCF file which is located in the same directory that NetWare is launched from. Usually C:\NWSERVER\STARTUP.NCF.

SET MINIMUM PACKET RECEIVE BUFFERS = 500 (or higher)
SET MAXIMUM PACKET RECEIVE BUFFERS = 2000 (or higher)

"SET MINIMUM PACKET" designates the minimum number of packet receive buffers the system shall allocate, and "SET MAXIMUM PACKET" designates the maximum. Make sure that the maximum setting is equal to or greater than the minimum setting. The number of buffers required is based on the number of VLANs and which teaming features are in use.

For every virtual adapter, team, or VLAN used, iANS requests 64 buffers. When in Adaptive Load Balancing modes (ALB, FEC/GEC and Link Aggregation - static IEEE 802.3ad), 64 buffers are needed for each adapter in the team. The non-load balancing AFT and SFT modes only require 64 buffers for the team.

As an example, an ALB team of two adapters that uses 12 VLANs would require 64 [buffers] * 2 [adapters] * 12 [VLANs] = 1536 for the minimum packet receive buffers. This number is in addition to any other buffers that the server may require for other purposes, including other adapters.

The default amount of memory used by each NetWare receive buffer is approximately 4K (varies slightly with different versions). If the extra memory requirement is a problem, the following configurations may help to reduce the impact.

NOTE:  This does not work if Ethernet Jumbo Frames are in use. In fact,  the packet size must be increased to accommodate the jumbo frames. 

Adding an Adapter into an Existing Team

A member can be added into team after a commit is done in the following way:

  1. Bind the new adapter to iANS using the 'bind" command. For example "Bind ians ce_100II".

  2. Use the "load ians recommit" command to complete the operation.

  3. You cannot hot-add an MVT adapter into a team that was created with Intel adapters only.

  4. For an FEC/GEC team the member should be added to the team with the link down. The reason is that when a port is added to a switch channel before the adapter is added to the ANS team, disconnections will occur because the switch will start forwarding traffic to the port before the member is actually configured. The opposite, where the member is first added to the ANS team and then added to the switch channel, is also problematic because ANS will forward traffic to the member before the port is added to the switch channel and disconnection will occur."

Removing an Adapter from an Existing Team

In an FEC/GEC team the member should be removed from the team with the link down for the same reason as described for Adding an Adapter into an Existing Team.

Hot Plug/Hot Unplug an Adapter

In FEC/GEC an adapter should be hot plugged or unplugged with the link down for the same reason as described for Adding an Adapter into an Existing Team.


Removing a Team

To remove a team, unload iANS.LAN by typing "UNLOAD IANS" at the console. Remember to comment out any lines you added to autoexec.ncf or a batch file if you do not want them to reload when you restart the server.


Receive Load Balancing (RLB)

Receive Load Balancing (RLB) is a method for dynamic distribution of received data traffic load among multiple physical channels. The purpose of RLB is to improve overall bandwidth and end station performance in teaming mode ALB, which currently receives all incoming traffic only on the primary member. RLB provides receive load balancing by dividing incoming IP traffic among all the adapters, with the ability to shift traffic away from any adapter that goes out of service.

Supported OS

·        NetWare 5.x and 6.x

Refreshing the load balancing

In order to achieve a fair distribution of clients among all enabled members the receive load balancing client table is refreshed at even intervals (default is 5 minutes). The refresh involves selecting new members for each client.

Adapter Failure:

In case of adapter failure the primary member will work in promiscuous mode until the return of the disabled member or hot removal of this adapter. Working in promiscuous mode can degrade performance.

In RLB mode there must be a designated preferred primary. If the user doesn't choose one, ANS will do so.

If the primary adapter has a failure another primary will take its place. As soon as the preferred primary is enabled, a fail back will occur and the preferred primary will return as the team primary.

Change Preferred Primary:

Change primary is not allowed in RLB mode of Netware ANS. The user has to reload the team in this case.

Removal of Preferred Primary:

Re-adding a dynamically removed preferred primary in a team is not recommended without the following provision. When dynamically removing a preferred primary from a team, its MAC address will be in use by the team and can therefore create a conflict on the LAN if the preferred primary is re-added back into the team or anywhere else on the network. To avoid this problem, you must reload the team after dynamically removing the primary adapter.

Hot Remove of Secondary:

All clients are failed over to the primary (such that the team is actually working in ALB mode).

The team will return to work in mode RLB gradually as clients refresh their ARP tables. This process may take up to 10 minutes, depending on the refresh timing of the client's ARP table .

Adapter Hot Add

The clients are rebalanced over the new adapter if it is in the highest speed group of adapters.

Enabling/Disabling RLB

RLB is enabled by default in ALB team mode. To disable RLB, load the team with ALB mode and write:

        rlbmode = disable

Enabling or disabling RLB cannot be done after commit.

Example: Load ans commit mode = ALB rlbmode = disable


Usage Notes

  1. Gigabit adapters - Intel copper-based gigabit server adapters (e.g., PRO/1000 T, PRO/1000 XT) may be teamed with Intel PRO/100 server adapters for AFT, ALB, SFT, and FEC modes. All teamed adapters must be configured for the same speed and duplex.

  2. Required Settings in STARTUP.NCF:

    SET MINIMUM PACKET RECEIVE BUFFERS = 500 (or larger)
    SET MAXIMUM PACKET RECEIVE BUFFERS = 2000 (or larger)

    See "Allocating Memory for VLANS" in Memory Configuration Requirements for more information.

  3. Display the current status for all adapters in a iANS team. Example:

         LOAD iANS STATUS team=n (where n=team number)

  4. Create the primary adapter. Example:

         BIND iANS CE100B_1_E82 PRIMARY

    This can only be done with the BIND command.

  5. When using iANS with SMP mode enabled, do not use the Stop Processors command after loading iANS.LAN.  The Novell OS does not currently have a mechanism for informing the intermediate driver of a change in the number of active processors, and this could lead to a server abend if iANS attempts to use a deactivated processor.

  6. Due to some potentially severe problems in registering our driver with NEB.NLM, a PCI Hot Plug capable server must have iANS.LAN loaded before the Hot Plug modules are loaded. To do this, make sure the iANS.LAN load lines precede the NCMCON.NLM load lines in the AUTOEXEC.NCF.

  7. When a multiple-adapter team is created (other than AFT), it uses all the functionality of the least-capable adapter in the team. In order to add an adapter to an existing team it must have the same base functionality enabled as that of the adapter with the lowest common feature set in the team.

  8. ANS probes are turned off by default when only Ethernet_802.2 and/or Ethernet_802.3 frame types are in use. This is due to the fact that Ethernet_802.2 and Ethernet_802.3 frame types may not allow ANS probes to be sent correctly and may cause one or more adapters to failover unnecessarily. If you intend to use the probe feature of ANS, Intel recommends that you load the frame types Ethernet_II or Ethernet_Snap. You do not need to bind any protocols to these frame types.


Troubleshooting

  1. When IANS.LAN forms a team, it may report the following error message: "Primary adapter does not sense any probes. Possible reason, partitioned team."

    In addition, if VLANs are enabled at the switch, the adapter will continuously fail over while there is no traffic on the network. This has been observed when connected to Extreme Summit2 and Summit4.2 switches.

    This fail-over does not affect performance. But the error message can be turned off with the command PROBES = OFF.

  2. If, at commit, you receive the error message: "Failed to create new team," ensure the following:

    1. All adapters have loaded drivers and have the same frame types loaded.

    2. iANS is loaded once for each VLAN and frame type.

    3. All adapters are connected to the same switch.

    4. iANS is not bound to an unsupported adapter.

    5. An "iANS BIND" command has been issued for each adapter and frame type in the team.

      NOTE: Novell's CONFIG command does not reflect the iANS BIND statements until after the COMMIT has been successfully issued.
  3. In NetWare, the following message may be displayed: "IPX received an incomplete packet from network 00001011:00A0C9AADFC3. 85 incomplete packets received."  This message occurs when NetWare's internal routing mechanism mistakes the Intel iANS driver's probe packets for NetWare RIP packets. To prevent this message from displaying, make sure that IPXRTR.NLM is loaded on every NetWare server on the LAN. For more information on IPXRTR, see Novell's documentation or their Support website: http://support.novell.com.


Last modified on 2/04/04 10:59p Revision 31