Index:Minimum RequirementsGeneral TopologyInterface LayoutProcedureValidation, Backup and Saving

This section contains instructions on how to install and configure the Top of Rack Switch (ToR Switch) inside the local FIBRE island, which is responsible for the control plane network of the local environment and to link the island to the FIBREnet backbone network.

 ToR Switch Minimum Requirements

 General Topology


General topology and connections for an initial island setup:


General topology and connections for a complete island:

 

 Interface Layout


Standard bayface and interfaces for the DM4100:

 

 Procedure

 
Before the Switch configuration - Firmware upgrade: 
 To execute a local firmware upgrade, is necessary to configure a TFTP server on a notebook. It's recommended to use the “Tftpd32 4.0” software related in the example below. All the example is for the DM4100 Switch:
 
  1. Create a new folder, download and copy the new firmware file to it, from: ftp://200.130.15.187/Firmware_DM4100/

  2. Download and decompress the Tftpd32 server application: ftp://200.130.15.187/Tftpd32/

  3. Configure a secondary IP address in the notebook ethernet interface: 10.0.0.1/24.

  4. Execute the Tftpd32 server e configure it in the main screen:

    Current Directory: [fill in wit the entire filepath for the new firmware file]

    Server interface: 10.0.0.1

  5. Connect the notebook ethernet interface to the mgmt-eth interface on the Datacom switch (cat RJ-45).

  6. Access the equipment console and execute the following commands (this example is for the firmware file “DM4100_48P_OF_1.0.8.im” – substitute the file name if applicable):

    !
    configure
    !
    interface vlan 1
    ip address 10.0.0.2/24
    no shutdown
    end
    !
    copy tftp 10.0.0.1 DM4100_48P_OF_1.0.8.im firmware
    !

     

     

  7. Verify if the new version is marked as the next firmware to load (S Flag):

    !
    show firmware
    !

     

     

  8. Execute an equipment reboot and verify wether the new version is active (R Flag):

    !
    reboot
    ...
    show firmware
    !

     

     

  9. End

 

Configuration steps for the ToR Switch: 

All the examples are for the DM4100 Switch.

The initial configuration assumes the island is connected via VPN.

The 11th step is a "migration" step, to switch the connection from the VPN to the dedicated backbone called FIBREnet:

  1. Hostname
  2. Timezone
  3. SSH server, Telnet and HTTP
  4. SNMP server
  5. Routing and Default Gateway
  6. NTP
  7. Interfaces
  8. VLANs
  9. STP
  10. Users
  11. VPN to FIBREnet migration


  1. Hostname - replace teh "[island-name]" string by the institution acronym:

    !
    hostname tor-[island-name].fibre.org.br
    !



  2. Timezone - the clock timezone is set to UTF -3, with nem "BRT"

    !
    clock timezone BRT -3
    !

     

     

  3. SSH server, Telnet and HTTP - is recommended to disable telnet access, as the SSH is the standard in FIBREs network.

    !
    ip ssh host-key generate
    ! here a new key will be generated !
    !
    no ip telnet server
    ip http server
    ip http secure-server
    !
    ip ssh server
    !



  4. SNMP server - the SNMP server is configured with local operator's contact info, equipment location and the official FIBRE read-only comunity. It's recommended to disable the public community.

    !
    ip snmp-server
    ip snmp-server contact [Operator's e-mail address, or fibre-rnp@rt.rnp.br]
    ip snmp-server location [Institution Acronnym, City, UF]
    no ip snmp-server community public ro
    ip snmp-server community FIBRE-BR ro
    !ip snmp-server community !@FIBRE-BR#$ ro
    !



  5. Routing - initially, a default route to the VPN template is created in order to forward the internal traffic through the VPN gateway.



    !
    ip routing
    !
    ip route 10.0.0.0/8 10.[Institution-ID].0.70
    !



  6. NTP - the NTP server is configured for timestamp syncronization:

     

    !
    sntp client
    sntp poll 3600
    sntp server 10.128.0.150
    !



  7. VLANs - The [Institution-ID] number utilized in the IP addressing is defined in this document. The recommendation is to use the VLAN ID 60 for the island's control plane and VLAN 50 for the CMC icarus control network.



    !
    vlan qinq
    !
    interface vlan 10
     name Internet
     set-member tagged ethernet 1/1
     set-member untagged ethernet 1/2
    !
    interface vlan 50
     name Rede CMC
     set-member untagged ethernet 1/14
     set-member untagged ethernet 1/16
     set-member untagged ethernet 1/18
     set-member untagged ethernet 1/20
     set-member untagged ethernet 1/22
    !
    interface vlan 60
     name Rede Controle
     ip address 10.[Institution-ID].0.1/24
     set-member tagged ethernet 1/1
     set-member untagged ethernet 1/4
     set-member untagged ethernet 1/5
     set-member untagged ethernet 1/6
     set-member untagged ethernet 1/7
     set-member untagged ethernet 1/11
     set-member untagged ethernet 1/12
     set-member untagged ethernet 1/13
     set-member untagged ethernet 1/15
     set-member untagged ethernet 1/17
     set-member untagged ethernet 1/19
     set-member untagged ethernet 1/21
    !
    interface vlan 3220
     name FIBREnet Controle
     no ip address
     set-member tagged ethernet 1/47
    !
    interface vlan 3221
     name FIBREnet Dados
     set-member untagged ethernet 1/3
     set-member tagged ethernet 1/48
    !
    vlan-group 1
    vlan-group 1 vlan all
    !



  8. Interfaces:

    !
    interface ethernet 1/1
     description VMServer1 controle
     no shutdown
    !
    interface ethernet 1/2
     description Internet Uplink
     switchport native vlan 10
     no shutdown
    !
    interface ethernet 1/3
     description OFSwitch1 dados (pronto)
     switchport native vlan 3221
     no shutdown
    !
    interface ethernet 1/4
     description OFSwitch1 c (pronto)
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/5
     description OFSwitch2 c (netfpga1)
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/6
     description OFSwitch3 c (netfpga2)
      switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/7
     description OFSwitch4 c (netfpga3)
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/8
     shutdown
    !
    interface ethernet 1/9
     shutdown
    !
    interface ethernet 1/10
     shutdown
    !
    interface ethernet 1/11
     description Perfsonar int1
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/12
     description Perfsonar int2
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/13
     description icarus1 c
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/14
     description icarus1 cmc
     switchport native vlan 50
     no shutdown
    !
    interface ethernet 1/15
     description icarus2 c
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/16
     description icarus2 c
     switchport native vlan 50
     no shutdown
    !
    interface ethernet 1/17
     description icarus3 cmc
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/18
     description icarus3 c
     switchport native vlan 50
     no shutdown
    !
    interface ethernet 1/19
     description icarus4 c
     switchport native vlan 60
     no shutdown
    !
    interface ethernet 1/20
     description icarus4 cmc
     switchport native vlan 50
     no shutdown
    !
    interface ethernet 1/21
     description icarus5 c
     switchport native vlan 60
     no shutdown
    ! 
    interface ethernet 1/22
     description icarus5 cmc
     switchport native vlan 50
     no shutdown
    !
    interface ethernet 1/23
     shutdown
    !
    interface ethernet 1/24
     shutdown
    !
    interface ethernet 1/25
     shutdown
    !
    interface ethernet 1/26
     shutdown
    !
    interface ethernet 1/27
     shutdown
    !
    interface ethernet 1/28
     shutdown
    !
    interface ethernet 1/29
     shutdown
    ! 
    interface ethernet 1/30
     shutdown
    !
    interface ethernet 1/31
     shutdown
    !
    interface ethernet 1/32
     shutdown
    !
    interface ethernet 1/33
     shutdown
    !
    interface ethernet 1/34
     shutdown
    !
    interface ethernet 1/35
     shutdown
    !
    interface ethernet 1/36
     shutdown
    !
    interface ethernet 1/37
     shutdown
    !
    interface ethernet 1/38
     shutdown
    ! 
    interface ethernet 1/39
     shutdown
    !
    interface ethernet 1/40
     shutdown
    !
    interface ethernet 1/41
     shutdown
    !
    interface ethernet 1/42
     shutdown
    !
    interface ethernet 1/43
     shutdown
    !
    interface ethernet 1/44
     shutdown
    !
    interface ethernet 1/45
     shutdown
    !
    interface ethernet 1/46
     shutdown
    ! 
    interface ethernet 1/47
     description FIBREnet Controle
     no shutdown
    !
    interface ethernet 1/48
     description FIBREnet Dados
     no shutdown
    !



  9. STP (Spanning Tree configuration for the FIBREnet standard is not used anymore):

     

    !
    !spanning-tree 1
    !spanning-tree 1 vlan-group 1
    !



  10. Users - is mandatory to create the default FIBRE users within the equiment and their respective perimissions. This information can be requested by e-mail along the island installation. Furthermore, the local island team is allowed to create any other user account, according to the local policies.

     

    !
    ! request user info via e-mail and create users
    !

     
      

  11. VPN to FIBREnet migration - skip this step if the island is going to use a VPN connection. These configurations needs to be applied in order to migrate the island federal connection from the VPN to the dedicated backbone links known as FIBREnet. A Default Gateway can be configured in this step, if the local network uses a firewall or another gateway for internet connection. This changes in the ToR Switch has to be made in conjunction with another configuration changes in Dom0 following the same objective. It's recommended to access the equipments via console or the island network (10.[Institution-ID].0.1) to prevent any loss of session connectivity during the activity.

     

    ip default-gateway [internet-gateway-ip-address]
    !
    ip route 10.0.0.0/8 192.168.0.136
    no ip route 10.0.0.0/8 10.[institution-ID].0.70
    !
    interface vlan 3220
     ip address 192.168.0.[Institution-ID]/24
    !

 


 Validation, Backup and Saving

It's recommended to save the configuration already made at this point, in a separate file for backup and in the switch's startup configuration file:

!
show firmware
!
show running-config
!
! Salvar a saída num arquivo texto e guardá-lo como backup
! Salvar a configuração na startup-config com o comando:
!
copy running-config startup-config
!

 

Next step: 2. Virtualization Server