Cisco InfiniBand 4x Betriebsanweisung

Stöbern Sie online oder laden Sie Betriebsanweisung nach Netzwerk-Switches Cisco InfiniBand 4x herunter. Cisco InfiniBand 4x User guide [en] Benutzerhandbuch

  • Herunterladen
  • Zu meinen Handbüchern hinzufügen
  • Drucken
  • Seite
    / 174
  • Inhaltsverzeichnis
  • LESEZEICHEN
  • Bewertet. / 5. Basierend auf Kundenbewertungen
Seitenansicht 0
ibm.com/redbooks
Redpaper
Front cover
Implementing Cisco
InfiniBand on IBM
BladeCenter
Khalid M Ansari
Robyn McGlotten
Matt Slavin
David Watts
Add high-speed 4X InfiniBand
networking to your BladeCenter
Plan and configure the solution for
your environment
Learn about the latest
InfiniBand DDR products
Seitenansicht 0
1 2 3 4 5 6 ... 173 174

Inhaltsverzeichnis

Seite 1 - Redpaper

ibm.com/redbooksRedpaperFront coverImplementing Cisco InfiniBand on IBM BladeCenterKhalid M AnsariRobyn McGlottenMatt SlavinDavid WattsAdd high-speed

Seite 2

viii Implementing Cisco InfiniBand on IBM BladeCenterTrademarksThe following terms are trademarks of the International Business Machines Corporation

Seite 3 - October 2007

86 Implementing Cisco InfiniBand on IBM BladeCenter5.1.3 Configuring IP over InfiniBandThe IPoIB protocol passes IP traffic over the InfiniBand netw

Seite 4 - Contract with IBM Corp

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 87ib0 Link encap:InfiniBand HWaddr 80:00:04:04:FE:80:00:00:00:00:00:00:0

Seite 5 - Contents

88 Implementing Cisco InfiniBand on IBM BladeCenter5.2 Configuring the IPoIB on a Windows hostThe follow sections cover the procedure to install the

Seite 6

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 892. Insert the InfiniBand host driver CD or download the Windows Server 2003 d

Seite 7 - Contents v

90 Implementing Cisco InfiniBand on IBM BladeCenter5.2.2 Verifying the installationFollow these steps to verify the installation:1. To verify the dr

Seite 8

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 913. Verify that a Topspin IPoIB Virtual Channel Adapters for each port on the

Seite 9

92 Implementing Cisco InfiniBand on IBM BladeCentergateway assigned), using the Network connections menu. The first port on the HCA becomes interface

Seite 10 - Trademarks

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 933. Enter an IP address, subnet mask, and optionally a default gateway. Then,

Seite 11

94 Implementing Cisco InfiniBand on IBM BladeCenterThis completes the IPoIB configuration process.5.2.4 Verifying the IPoIB communication To verify

Seite 12

© Copyright IBM Corp. 2007. All rights reserved. 95Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gatewayThis chapter describes

Seite 13 - Comments welcome

© Copyright IBM Corp. 2007. All rights reserved. ixPrefaceInfiniBand® networking offers a high speed, low latency interconnect that is often a require

Seite 14

96 Implementing Cisco InfiniBand on IBM BladeCenter6.1 Benefits of booting from high-speed InfiniBand The BoIB feature on the IBM BladeCenter blade

Seite 15 - IBM BladeCenter products and

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 976.2.1 Pre-configuration checklistBefore proceeding to implement the

Seite 16 - 1.1 BladeCenter chassis

98 Implementing Cisco InfiniBand on IBM BladeCenterLook for the .Boot extension in the description as shown in Figure 6-2 to confirm that Boot over I

Seite 17

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 992. From the Element Manager UI, select Fibre Channel → Storage Manage

Seite 18

100 Implementing Cisco InfiniBand on IBM BladeCenter6. Click Next. The WWNN is assigned to the SRP host shown in Figure 6-6.Figure 6-6 Define SRP h

Seite 19 - 1.2 Blade servers

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 1017. Select Finish to complete the SRP host creation process as shown

Seite 20

102 Implementing Cisco InfiniBand on IBM BladeCenterThe new SRP host Blade 10 (Linux BoIB) that you just created is now listed under the SRP Hosts fo

Seite 21

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 103To configure Fibre Channel SAN, follow these steps:1. Configure Fibr

Seite 22

104 Implementing Cisco InfiniBand on IBM BladeCenter6.2.5 Configuring storageTo configure storage on the DS4000 storage subsystems, perform the foll

Seite 23

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 1056.2.6 Discovering storageIf the FC zoning, host definition and LUN

Seite 24

x Implementing Cisco InfiniBand on IBM BladeCenterinclude infrastructure design and support, with a special focus on high density data center network

Seite 25

106 Implementing Cisco InfiniBand on IBM BladeCenterFigure 6-13 Default host access policies6.2.8 Port maskingThis section covers port masking on

Seite 26

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 107Figure 6-14 SRP Host: List target devices accessible 3. Double-cli

Seite 27 - 4X InfiniBand networking only

108 Implementing Cisco InfiniBand on IBM BladeCenter4. Click the button shown in Figure 6-15 to open the Select Ports menu that gives the option to

Seite 28

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 1096.2.9 Discovering the boot LUNThis section covers the procedure to

Seite 29 - BladeCenter

110 Implementing Cisco InfiniBand on IBM BladeCenter4. Verify that the Boot LUN ID is 0 by selecting the boot LUN that was added in the preceding ste

Seite 30

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 111The blade boot sequence can be configured from the Advanced Manageme

Seite 31

112 Implementing Cisco InfiniBand on IBM BladeCenter2. Change specific servers by clicking the server name. Figure 6-22 opens, and you can select the

Seite 32

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 1133. Upon power on or restart, the service initializes into the Boot o

Seite 33

114 Implementing Cisco InfiniBand on IBM BladeCenter4. The blade power-on initialization process proceeds to look for a DHCP server using PXE. Upon s

Seite 34

Chapter 6. Boot from InfiniBand using the Cisco 3012 InfiniBand to FC gateway 1158. If there was a previous image installed on the boot disk, then it

Seite 35

Preface xiBecome a published authorJoin us for a two- to six-week residency program! Help write a book dealing with specific products or solutions,

Seite 36

116 Implementing Cisco InfiniBand on IBM BladeCenter10.Figure 6-27 shows the storage configuration and confirms that the Fibre Channel-attached boot

Seite 37

© Copyright IBM Corp. 2007. All rights reserved. 117Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gatewayThis chapter discusses the Cis

Seite 38 - OK (Green) and Fault (Yellow)

118 Implementing Cisco InfiniBand on IBM BladeCenter7.1 Introduction to the Ethernet gateway moduleThe optional InfiniBand to Ethernet gateway modul

Seite 39 - 2.6.2 InfiniBand DDR HCAs

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 119Features and characteristics of the Ethernet gateway module include:򐂰 Two inw

Seite 40

120 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 The 3012 (and thus the Ethernet gateway module) supports four different options for configuring

Seite 41

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1217.1.2 Some general items to considerSome general topics that are important t

Seite 42

122 Implementing Cisco InfiniBand on IBM BladeCenterYou can find more information about understanding and configuring the Ethernet gateway module in

Seite 43

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 123After you have configured these two clients and they can ping each other, you

Seite 44

124 Implementing Cisco InfiniBand on IBM BladeCenterCisco 4948 Ethernet Switch, the 4948 will put any untagged packet on to VLAN 88 inside the 4948).

Seite 45 - InfiniBand technology

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1254. Commit the new Bridge Group information and review (step 4 on page 129).5.

Seite 46 - 3.1.2 Transport layer

xii Implementing Cisco InfiniBand on IBM BladeCenter

Seite 47 - 3.1.4 Data link layer

126 Implementing Cisco InfiniBand on IBM BladeCenterAssuming a fresh installation, Figure 7-6 opens.Figure 7-6 Bridge group page with no bridge gro

Seite 48 - 3.1.5 Physical layer

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 127• Delayed proxy ARP transaction: By default, the delayed proxy ARP transactio

Seite 49 - 12X Link

128 Implementing Cisco InfiniBand on IBM BladeCenter3. After making changes to the Add Bridge Group Groups tab, you need to configure the Forwarding

Seite 50 - 3.2.3 Storage RDMA Protocol

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 129Figure 7-9 Subnet tab on Add Bridge Group windowIn the example in Figure 7-

Seite 51

130 Implementing Cisco InfiniBand on IBM BladeCenter5. The final step on the 3012 is to save this configuration to NVRAM.From the main page of the El

Seite 52 - 3.4 Management

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 131After logging into the 4948 with sufficient privileges, execute the commands

Seite 53 - 3.5 Common markets

132 Implementing Cisco InfiniBand on IBM BladeCenter7.3.5 CLI reference for this sectionThis section includes what the appropriate CLI looks like af

Seite 54 - 3.5.3 Storage area networks

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1337.4.1 Some comments on VLANs and the Ethernet gateway moduleWhen discussing

Seite 55 - QLogic bridges

134 Implementing Cisco InfiniBand on IBM BladeCenter7.4.2 Summary of steps to create a tagged VLAN bridge group designHere, we provide a summary of

Seite 56 - 4.1.1 Installing the drivers

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1352. After you have configured all of the tabs of the Add Bridge Group window,

Seite 57

© Copyright IBM Corp. 2007. All rights reserved. 1Chapter 1. IBM BladeCenter products and technologyBlade servers are thin servers that insert into a

Seite 58

136 Implementing Cisco InfiniBand on IBM BladeCenterAfter logging into the 4948 with sufficient privileges, execute the commands as shown in Example

Seite 59

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 137Example 7-8 is the CLI of the 3012 creating a new BG1 to use VLAN tagging on

Seite 60

138 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 Seven different types of frame distribution algorithms are supported with this product, as show

Seite 61

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 139Configuring aggregation must be done on both sides of the links (in the Ether

Seite 62

140 Implementing Cisco InfiniBand on IBM BladeCenter8. Re-attach or bring up the links between the 3012 and the 4948 (step 8 on page 143).9. Test the

Seite 63

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1414. Click Insert, select a trunk port number to assign (trunk group 1 in this

Seite 64

142 Implementing Cisco InfiniBand on IBM BladeCenter5. After configuring the trunk group (TG1) to use the gateway module Ethernet ports 4/1 and 4/2,

Seite 65

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1437. After the 3012 configuration is saved, you need to configure the 4948. In

Seite 66

144 Implementing Cisco InfiniBand on IBM BladeCentera time) and make sure pings continue to work, regardless of if we have either cable plugged in, o

Seite 67

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 145Example 7-12 is the CLI of the 3012 after creating trunk group TG1 and adding

Seite 68

2 Implementing Cisco InfiniBand on IBM BladeCenter1.1 BladeCenter chassisThere are four chassis in the BladeCenter family:򐂰IBM BladeCenter provides

Seite 69

146 Implementing Cisco InfiniBand on IBM BladeCenter7.6 Implementing a design using gateway redundancyIn many cases, customers want to make sure the

Seite 70

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 147Figure 7-22 represents the test environment for demonstrating gateway redunda

Seite 71

148 Implementing Cisco InfiniBand on IBM BladeCenter6. Create the redundancy group using the two previously created bridge groups (step 6 on page 151

Seite 72

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1494. When the mgmt-ib interface is configured, proceed to create BG1 and BG2.No

Seite 73

150 Implementing Cisco InfiniBand on IBM BladeCenterFigure 7-24 Bridge group BG2 ready to be addedWhen BG1 and BG 2 are added, the Bridging window

Seite 74 - 4.1.5 Hints and tips

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 1515. After BG1 and BG 2 have been created, close the Bridging window and from t

Seite 75

152 Implementing Cisco InfiniBand on IBM BladeCenterClick Apply to create the Redundancy group as shown in Figure 7-28.Figure 7-28 Redundancy Group

Seite 76

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 153! Exit config modeend! Save config to NVRAMwrite9. After the changes have bee

Seite 77

154 Implementing Cisco InfiniBand on IBM BladeCenterNote that we include only the commands that effect the area of the device that we configured. We

Seite 78

Chapter 7. Configuring the Cisco 3012 InfiniBand to Ethernet gateway 155Example 7-19 is the CLI of the 4948 after changes to support this bridge grou

Seite 79

Chapter 1. IBM BladeCenter products and technology 3Figure 1-1 and Figure 1-2 display the front view of an IBM BladeCenter H.Figure 1-1 BladeCenter

Seite 80

156 Implementing Cisco InfiniBand on IBM BladeCenter7.7.2 For the 4948Table 7-3 shows some useful commands for supporting an upstream Cisco IOS devi

Seite 81

© Copyright IBM Corp. 2007. All rights reserved. 157AC alternating currentAMD Advanced Micro Devices™API application programming interfaceARP Address

Seite 82

158 Implementing Cisco InfiniBand on IBM BladeCenterMSI Microsoft InstallerMTU maximum transfer unitNEBS network equipment building systemNFS network

Seite 83

© Copyright IBM Corp. 2007. All rights reserved. 159Related publicationsWe consider the publications that we list in this section particularly suitabl

Seite 84

160 Implementing Cisco InfiniBand on IBM BladeCenterOnline resourcesThese Web sites are also relevant as further information sources:IBM Web sites򐂰 I

Seite 85

4 Implementing Cisco InfiniBand on IBM BladeCenterThe key features on the front of the BladeCenter H include:򐂰 A media tray at the front right, with

Seite 86

Chapter 1. IBM BladeCenter products and technology 5Figure 1-4 Diagram of BladeCenter H rear view showing the key features of the BladeCenter H cha

Seite 88

6 Implementing Cisco InfiniBand on IBM BladeCenterThe benefits of the blade approach will be obvious to anyone tasked with running down hundreds of c

Seite 89 - Figure 4-66 Active SRP Map

Chapter 1. IBM BladeCenter products and technology 7brownouts, rather than shut down completely, or fail, the HS21 XM reduces the processor frequency

Seite 90 - Figure 4-67 Add Direct Map

8 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 Three-year, on-site limited warranty.Table 1-2 provides details on the features of the HS21 XM. T

Seite 91

Chapter 1. IBM BladeCenter products and technology 9Figure 1-6 IBM BladeCenter LS21 Table 1-3 summarizes the features of the LS21.Table 1-3 Featu

Seite 92

10 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 One slot for an I/O expansion adapter, either standard form factor (StFF) or small form factor (

Seite 93 - Figure 4-70

Chapter 1. IBM BladeCenter products and technology 11IBM BladeCenter H chassis has a total of ten I/O bays. Each blade bay has a total of eight dedic

Seite 94

12 Implementing Cisco InfiniBand on IBM BladeCenterThe I/O bays are connected to two separate and redundant midplanes and the blade servers and expan

Seite 95

Chapter 1. IBM BladeCenter products and technology 13For list of supported combinations of high-speed modules and high-speed expansion cards for Blad

Seite 96

14 Implementing Cisco InfiniBand on IBM BladeCenterThe 4X InfiniBand HCAs installed in the blade servers are PCI Express x4 cards with two output por

Seite 97

© Copyright IBM Corp. 2007. All rights reserved. 15Chapter 2. InfiniBand products for BladeCenterThis chapter discusses the InfiniBand products that a

Seite 98

International Technical Support OrganizationImplementing Cisco InfiniBand on IBM BladeCenterOctober 2007REDP-3949-01

Seite 99

16 Implementing Cisco InfiniBand on IBM BladeCenter2.1 Cisco 4X InfiniBand Switch ModuleThe Cisco 4X InfiniBand Switch Module for IBM BladeCenter, p

Seite 100

Chapter 2. InfiniBand products for BladeCenter 17management modules) to facilitate setup and management. After you set up a switch module and bring i

Seite 101

18 Implementing Cisco InfiniBand on IBM BladeCenterYou can manage your InfiniBand switch module with any of the following interfaces:򐂰 Simple Network

Seite 102

Chapter 2. InfiniBand products for BladeCenter 19The Cisco 4X InfiniBand HCA Expansion Card is a High Speed Form Factor (HSFF) card (see Figure 2-3).

Seite 103

20 Implementing Cisco InfiniBand on IBM BladeCenterThe Ethernet Bridge Module offers six 10/100/1000 Mbps external Ethernet RJ45 ports to connect to

Seite 104

Chapter 2. InfiniBand products for BladeCenter 21– Unconnected򐂰 Six external autosensing 10/100/1000 Mbps RJ-45 Ethernet (copper) ports򐂰 802.3ad link

Seite 105 - 5.2.3 Configuring IPoIB

22 Implementing Cisco InfiniBand on IBM BladeCenterthrough Ethernet ports that connect from the bridge module to the BladeCenter H Management Module

Seite 106

Chapter 2. InfiniBand products for BladeCenter 23򐂰 Logical unit number (LUN) mapping and masking򐂰 SCSI-SRP, SCSI-FCP, and FC-PH-3 compliant򐂰 Switched

Seite 107

24 Implementing Cisco InfiniBand on IBM BladeCenter2.6.1 InfiniBand 4X DDR Pass-thru ModuleWith 14 InfiniBand 4X DDR ports towards the servers and 1

Seite 108

Chapter 2. InfiniBand products for BladeCenter 252.6.2 InfiniBand DDR HCAsThree recently added 4X DDR HCAs are also available, resulting from a part

Seite 109 - Cisco 3012 InfiniBand to FC

© Copyright International Business Machines Corporation 2007. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication

Seite 110 - 6.2 Blade booting from a SAN

26 Implementing Cisco InfiniBand on IBM BladeCenterHCAs. However, there are plans to support both the existing Cisco 4X InfiniBand HCA, and the Cisco

Seite 111 - Installing hardware

Chapter 2. InfiniBand products for BladeCenter 27Figure 2-8 Left - 2 port Fibre Channel gateway module, right - 6 port Ethernet gateway moduleThe C

Seite 112

28 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 Load Distribution– Supports redundancy groups across multiple gateways and multiple chassis򐂰 Hig

Seite 113

Chapter 2. InfiniBand products for BladeCenter 29Table 2-4 lists the SFS 3012R InfiniBand switch card specifications.Table 2-4 SFS 3012R InfiniBand

Seite 114 - Figure 6-6 Define SRP host

30 Implementing Cisco InfiniBand on IBM BladeCenterTable 2-6 lists the SFS 3012R Fibre channel gateway module specifications.Table 2-6 SFS 3012R Fi

Seite 115

© Copyright IBM Corp. 2007. All rights reserved. 31Chapter 3. InfiniBand technologyInfiniBand is an industry standard, switch based serial I/O interco

Seite 116

32 Implementing Cisco InfiniBand on IBM BladeCenter3.1 InfiniBand Network Layer ModelInfiniBand uses a multi-layer architecture (similar to the seve

Seite 117

Chapter 3. InfiniBand technology 33InfiniBand specifies multiple transport services for data reliability. Table 3-1 describes each of the supported s

Seite 118 - 6.2.5 Configuring storage

34 Implementing Cisco InfiniBand on IBM BladeCenterAs a packet traverses the subnet, a service level (SL) is defined to ensure its QoS level. Each li

Seite 119 - 6.2.6 Discovering storage

Chapter 3. InfiniBand technology 35Figure 3-2 InfiniBand physical link3.2 InfiniBand protocolsInfiniBand supports a number of upper-layer protocol

Seite 120 - 6.2.8 Port masking

© Copyright IBM Corp. 2007. All rights reserved. iiiContentsNotices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Seite 121

36 Implementing Cisco InfiniBand on IBM BladeCenterenqueue the requested work, and return immediately with the request not completed (often, not even

Seite 122

Chapter 3. InfiniBand technology 37underlying iSCSI-like mechanism. This does not preclude emulating a Fibre Channel connection, while enabling a muc

Seite 123

38 Implementing Cisco InfiniBand on IBM BladeCenter3.3 HardwareThis section describes the hardware used in an InfiniBand configuration:򐂰 Host Channe

Seite 124

Chapter 3. InfiniBand technology 39service and does not follow the same flow control restriction as other VLs on the links. Subnet management informa

Seite 125

40 Implementing Cisco InfiniBand on IBM BladeCenter3.5.1 Application clusteringThe Internet today has evolved into a global infrastructure supportin

Seite 126

© Copyright IBM Corp. 2007. All rights reserved. 41Chapter 4. Configuring InfiniBand with QLogic bridgesIn this chapter, we discuss the steps to conne

Seite 127

42 Implementing Cisco InfiniBand on IBM BladeCenter4.1 Configuring QLogic InfiniBand Bridge Modules with LinuxThis section shows how to install the

Seite 128 - Loading ts_srp_host driver

Chapter 4. Configuring InfiniBand with QLogic bridges 43Figure 4-3 InfiniServ driver install menu6. Follow the driver installation prompts and acce

Seite 129

44 Implementing Cisco InfiniBand on IBM BladeCenterFigure 4-4 Bridge Module Linux Driver installation promptsInstalling IB Network Stack...Adding m

Seite 130

Chapter 4. Configuring InfiniBand with QLogic bridges 457. The installation continues. As shown in Figure 4-5, when prompted to install the SilverSto

Seite 131 - Configuring the Cisco 3012

iv Implementing Cisco InfiniBand on IBM BladeCenter3.5.2 Interprocessor communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Seite 132

46 Implementing Cisco InfiniBand on IBM BladeCenterTo create virtual network interfaces on a Linux blade server, complete the following steps.1. Get

Seite 133

Chapter 4. Configuring InfiniBand with QLogic bridges 47For this setup, we used the settings (config file) shown in Figure 4-10.Figure 4-10 The eio

Seite 134

48 Implementing Cisco InfiniBand on IBM BladeCenter7. Verify your connection works by pinging another device on the same subnet as shown in Figure 4-

Seite 135

Chapter 4. Configuring InfiniBand with QLogic bridges 492. Open the IPoIB config file /etc/sysconfig/ipoib.cfg in a text editor.3. Uncomment the line

Seite 136 - 7.2 Client information

50 Implementing Cisco InfiniBand on IBM BladeCenter6. Check the new IP settings with the ifconfig command as shown in Figure 4-18.Figure 4-18 Netwo

Seite 137 - 7.3.1 The design

Chapter 4. Configuring InfiniBand with QLogic bridges 51To create virtual SRP interfaces on a Linux blade server, complete these steps:1. Each bridge

Seite 138

52 Implementing Cisco InfiniBand on IBM BladeCenter3. Uncomment the lines in the ics_srp.cfg file as shown in Figure 4-20 (at the bottom) and update

Seite 139

Chapter 4. Configuring InfiniBand with QLogic bridges 535. Open the Web interface of your bridge module and click SRP Initiator Discovery, then click

Seite 140

54 Implementing Cisco InfiniBand on IBM BladeCenter8. Each Fibre Channel port on the bridge module has a port world wide name (WWN). On the storage d

Seite 141

Chapter 4. Configuring InfiniBand with QLogic bridges 5510.Click Configure to designate a name for the device as shown in Figure 4-27. This name will

Seite 142

Contents v7.4 Using a VLAN tagged bridge group design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1327.4.1 Some comments on

Seite 143

56 Implementing Cisco InfiniBand on IBM BladeCenterFigure 4-29 Add explicit map14.Select a storage target and identify the host LUN and target LUN.

Seite 144

Chapter 4. Configuring InfiniBand with QLogic bridges 57Figure 4-30 Explicit map configuration15.Restart SRP on the host so that it picks up the ne

Seite 145

58 Implementing Cisco InfiniBand on IBM BladeCenterFigure 4-32 Choose Direct Map19.Select a storage target and click Finish (Figure 4-33).Figure 4-

Seite 146

Chapter 4. Configuring InfiniBand with QLogic bridges 5920.Restart SRP on the host so that it picks up the new mapping. Check for the remote drives u

Seite 147 - Green = Ethernet

60 Implementing Cisco InfiniBand on IBM BladeCenter4.1.5 Hints and tips The following information might help you resolve connectivity issues with yo

Seite 148

Chapter 4. Configuring InfiniBand with QLogic bridges 61򐂰 Restart network services.If the virtual NIC does not come up, restart the linux network ser

Seite 149

62 Implementing Cisco InfiniBand on IBM BladeCenter򐂰 Check the InfiniBand switch Web interface to verify that you have an active connection to the br

Seite 150

Chapter 4. Configuring InfiniBand with QLogic bridges 637. Click Next to continue the driver installation. The Wizard finds and installs InfiniHost M

Seite 151

64 Implementing Cisco InfiniBand on IBM BladeCenter4. Click Next to continue driver installation. When the driver instillation is complete, you see t

Seite 152

Chapter 4. Configuring InfiniBand with QLogic bridges 659. Make an Ethernet connection from the bridge to a test or other network. Inside Network Con

Seite 153 - Int Po5

vi Implementing Cisco InfiniBand on IBM BladeCenter

Seite 154

66 Implementing Cisco InfiniBand on IBM BladeCenterneeded. However without the bridge module, you cannot ping a true Ethernet device—just another Inf

Seite 155

Chapter 4. Configuring InfiniBand with QLogic bridges 676. Set an IP address as you would with any Ethernet adapter and save your settings. See Figur

Seite 156

68 Implementing Cisco InfiniBand on IBM BladeCenter3. Click Next to continue driver installation. When the driver instillation is complete, you see t

Seite 157

Chapter 4. Configuring InfiniBand with QLogic bridges 697. Repeat steps 4 through 6 for the other VFx Controller. There are a total of two controller

Seite 158

70 Implementing Cisco InfiniBand on IBM BladeCenter9. Click Configure to designate a name for the initiator (Figure 4-54). This name will be used lat

Seite 159

Chapter 4. Configuring InfiniBand with QLogic bridges 7112.After you establish the bridge port to remote drive mapping, you need to establish another

Seite 160

72 Implementing Cisco InfiniBand on IBM BladeCenterFigure 4-61 Add Explicit Map17.Select a storage target and identify the host LUN and target LUN,

Seite 161

Chapter 4. Configuring InfiniBand with QLogic bridges 73Figure 4-62 Add Remote DriveFigure 4-63 Completed SRP Map18.Back on the Windows blade, op

Seite 162

74 Implementing Cisco InfiniBand on IBM BladeCenterit. The exclamation point (!) should go away, and your mapping is now active. I/O Controller 1 was

Seite 163

Chapter 4. Configuring InfiniBand with QLogic bridges 7520.On the bridge web GUI, the boxes beside the SRP initiator name and the IOC Map name indica

Seite 164 - Figure 7-25

© Copyright IBM Corp. 2007. All rights reserved. viiNoticesThis information was developed for products and services offered in the U.S.A. IBM may not

Seite 165

76 Implementing Cisco InfiniBand on IBM BladeCenter22.Give the map a name and select Direct type, and then click Next.Figure 4-67 Add Direct Map

Seite 166

Chapter 4. Configuring InfiniBand with QLogic bridges 7722. Select a storage target and click Finish.Figure 4-68 Choose Direct Map

Seite 167

78 Implementing Cisco InfiniBand on IBM BladeCenter23.Back on the windows blade, open device manager. Find the VFx (I/O) Controller that you used to

Seite 168

Chapter 4. Configuring InfiniBand with QLogic bridges 7924.You should now see your additional remote drives in disk management as shown in Figure 4-7

Seite 169 - 7.7 Useful CLI commands

80 Implementing Cisco InfiniBand on IBM BladeCenter25.On the bridge web GUI, the boxes beside the SRP initiator name and the IOC Map name indicate th

Seite 170 - 7.7.2 For the 4948

© Copyright IBM Corp. 2007. All rights reserved. 81Chapter 5. Configuring IP over InfiniBand for use with Cisco switchesThis chapter covers the detail

Seite 171 - Abbreviations and acronyms

82 Implementing Cisco InfiniBand on IBM BladeCenterFigure 5-1 BCH high speed InfiniBand Switch Module GUIThe topics that we discuss in this chapter

Seite 172

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 83The choice of which driver to use will be based on the customers specific req

Seite 173 - Related publications

84 Implementing Cisco InfiniBand on IBM BladeCenter libmthca-1.0.2-1 dapl-devel-1.2.0-1 libibcommon-devel-1.0-1 libipathverbs-devel-1.0-1 opensm

Seite 174 - Help from IBM

Chapter 5. Configuring IP over InfiniBand for use with Cisco switches 85 28:mpitests_mvapich2_gcc ########################################### [ 53%

Kommentare zu diesen Handbüchern

Keine Kommentare