Ammasso 1100 Betriebsanweisung Seite 1

1 The Ammasso 1100 High Performance Ethernet Adapter User’s Guide Ammasso, Inc. 345 Summer St. Boston, MA 02210 Main: 617-532-8

10 1.3 Product Components • AMSO-1100 High Performance Ethernet Adapter • A .tgz archive file containing all software files • The Ammasso 1100 H

11 2 Hardware System Requirements and Installation 2.1 Safety and Emissions This equipment has been tested and found to comply with the limits for

12 2.2 System Hardware Requirements • Intel IA-32 and EM64T or AMD Opteron 32/64 compatible • Bus Type: PCI-X 1.0 • Bus Width: 64-bit • Bus Slo

13 2.3.3 Adapter Insertion Procedure To install the Ammasso 1100 High Performance Ethernet Adapter into a server: 1. Power off the server and unp

14 3 Adapter Software Installation This section provides information on installing the Ammasso adapter software. 3.1 System Software Requirements

15 Enter the AMSO1100 install directory. This is where the commands, libraries, and drivers will be installed. Q2: Where would you like your con

16 3.3.1 Makefile Targets The AMSO1100 package top-level Makefile supports the following targets: all: Configures and builds AMSO1100 (defau

17 3.3.2 Makefile Configuration Variables The configuration file Config.mk has the following variables: GCC_PATH The command name of the compile

18 3.4 Distribution Specific Build Details The following are known issues with building on common distributions. Please refer to the Ammasso suppor

19 For 2.4 Kernels: # make dep For 2.6 Kernels: # make prepare 3.4.2 SuSE The following section lists SuSE distribution specific detai

2 Information in this document is subject to change without notice. This document is provided for information only. Ammasso, Inc. makes no warrant

20 For 2.6 Kernels: # make prepare # make 3.4.3 Kbuild When compiling against 2.6 kernels, the 'kbuild' style Makefiles allo

21 Ammasso 1100 Commands and Utilities Man pages are available for the following commands. A short description is included here. amso_cfg(8) Confi

22 3.5 Configuring the Ammasso 1100 Adapter The Ammasso 1100 configuration file stores the necessary networking information for both the RDMA and L

23 3.5.1 Configuration Entries The rnic_cfg file entries have the following format: function amso_[type]_[rnic]_[instance] { AMSO_IPADDR=[ipad

24 3.5.2 Sample Configuration File NOTE: Since the configuration file is in Bourne shell script syntax, you can use the "#" comment chara

25 # /bin/ping 10.40.48.53 PING 10.40.48.53 (10.40.48.53) 56(84) bytes of data. 64 bytes from 10.40.48.53: icmp_seq=1 ttl=64 time=0.889 ms To v

26 4 The Ammasso MPI Library 4.1 Overview MPICH is a portable implementation of the Message Passing Interface (MPI) standard. Currently the Ammas

27 Q1: Enter the AMSO1100 build path: This is the full path to the AMSO1100 source code. Ammaso’s MPICH implementation needs access to some files

28 By default, Ammasso’s MPICH searches only for a FORTRAN 77 compilation suite. Standard Linux distributions install GNU’s F77 to /usr/bin/f77.

29 all: Configures and builds Ammasso MPICH, this is the default target for make. config: Configures the Ammasso MPICH source tree, this rule cre

3 Table of Contents 1 OVERVIEW 6 1.1 INTRODUCTION 6 1.2 THEORY OF OPERATION 7 1.2.1 HOW THE AMMASSO ADAPTER WORKS 7 1.2.2 HOW REMOTE DIRECT

30 INSTALL_DIR This is the full pathname into which Ammasso MPICH will be installed. It must be an absolute pathname not relative. RSHCOMMAND This

31 The libraries and files associated with using MPICH and the Ammasso 1100 are located in a directory within the Ammasso installation environment:

32 Link it by using the following command (note that in this case, it requires the math library): # mpicc -o cpi cpi.o -lm NOTE: A Makefile is pr

33 NOTE: The maximum number of instances supported on a single node is four. NOTE: When running multiple processes of a program on a single host, t

34 Process 1 on hostB Process 0 on hostA pi is approximately 3.1416009869231241, Error is 0.0000083333333309 wall clock time = 0.000263 4.6 Removi

35 NP is the total number of processes in the MPI run. Assuming each node runs the same number of processes, NP is computed as: NP = (NC * LP)

36 VIADEV_MAX_RENDEZVOUS 209715200 Bytes The maximum amount of user buffer memory that will be locked down at any point in time for zero-copy IO.

37 Each MPI process is allowed to lock down up to VIADEV_MAX_RENDEZVOUS amount of application buffer memory. Once this limit is reached, IO buffers

38 5 The Ammasso DAPL Library 5.1 Overview The Ammasso 1100 DAPL release is composed of one source code tar package. AMSO_DAPL.tgz is the DAPL pa

39 5.2.1 Makefile Targets The DAPL Makefile supports the following targets: all: Configures and builds DAPL (default target). config: Con

4 4.1 OVERVIEW 26 4.1.1 COMPILER SUPPORT 26 4.2 INSTALLATION 26 4.2.1 MAKEFILE TARGETS 28 4.2.2 MAKEFILE CONFIGURATION VARIABLES 29 4.3 L

40 KERNEL_SOURCE The pathname to the kernel source tree for the kernel you are running. O Path to alternate kernel build output directory. 5.3 Co

41 DT_cs_Client: ========== End of Work -- Client Exiting ... This test will take a few minutes to complete. 5.4.2 kDAPL Installation Verificat

42 on Infiniband DAPL providers that are ported to the Ammasso DAPL provider. The solution is to explicitly set local read access on memory regions

43 6 Cluster Installation 6.1 Introduction The purpose of this chapter is to provide a sample install session for the Ammasso 1100 adapter softwar

44 6.2.2 Install AMSO1100 and Build Binary for Cluster Deployment 1. Untar the AMSO1100 package on the build system. The unzip/tar-extract for th

45 /etc/init.d/rc5.d/S06amso1100 /etc/init.d/rc5.d/K16amso1100 /etc/init.d/amso1100 # ls /etc/ammasso.conf /etc/ammasso.conf # cat /etc/ammasso.

46 6.2.3 Install MPICH and Build Binary for Cluster Deployment 1. Use the tar(1) command to unarchive the MPICH package on the build system. Th

47 INSTALL_DIR='/usr/opt/ammasso/mpich-1.2.5' RSHCOMMAND='/usr/bin/rsh' ENABLE_SHAREDLIB='no' F90='' F90FLAG

48 Overwrite existing mpich installation (y or n)? [no] y ... output of install ... Created binary self-extracting image --> /tmp/AMSO_MPI

49 3. Build an AMSO_DAPL binary package for install on the cluster nodes. # ls . .. Build Config Installed Makefile dapl-1.2 dapl_cset e

5 7.4.3 POPULATING THE RAMDISK 57 7.4.4 BUILDING THE RAMDISK IMAGE 59 7.5 INSTALLING AND CONFIGURING A DHCP SERVER 59 7.6 INSTALLING AND CONF

50 # /tmp/ammasso1100.bin –d /usr/opt/ammasso/data/app64-02 Ammasso 1100 binary installer created on ... Using /usr/opt/ammasso/data/app64-02 as

51 5. On each cluster node, install the MPICH software. This step needs to be done as the root user. Copy or transfer the image-mpich-1.2.5.bin s

52 AMSO1100, MPICH, and DAPL are now installed on the cluster node system. Repeat the above steps for each node in the cluster. Set RDMA and CCILN

53 7 Using the Ammasso 1100 with PXE Boot PXE boot is a way to boot and load x86 based servers with a system image (or any other type of program)

54 1. A target machine that is PXE capable. The Ammasso 1100 card must be selected as the network boot device. 2. The Ammasso driver, ccil.{o,ko}

55 5. A system configured to be a PXE boot server. This system must be running DHCP (dynamic host configuration protocol), TFTP (trivial file tran

56 To simplify building, all applications are shown to be linked statically. For the next few steps we assume that the shell variables, BUSYBOX an

57 7.4.3 Populating the Ramdisk Create an initrd directory structure to be used for the ramdisk. For this example, the initrd directory structure

58 insmod /modules/net/ccil.o Example 3 [FC 3, 2.6.9-1.667smp]: # load the drivers insmod /modules/net/sunrpc.ko insmod /modules/net/lockd.ko insmo

59 # Display the routing table route -n # Mount what will become the root filesystem mount -t nfs -o nolock,rsize=8192,wsize=8192,hard,intr ${NFSD

6 1 Overview 1.1 Introduction The Ammasso 1100 High Performance Ethernet Adapter is a RDMA enabled Ethernet Server adapter offering high performan

60 There needs to be one entry for each MAC address in the "group" section of the file. A sample dhcpd.conf configuration file is shown h

61 7.7 Building PXELINUX In this example, we use PXE to boot Linux with PXELINUX from the syslinux package. This can be obtained from the followi

62 7.8.1 Configure a Root File System The idea behind a diskless node is to use the distribution that has been installed on a remote server via NFS

63 7.9 Updating the Ammasso 1100 Option ROM Image The Ammasso 1100 network adapter was factory flashed with an option ROM image. If you are updat

64 Appendix A: Support Obtaining Additional Information Additional information, the latest software revisions, and documentation is always availab

65 Appendix B: Warranty Ammasso High Performance Server Adapter LIMITED LIFETIME HARDWARE WARRANTY Ammasso warrants to the original owner tha

66 AMMASSO'S TOTAL LIABILITY FOR ALL CLAIMS UNDER THIS AGREEMENT SHALL NOT EXCEED THE PRICE PAID FOR THE PRODUCT. THESE LIMITATIONS ON POTENTIA

7 1.2 Theory of Operation 1.2.1 How the Ammasso Adapter Works The Ammasso 1100 is an Ethernet adapter designed to take advantage of various standa

8 Figure 1 Transfer of Data RDMA reduces demand on the host CPU by enabling applications to directly issue data transfer request commands to the a

9 Figure 2 RDMA vs. Traditional Adapter 1.2.3 What Is MPI and Why Is It Used? The Message Passing Interface (MPI) is a library for message-passin