Wind River Systems, Inc. 1997

Appendix - EAppendix - E

Shared Memory Network

E-2

Shared Memory Network Overview

E-3

Shared Memory Network Implementation

Uses global (“shared”) memory accessible to all CPU’s on the backplane.

Uses test-and-set atomic operation to mutually exclude access to global data structures.

Destination CPU notified of packet arrival via:Mailbox interrupts (most flexible).VMEbus interrupts (limited number).Polling (if interrupts not available).

E-4

Terminology

Shared Memory Master - CPU 0. Initialize the shared memory network. Once initialized, all boards are peers.

Shared Memory - Memory used for passing packets between processors. Can reside on CPU0 or a memory card.

Anchor - Structure containing ready value and pointer to shared memory. Must be at a known address.

Heartbeat - Integer incremented once per second by CPU0, used to indicate that the shared memory network is alive.

Ready Value - Indicates if the shared memory network is initialized.

E-5

Choosing a Network Number

Shared memory network number options:Distinct network In an isolated environment, choose

any unused network number.

Subnetwork If the host is already on a subnet, choose an unused subnet.

Proxy network Use the same network number as the host. Easiest to configure.

Examples:

Relationship Host network SM networkDistinct 90.0.0.0 100.0.0.0Subnet 90.1.0.0:ffff0000 90.2.0.0:ffff0000Proxy 90.1.0.0:ffff0000 90.1.0.0:ffff0000

E-6

Sequential Addressing

The internet address of CPU0 on the SM network:If INCLUDE_PROXY_DEFAULT_ADDR is defined, it is one more than inet on ethernet (e).Otherwise, assigned in the boot parameters.

The internet address of other boards on the SM network:If INCLUDE_SM_SEQ_ADDR is defined, will be sequentially ordered (by processor number). Otherwise, assigned in the boot parameters.

E-7

Example Configuration Overview

1. Configure the host’s network softwre:Modify the host table.On UNIX, modify .rhosts. (UNIX Host)

2. ConfigAll.h specifies as defaults:shared memory network supportsequential addressingproxy client support

3. Modify config.h for CPU0 to specify:proxy server supportproxy default addressing.where shared memory resides, how large it is.

4. Calculate the address of the anchor.

E-8

Example Configuration

E-9

UNIX: Host Configuration

Configure host so that targets can boot:Host tableRouting (if necessary -- when proxy ARP not used)Permissions

Example/etc/hosts file:90.0.0.49 vxEth090.0.0.50 vxSm090.0.0.51 vxSm1

Example ~/.rhosts file (if using RSH):vxEth0vxSm0vxSm1

E-10

Installing the Shared Memory Network

These defaults are specified in configAll.h:

#define INCLUDE_PROXY_CLEINT...#define INCLUDE_SM_NET#define INCLUDE_SM_SEQ_ADDR...#define SM_TAS_TYPE SM_TAS_HARD#define SM_ANCHOR_ADDR ... /* arch. specific */

Modify CPU0’s config.h file to specify:

#define INCLUDE_PROXY_SERVER#define INCLUDE_PROXY_DEFAULT_ADDR#define SM_OFF_BOARD FALSE#define SM_MEM_ADRS ...#define SM_MEM_SIZE ...

E-11

Shared Memory Location / Size

Example wind/target/config/target/config.h stub:

#define SM_OFF_BOARD FALSE#if SM_OFF_BOARD#undef SM_ANCHOR_ADRS#define SM_ANCHOR_ADRS ((char *) 0xfb800000)#define SM_MEM_ADRS SM_ANCHOR_ADRS#define SM_MEM_SIZE 0X80000 /* 512k */#else #define SM_MEM_ADRS NONE /* NONE = ALLOCATE */#define SM_MEM_SIZE 0X10000 /* 64k */#endif /* SM_OFF_BOARD */

E-12

CPU0 Boot Parameters

boot device : inprocessor number : 0host name : ohiofile name : wind/target/config/frc40/vxWorksinet on ethernet (e) : 90.0.0.49inet on backplane (b):host inet (h) : 90.0.0.2user (u) : marc

E-13

CPU0 Booting Example

...Attaching network interface ln0... done.Attaching network interface lo0... done.Loading... 393496 + 102988 + 25610Starting at 0x1000...

Attaching network interface ln0... done.Initializing backplane net with anchor at 0x600...done.Backplane anchor at 0x600... Attaching networkinterface sm0... done.Backplane address: 90.0.0.50

Creating proxy network: 90.0.0.50...

E-14

Booting Other CPU’s

Other boards must be able to access the anchor, through the VMEbus, in order to boot.

CPU’s local anchor address (0x600) is not identical to its corresponding VMEbus address.

To attach to the Shared Memory Network, other CPU’s must:1. Determine the VMEbus address of the anchor (how CPU0

dual-ports its local RAM).2. Determine the local address, on CPU1, which is used to

access the anchor’s VMEbus address (how CPU1 maps its address space).

E-15

VMEbus Address Spaces

The VMEbus provides several address spaces to accommodate varying needs.

Address space is specified by an address modifier, which encodes:Address size 32 bit (extended), 24 bit(standard), or 16 bit (short I/O).

Privilege level User or supervisor. VxWorks always uses supervisor mode.

Access type Data, instruction, or block transfer (DMA).

Symbolic constants for address modifiers defined in vme.h (e.g., VME_AM_EXT_SUP_DATA).

E-16

Off Board Memory Access

E-17

Calculating the VMEbus Address ofthe Anchor from CPU0

-> busAdrs = 0new symbol “busAdrs” added to symbol table.value = 0 = 0x0

-> sysLocalToBusAdrs (0xd, 0x600, &busAdrs)value = 0 = 0x0

-> busAdrsbusAdrs = 0xffe7d0: value = 16778752 = 0x1000600value = 0 = 0x0

E-18

Local Address of Anchor As Seen from Other Boards

-> localAdrs = 0new symbol “localAdrs” added to symbol table.localAdrs = 0x3fe7d0: value = 0 = 0x0

-> sysBusToLocalAdrs (0xd, 0x1000600, &localAdrs)value = 0 = 0x0

-> localAdrslocalAdrs = 0x3fe7d0: value = 16778752 = 0x1000600

Can’t perform these steps from CPU1, since not yet booted.Can perform from CPU0 if boards are the same.Can examine sysLib source code otherwise.

E-19

Calculating the Anchor

1. Call sysLocalToBusAdrs( ) on CPU0 to calculate the VMEbus address of the anchor.

2. Call sysBusToLocalAdrs( ) on the other CPU boards to calculate the local address which maps to the anchor’s VMEbus address. May need to examine source code for this routine.

E-20

CPU1 Boot Parameters

boot device : sm=0x1000600processor number : 1host name : ohiofile name : wind/target/config/frc40/vxWorksinet on backplane (b) : host inet (h) : 90.0.0.2gateway inet (g) : user (u) : marcflags (f) : 0x100

E-21

Booting CPU1

...Backplane anchor at 0x1000600... Attaching networkinterface sm0... done.Backplane inet address: 90.0.0.51registering proxy client: 90.0.0.51.doneAttaching network interface lo0... done.Loading ... 393496 + 102988 + 25610Starting at 0x1000......

E-22

Other Configurations

If not using a proxy network:Must modify the host’s routing table.Slaves must specify gateway inet (g) boot parameter.

Can use BOOTP with the proxy network.

See the Programmer’s Guide for details.

E-23

Avoiding Bus Errors

Default bus arbitration favors boards close to first slot.

Distant boards may experience bus (timeout) errors while booting.

Possible solutions:Stagger the boot countdown for each board by modifying wind/target/config/all/bootConfig.c.Increase bus timeout value.Modify the bus arbitration scheme (software changes are typically made in sysProcNumSet( ) and sysHwInit( )).

Software solutions require making new boot ROM’s which is discussed in the Reconfiguration chapter.

E-24

A Troubleshooting Tool

-> smNetShowAnchor Local Addr: 0x1000600Sequential addressing enabled. Master address:90.0.0.50

heartbeat = 6712, header at 0x1fb8144, free pkts = 29

cpu int type arg1 arg2 arg3 queued pkts----- -------- ---- ----- ------ ---------------0 mbox-1r 0x2d 0x8002 0 01 mbox-1r 0x2d 0x8102 0 0

input packets = 109 output packets = 109input errors = 0 output errors 0collisions = 0value = 0 = 0x0

E-25

Summary

Host configuration:Modify host tables.Modify ~/.rhosts. (UNIX Host)Modify routing table if not using a proxy network.

Defaults in configAll.h:INCLUDE_SM_NET and INCLUDE_SM_SEQ_ADDRSM_ANCHOR_ADRSINCLUDE_PROXY_CLIENT

CPU0 (the shared memory master) specifiesSM_OFF_BOARDSM_MEM_SIZE and SM_MEM_ADRSINCLUDE_PROXY_SERVERINCLUDE_PROXY_DEFAULT_ADDR

E-26

Summary

Additional boot parameter for CPU0:inet on backplane (b):If using default proxy address, leave blank.

Accessing the anchor from other boards:sysLocalToBusAdrs( )sysBusToLocalAdrs( )

Additional boot parameters for CPU1:boot device: sm=localAdrsOfAnchorinet on backplane (b). If using sequential addresses, leave blank.gateway inet(g). If using proxy ARP, leave blank.register proxy client boot flag (0x100)