device driver for generic asc module project presentation by: yigal korman erez fuchs instructor:...

Device Driver for Device Driver for Generic ASC ModuleGeneric ASC Module

Project Project PresentationPresentation

By:By:Yigal KormanYigal KormanErez FuchsErez Fuchs

Instructor:Instructor:Evgeny Evgeny

FiksmanFiksman

Sponsored by:Sponsored by:

High Speed Digital High Speed Digital Systems LabSystems Lab

Parallel Systems Parallel Systems LabLab

Table of ContentsTable of Contents

AbstractAbstract ASC – A Stream CompilerASC – A Stream Compiler Project GoalProject Goal Development PlatformDevelopment Platform

– Hardware PlatformHardware Platform– Software PlatformSoftware Platform

Project Research & DevelopmentProject Research & Development– Part A: Hardware SidePart A: Hardware Side– Part B: Software SidePart B: Software Side

AbstractAbstract

ProblemProblem– There are many complex functions which require a There are many complex functions which require a

lot of CPU resourceslot of CPU resources– Find the best way to implement these functions Find the best way to implement these functions

under the constrains of resources and costunder the constrains of resources and cost Possible SolutionsPossible Solutions

– Pure software implementationPure software implementation Low costLow cost Low performanceLow performance

– Pure hardware implementationPure hardware implementation High costHigh cost High performanceHigh performance

– Combination of software & hardwareCombination of software & hardware ASC technologyASC technology Fine balance between cost and performanceFine balance between cost and performance

Vision - Future Vision - Future WorkflowWorkflow

1.1. Write a conventional software Write a conventional software implementationimplementation

2.2. Locate critical code sectionsLocate critical code sections3.3. Convert these sections into hardware Convert these sections into hardware

implementationimplementation4.4. Compile the software program and create a Compile the software program and create a

hardware specific netlisthardware specific netlist5.5. Communication between the software and Communication between the software and

the hardware will be automatically added the hardware will be automatically added during compilationduring compilation

6.6. Load the netlist into the FPGA and execute Load the netlist into the FPGA and execute the program alongsidethe program alongside

combining hardware & software combining hardware & software designdesign


AbstractAbstract ASC – A Stream CompilerASC – A Stream Compiler Project GoalProject Goal Development PlatformDevelopment Platform



ASC – A Stream ASC – A Stream CompilerCompiler Combinational (SW/HW) codeCombinational (SW/HW) code

– Write the software code in C/C++Write the software code in C/C++– Add hardware sections in Add hardware sections in

between using C++ codebetween using C++ code– Hardware is described by special Hardware is described by special

C++ librariesC++ libraries Hardware is complied into Hardware is complied into

standard NetList output (.edif)standard NetList output (.edif)– Supported by standard CAD toolsSupported by standard CAD tools– Supported by Xilinx FPGA Supported by Xilinx FPGA

architecturesarchitectures Provides HW optimizationProvides HW optimization Internal Design

ASC Code ExampleASC Code Example

#include "asc.h"

main(int argc, char **argv) {

printf("Hello World\n");

STREAM_START; // ASC code start

// Hardware Variable Declarations

HWint a(IN, 32),b(OUT, 32);

STREAM_LOOP(SIZE);

b = a + 1;

STREAM_END; // ASC code end

}

Software

Hardware

ASC – Not Everything ASC – Not Everything Is PerfectIs Perfect

No automatic addition of communication No automatic addition of communication between software and hardwarebetween software and hardware

ASC creates the hardware Netlist and the ASC creates the hardware Netlist and the software program but doesn’t provide an software program but doesn’t provide an interface for communication between theminterface for communication between them

The designer is left with the difficult task of The designer is left with the difficult task of implementing an interconnection between implementing an interconnection between the HW & SW and integrating it into the the HW & SW and integrating it into the designdesign

This is where we come in play...This is where we come in play...


AbstractAbstract ASC – A Stream CompilerASC – A Stream Compiler Project GoalsProject Goals Development PlatformDevelopment Platform



Project Goal: Complete Project Goal: Complete The EquationThe Equation

Create a generic communication Create a generic communication system between the software side system between the software side and the hardware sideand the hardware side– Demonstrate a working systemDemonstrate a working system

Embedded OS

Software

Hardware System(FPGA)

Hardware Logic (ASC)

Stream Driver

Device Driver for Generic Device Driver for Generic ASC Module: Stream ASC Module: Stream DriverDriver Define a simple generic dual-sided Define a simple generic dual-sided

interface through which the hardware and interface through which the hardware and the software will communicatethe software will communicate

Implement the interconnect device driverImplement the interconnect device driver– The hardware will use specific ports as The hardware will use specific ports as

inputs/outputsinputs/outputs– The software will use specific function calls to The software will use specific function calls to

send data into and out of the hardware coresend data into and out of the hardware core– The device driver will manage the transport of The device driver will manage the transport of

data in between and all other communicationdata in between and all other communication

Hardware PlatformHardware Platform

Xilinx ML310 Xilinx ML310 development boarddevelopment board– Based on the Virtex-II Based on the Virtex-II

Pro FPGA chip which Pro FPGA chip which includes a versatile includes a versatile programmable logic programmable logic array and 2 PowerPC array and 2 PowerPC CPUsCPUs

– Includes all the Includes all the current hardware current hardware components found on components found on computer systems: computer systems: IDE, NIC, USB, SDRAM, IDE, NIC, USB, SDRAM, PCI and more.PCI and more.

Xilinx ML310 Development Board

Hardware ToolsHardware Tools

Xilinx Platform Studio 6.3Xilinx Platform Studio 6.3– IDE for creating programmable IDE for creating programmable

platform designsplatform designs– Runs library generation, compiler tool Runs library generation, compiler tool

chains and linker script generationchains and linker script generation– Creates implementation and Creates implementation and

simulation netlistssimulation netlists– Development, debug and verification Development, debug and verification

toolstools– Board support package for the ML310 Board support package for the ML310

Software PlatformSoftware Platform

MontaVista Embedded LinuxMontaVista Embedded Linux– Based on open source Linux kernel Based on open source Linux kernel

2.4.x2.4.x– Independent file systemIndependent file system– Can develop hardware specific Can develop hardware specific

drivers for the operating systemdrivers for the operating system– Supports the Xilinx ML310 boardSupports the Xilinx ML310 board: :

Device drivers written specifically for Device drivers written specifically for ML310 peripheralsML310 peripherals

Software ToolsSoftware Tools

MontaVista DevRocket 1.1MontaVista DevRocket 1.1– Eclipse-based IDE to write code Eclipse-based IDE to write code

projects in C/C++ for the MV projects in C/C++ for the MV Embedded LinuxEmbedded Linux

– Configure target settingsConfigure target settings– Develop & debug platform codeDevelop & debug platform code– Create deployable platform imagesCreate deployable platform images– Includes a cross-compiler for the Includes a cross-compiler for the

PowerPC CPUs on the ML310 boardPowerPC CPUs on the ML310 board


AbstractAbstract ASC – A Stream CompilerASC – A Stream Compiler Project GoalsProject Goals Development Platform



Project Goals – Part A: Project Goals – Part A: Hardware sideHardware side

Study the Xilinx development platformStudy the Xilinx development platform Study the use of interrupts & DMA with Study the use of interrupts & DMA with

IP coreIP core Design a communication protocol Design a communication protocol

between the ASC IP core and the between the ASC IP core and the softwaresoftware

Implement an example for direct Implement an example for direct communication between a specific IP communication between a specific IP core and its software wrapper running core and its software wrapper running on the CPU (non OS)on the CPU (non OS)

Hardware Interface:Hardware Interface:ASC BackendASC Backend

ASC couples the hardware logic it ASC couples the hardware logic it creates with a generic backend:creates with a generic backend:– Provides a way to communicate with Provides a way to communicate with

the logicthe logic Read and Write FIFOs for transferring data Read and Write FIFOs for transferring data

into and from the logicinto and from the logic Data registers for internal status queries as Data registers for internal status queries as

well as fine tuning the logicwell as fine tuning the logic We will create a driver that will We will create a driver that will

communicate with this backendcommunicate with this backend

Hardware Interface:Hardware Interface:ASC BackendASC BackendContinued…Continued…

Read FIFO

Write FIFO

ConfigurationRegisters

ASC Backend

Clock

Reset

PipeFlushing

Flush

Data In

Data Out

Registers Access

ASC

Stream

Processing

Pipe

ASC – Pipe WorkflowASC – Pipe Workflow

Data is inserted Data is inserted into the Write into the Write FIFO and pushed FIFO and pushed through the through the Processing PipeProcessing Pipe

The result of the The result of the calculations is calculations is pushed from the pushed from the pipe to the Read pipe to the Read FIFO in a cyclic FIFO in a cyclic orderorder

Garbage data is Garbage data is inserted to the inserted to the pipe during flush pipe during flush operationoperation

Processing Pipe Read FIFOWrite FIFO

# #

garbage data


5 3 2 # #


5 3 4# #


2594* * # #


2594* *11 10 7 # #

flush


11 10

data overwritten

25949* *

Part A - InvestigatePart A - Investigate

PLB IPIF CorePLB IPIF Core– Provides a bi-directional interface between a Provides a bi-directional interface between a

user IP core and the PLB 64-bit bus standarduser IP core and the PLB 64-bit bus standard– Supports various services and features that Supports various services and features that

can be optioned in or out:can be optioned in or out: Local IP interrupt connection with user softwareLocal IP interrupt connection with user software Programmable enables/disablesProgrammable enables/disables User software triggered reset generator for User software triggered reset generator for

localized reset of user’s corelocalized reset of user’s core User configured RdFIFO and WrFIFO with optional User configured RdFIFO and WrFIFO with optional

IP packet supportIP packet support DMA function with optional Scatter/Gather DMA function with optional Scatter/Gather

mechanizationmechanization Configurable user address ranges, which can be Configurable user address ranges, which can be

used to directly address the user IP on the PLB busused to directly address the user IP on the PLB bus

Part A – Hardware Part A – Hardware DesignDesignConnecting the ASC to the PLB BusConnecting the ASC to the PLB Bus

Use IPIF facilities to Use IPIF facilities to implement the implement the hardware interface hardware interface to the ASCto the ASC– DMA controller for DMA controller for

data transferdata transfer– Interrupt lines for Interrupt lines for

asynchronous asynchronous transferstransfers

– Reset LinesReset Lines The ASC core will The ASC core will

connect to the IPIF connect to the IPIF and be accessed and be accessed through the user through the user addressable addressable memory the IPIF memory the IPIF providesprovides

Use PLB clock Use PLB clock signal + clock signal + clock divider for the ASC divider for the ASC stream clockstream clock

PL

B B

us

PLB IPIF ASC Backend

DMA/Scatter Gather

Master Attachment

Slave Attachment

ByteSteering

Slave Reply

PLB Requests

Master PLBRequests

PLB MasterReply

TransferRequest

RequestStatus

Interrupt

Device ISC

IPIF Interrupts

Device Interrupt

S/W Reset

Reset

ClockDivider

PLB Clock

ASC Clock

Load TransferRequest

Load TransferReply

MasterRequestArbiter

PLB to IP Data

IP to PLB Data

PLB to IP Address

IP Reset

PLB to IP Rd/Wr Request

Part A – Hardware Part A – Hardware Design Design Data FlowData Flow

PLB

Bus

PPC 405CPU

PLBIPIF

InterruptController

ASCLogic

ASC

Back

end

R/WFIFOs

DMA InterruptSupport

MemoryData to ASC

Data to Memory

Set DMA

DMA Done

Part A – Hardware TestPart A – Hardware Test

Configure and test the hardware:Configure and test the hardware:– Load ML310 with default Load ML310 with default

configurationconfiguration SDRAM support – large memory SDRAM support – large memory

capacity capacity Local Area Network support – data Local Area Network support – data

transfer between development transfer between development environment and target board via FTP environment and target board via FTP (for software development part)(for software development part)

PCI support – for future extensionPCI support – for future extension

– Add a simple ASC core to the systemAdd a simple ASC core to the system

Part A – Hardware Test Part A – Hardware Test Continued…Continued…

– Write a simple test to transfer data between Write a simple test to transfer data between the OS memory and ASC corethe OS memory and ASC core

– Use Xilinx standalone OS API and code Use Xilinx standalone OS API and code examples: examples:

Data IOData IO DMA TransferDMA Transfer Interrupt handlingInterrupt handling

Check that the underlying hardware Check that the underlying hardware works correctlyworks correctly

The test program will provide a The test program will provide a conceptual foundation for the driverconceptual foundation for the driver

Part A – Hardware Test Part A – Hardware Test Flow DiagramFlow Diagram

Start

• Prepare DMA Transaction• DMA write data to ASC

Initialization: • Init DMA status• Init IPIF coreEnable Interrupts:• DMA interrupts on IPIF• CPU exceptions• Interrupt controller• Register interrupt handler in interrupt controller

Wait on DMA interrupt

Part A – Hardware Test Part A – Hardware Test Flow Diagram Flow Diagram Continued…Continued…

• Prepare DMA Transaction• DMA read data from ASC

• ACK DMA interrupt handling• Return transaction status

Finish

Check read data

Wait on DMA interrupt

DMA Done?

• ACK DMA interrupt handling• Return transaction status

DMA Done?

Yes

Yes

No

No

Error

Interrupt Handler

Interrupt Handler

Project Goals – Part B: Project Goals – Part B: Software sideSoftware side Hardware is ready and functionalHardware is ready and functional Install LinuxInstall Linux Study the MontaVista Study the MontaVista

development environmentdevelopment environment Study writing Linux device driversStudy writing Linux device drivers Design device driver specificationDesign device driver specification Write the device driverWrite the device driver

Design: A Stream Design: A Stream DriverDriver The driver’s functionality is to The driver’s functionality is to

communicate with the ASC backendcommunicate with the ASC backend The ASC core always work as a The ASC core always work as a

stream hardware – a pipeline of data stream hardware – a pipeline of data is streamed through the core – being is streamed through the core – being evaluated, calculated and outputtedevaluated, calculated and outputted

The driver will allow flexibility to the The driver will allow flexibility to the user and comply with Linux driver user and comply with Linux driver standardsstandards

Device Driver: Module Device Driver: Module vs. Kernelvs. Kernel

KernelKernelModuleModulePreloaded on startupPreloaded on startupNeeds to be manually Needs to be manually

loaded before useloaded before use

Recompile whole kernel Recompile whole kernel on changeon change

Recompile only module Recompile only module on changeon change

Active and resource Active and resource hogging even when hogging even when unneededunneeded

Active only when Active only when neededneeded

Our choice: implement the driver as Our choice: implement the driver as MODULEMODULE

Linux Device Driver Linux Device Driver System StackSystem Stack

Custom & Third Party Applications

Middleware & Application Services

MontaVista Graphics

Advanced Embedded Features Networking &ApplicationPackages

MontaVista Linux Kernel

Device Drivers

Reference Hardware

Real-timeFunctionality

RobustSecurity

High PerformanceNetworking

HighReliability

ASC StreamDriver

ASCModule

TestApplication

The Stream Driver The Stream Driver FeaturesFeatures Read & Write to ASC FIFOsRead & Write to ASC FIFOs ASC Register accessASC Register access Fast asynchronous data transfers Fast asynchronous data transfers

(Interrupts & DMA)(Interrupts & DMA) ASC pipe flush & garbage avoidanceASC pipe flush & garbage avoidance Status informationStatus information Mapping hardware memory to user Mapping hardware memory to user

spacespace Support of multiple ASC modulesSupport of multiple ASC modules

Stream Driver Stream Driver WorkflowWorkflow

Load the device driver Load the device driver modulemodule

Create a file in the file Create a file in the file system to represent the system to represent the device (ASC core)device (ASC core)

Open the device fileOpen the device file Configure the driver Configure the driver

with device parameterswith device parameters Write data into the Write data into the

devicedevice Flush processing pipe to Flush processing pipe to

receive resultsreceive results Read results from the Read results from the

devicedevice Close the device fileClose the device file

# # insmod stream_driver.oinsmod stream_driver.o

This will load the device This will load the device driver into the kernel driver into the kernel and will allow users to and will allow users to use the driver’s use the driver’s functionalityfunctionality

The driver has a unique The driver has a unique hard-coded code (major hard-coded code (major number) that identifies number) that identifies device files that should device files that should use this driver (our use this driver (our number is 120)number is 120)

The driver can be used The driver can be used simultaneously with simultaneously with more than one devicemore than one device

The best way to explain something is by example


# # mknod /dev/stream_device0 c 120 0mknod /dev/stream_device0 c 120 0

This create a single This create a single device file that is called device file that is called ‘stream_drevice0’ and ‘stream_drevice0’ and will connect it to our will connect it to our driver using the major driver using the major number variable (120)number variable (120)

More that one file can More that one file can be created with different be created with different name and different name and different minor number (0) to minor number (0) to represent more than represent more than one ASC core in the one ASC core in the systemsystem
















device_file = open(‘/dev/stream_device0’, O_RDWR);device_file = open(‘/dev/stream_device0’, O_RDWR);

This is part of a c program This is part of a c program using the Linux kernel APIusing the Linux kernel API

Supply the location of the Supply the location of the file and the mode in which file and the mode in which to operate – usually read to operate – usually read & write (O_RDWR)& write (O_RDWR)

This will load the driver This will load the driver initialization code in the initialization code in the backgroundbackground

Our driver will not fully Our driver will not fully initialize until some initialize until some configuration information configuration information will be passed so it will be will be passed so it will be able to correctly identify able to correctly identify the devicethe device









ioctl(device_file, STREAM_IOCTL_CONFIG, &device_conf);ioctl(device_file, STREAM_IOCTL_CONFIG, &device_conf);

Use a special function to Use a special function to communicate with the communicate with the driver (ioctl) pass the driver (ioctl) pass the device file (device_file) device file (device_file) and command and command (STREAM_IOCTL_CONFIG(STREAM_IOCTL_CONFIG))

The last argument is the The last argument is the configuration info itself:configuration info itself:struct stream_config device_conf = { module_base_address: XPAR_TEST_PLB_CORE_0_BASEADDR, module_high_address: XPAR_TEST_PLB_CORE_0_HIGHADDR, dma_channels_offset: PLB_STREAM_IPIF_DMA_SG_SPACE_OFFSET, irq: XPAR_OPB_INTC_0_TEST_PLB_CORE_0_IP2INTC_IRPT_INTR, stream_base_address: XPAR_TEST_PLB_CORE_0_AR0_BASEADDR, stream_high_address: XPAR_TEST_PLB_CORE_0_AR0_HIGHADDR, stream_num_inputs: NUM_INS, stream_num_outputs: NUM_OUTS, stream_letancy: STREAM_LATENCY, stream_cycle: STREAM_CYCLE,};









write(device_file, data_in, sizeof(data_in));write(device_file, data_in, sizeof(data_in));

Use the write command Use the write command to pass a data buffer to pass a data buffer (data_in) to the device(data_in) to the device

This command is This command is blocking – it will lock the blocking – it will lock the program until the program until the transfer is complete BUT transfer is complete BUT it will not lock the it will not lock the system because we use system because we use interrupts to notify the interrupts to notify the system when the system when the transfer is finishedtransfer is finished









fsync(device_file);fsync(device_file);

This command requests This command requests the driver to activate the the driver to activate the flush mechanism in the flush mechanism in the ASC module. The driver ASC module. The driver also calculates the also calculates the location and quantity of location and quantity of garbage data inserted garbage data inserted during the flushduring the flush

Reading is as simple as Reading is as simple as writing – the user has no writing – the user has no need to locate the need to locate the correct position in the correct position in the read FIFO – all done read FIFO – all done automatically in the automatically in the driverdriver

read(device_file, data_out, sizeof(data_out));read(device_file, data_out, sizeof(data_out));









close(device_file);close(device_file);

Always close the device Always close the device after use. This will clear after use. This will clear valuable resources valuable resources occupied by the driveroccupied by the driver

You can also unload the You can also unload the driver from the kernel withdriver from the kernel with

There are more operations There are more operations in the driver such as direct in the driver such as direct memory allocation memory allocation (mmap), register access (mmap), register access (STREAM_IOCTL_REGISTER(STREAM_IOCTL_REGISTER_READ/WRITE) and others _READ/WRITE) and others – see project book for – see project book for more detail more detail

# rmmod stream_driver# rmmod stream_driver

DemonstrationDemonstration

Load DriverLoad Driver Run Test Run Test

ApplicationApplication Test ResultsTest Results Unload DriverUnload Driver

The FutureThe Future

Loading Custom Hardware during Loading Custom Hardware during runtime – no need to boot and runtime – no need to boot and reload the netlistsreload the netlists– All you will need is to tell the driver All you will need is to tell the driver

the location of the ASC module the location of the ASC module hardware implementation and it will hardware implementation and it will load it into the FPGA chip:load it into the FPGA chip:

struct module myAsc = {netlist: ‘my_asc.netlist’,device_file: ‘’,

};ioctl(module_generator, STREAM_IOCTL_GENERATE, &myASC);write(myAsc.device_file, data_in, sizeof(data_in));...

The FutureThe Future ContinuedContinued……

Fully automated platform for writing and Fully automated platform for writing and running ASC-oriented applicationsrunning ASC-oriented applications

Write the program, run the compiler and receive a Write the program, run the compiler and receive a a running application that uses custom made a running application that uses custom made hardware accelerationhardware acceleration

An application that will identify critical code An application that will identify critical code sections in the program, convert them into ASC sections in the program, convert them into ASC hardware module and insert the appropriate calls hardware module and insert the appropriate calls to the device driverto the device driverSoftwar

eSoftware

Operating

System

Hardware

Thanks…Thanks…

Great thanks to our instructor Great thanks to our instructor Envgeni for his teaching, ideas Envgeni for his teaching, ideas and helpand help

Many thanks to the High Speed Many thanks to the High Speed Digital Systems Laboratory for Digital Systems Laboratory for their support and patience: Eli, their support and patience: Eli, Ina, Mony and of course BruriyaIna, Mony and of course Bruriya

THE ENDTHE END

device driver for generic asc module project presentation by: yigal korman erez fuchs instructor:...

Documents

software hardware slide

hardware netlist

hardware software design

software code

hardware sections

edif hardware

c code hardware

software program