Introduction to CUDA!

Jonathan Baca!

Installing CUDA!

 CUDA-Ready System! Make sure you have the right Linux! Make sure that gcc is installed! Download the software! Install the drivers (NVIDIA)! Install CUDA!

CUDA-Ready Systems!

 In linux, run the following command:! lpsci | grep -i nvidia!

 This will show the settings on your system.!

 A list of CUDA compliant GPUs can be found at:! nvidia.com/object/cuda_gpus.html

Make sure you have the right Linux! Only certain Linux distributions are supported.!

  These are very specific and unfortunately rigid standards.!

 The following command can be run:!  uname -m && cat /etc/*release!

 A list of Linux distributions can be found on the CUDA website.!

Make sure that gcc is installed!

 Pretty self-explanatory.!

Download the Software!

 The software can be found at:! nvidia.com/object/cuda_get.html!

 You must download:! The NVIDIA Drivers! The CUDA Toolkit! The GPU Computing SDK!

Install the NVIDIA Drivers!

 Make sure everything is downloaded and you know where it is.!

 Exit the GUI using the command:! Ctrl-Alt-Backspace (sometimes twice)! or “sudo /etc/init.d/gdm stop”! or “/sbin/init 3”!

 This will vary from system to system!

Install the NVIDIA Drivers (cont)!

 You must have access to your system as a superuser.!

 Run the installation script.! Make sure that the right version is installed:!

  cat /proc/driver/nvidia/version! Next, what to do if you never use a GUI

environment.!

Install the NVIDIA Drivers (cont.)! Make sure that “/dev/nvidia*” exist.!  #/bin/bash!  /sbin/modprobe nvidia!  if [ "$?" -eq 0 ]; then!  # Count the number of NVIDIA controllers found.!  NVDEVS=`lspci | grep -i NVIDIA`!  N3D=`echo "$NVDEVS" | grep "3D controller" | wc -l`!  NVGA=`echo "$NVDEVS" | grep "VGA compatible controller" | wc -l`!  N=`expr $N3D + $NVGA - 1`!  for i in `seq 0 $N`; do!  mknod -m 666 /dev/nvidia$i c 195 $i!  done!  mknod -m 666 /dev/nvidiactl c 195 25!  else!  exit 1!  fi!

 Is a startup script to load the driver Kernel at boot time. (Must be a superuser obvs)!

Install the NVIDIA Drivers (cont..)!

 Restart the GUI environment using either:!  startx!  sudo /etc/init.d/gdm start!

 This is different from system to system.!

Install CUDA!

 Uninstall any previous incarnations of cuda:!  Delete all files from!

  /usr/local/cuda!  ~/NVIDIA_GPU_Computing_SDK or!  ~/NVIDIA_CUDA_SDK (for older installs)!

 Use the given “.run” file to install the files you need for CUDA.!

 You must define environment variables:!  export PATH=/usr/local/cuda/bin:$PATH!  export LD_LIBRARY_PATH=/usr/local/cuda/lib:

$LD_LIBRARY_PATH!

Install SDK!

 The second “.run” file must be run as a regular user (not a superuser)!

 The SDK will include many coding samples.!

Verifying Installation!

 A quick way to verify installation is to go to the C directory of the SDK installation and run:!  “make”!

 This will compile all of the examples.! Run “deviceQuery” (in the C/bin/<system>/

release directory of the SDK installation)! <system> is linux on a linux system.!

Installing on a Mac!

 On a Mac, you download the .dmg files.! You must still set the environment

variables.! Compiling the SDK is still done the same

way.!

Programming with CUDA!

 Introduction! Pros! Cons! Paradigms!

Introduction!

 Complete Unified Device Architecture! NVIDIA!

Pros!

 Extremely fast! 8-Series GPUs 25-200+!

 Scalable! Shared Memory! SIMD!

Cons!

 Eschews recursion! Needs a lot of data! SIMD!

Paradigms!

 SIMD! Threading! Groups! Blocks!

SIMD!

 Single Instruction Multiple Data! One function is run on all threads! Threads within the same block can

communicate with one another!

Grid!

 The top level of thread organization! Contains a 2D array of uniformly sized

blocks! Declared thusly:!

 dim3 dimGrid(64, 64)!

Block!

 Each block is a three dimensional array of threads!

 The threads within a block can communicate with one another!

 Declared thusly:! dim3 dimBlock(16, 16, 16)!

Shared Memory!!

 The amount of bytes of shared memory must also be declared.!

Calling a Function!

 CUDA Functions are called with a special syntax.!

 Suppose our function is:!  __global__ void Asbestos(args)!  DimGrid is our grid dimension!  DimBlock is our block dimension!  SharedMemBytes are the number of bytes of

shared memory! We would call this function using the

command:!  Asbestos <<< DimGrid, DimBlock,

SharedMemBytes >>> (args);!

CUDA Function Declaration!

 __device__!  Executed on the device and called from the device!

 __host__!  Executed on the device and called from the host! May be used with __device__!

 __global__!  Executed on the host and called from the host!  Kernel function! Must return void!

Memory Functions!

 cudaMalloc(target, size)! cudaMemcpy(target, original, size, *)! * indicates the keywords:!

 cudaMemcpyHostToDevice! cudaMemcpyDeviceToHost!

 cudaFree(target)!

Thread Functions!

 Thread Identification! threadIdx!

  threadIdx.x, threadIdx.y, threadIdx.z! blockIdx!

 blockIdx.y, blockIdx.y!

 Synchronization! __syncthreads()!

Granulation!

 Depending on the GPU, blocks cannot take certain sizes (this must be checked every time)!

 Choosing too large of blocks will slow down the hardware immensely.!

 Similarily, too small of blocks will.! Assigned to SMs (Streaming Multiprocessors)!

Summary!

 Installation! Verification! Demo! Paradigms! Summary!