© Armir Bujari – abujari@math.unipd.it

Universita Degli Studi di Padova

Concurrent and Distributed Programming

A.A. 2018/2019Bachelor's Degree in Computer Science

22

•This lecture series

• java.io java.nio (Why?)

• java.nio java.nio.2 (Why?)

33

NIO: New I/O

• Prior to the J2SE 1.4 release of Java, I/O had become a bottleneck.

• java.io based on the stream metaphor: byte-in / byte-out;

• No way to multiplex data from multiple sources without incurring thread context switches;

• No support for modern OS tricks for high performance I/O, like memory mapped files.

• New I/O changes that by providing

• A hierarchy of dedicated buffer classes that allow data to be moved from the JVM to the OS with minimal memory-to-memory copying,

• A unified family of channel classes that allow data to be fed directly from buffers to files and sockets, without going through the intermediaries of the old stream classes.

• A family of classes to directly implement selection (AKA readiness testing, AKA multiplexing) over a set of channels.

• NIO also provides file locking for the first time in Java.

44

Java IO vs NIO

I O NIO

Stream-oriented Buffer-oriented

Blocking I/O Non-blocking I/O

Selectors

Channels

55

Java.IO - Stream-oriented

Data Source Program

Data Destination Program

001001001111001001001001011

001001001111001001001001011

66

Socket Thread

read data, block unti l ready

read data, block unti l ready

write data, block unti l ready

write data, block unti l ready

Java.IO - Blocking I/O

77

Java.IO —Work with data

// read from socketScanner sc = new Scanner(socket.getInputStream());

String string = sc.nextLine();

System.out.println("Received " + string);

// write to socket

PrintWriter pw = new

PrintWriter(socket.getOutputStream());

pw.println("Hello");

88

Kernel

Disk

ControllerBufferBuffer DISK

Java IO

DMA

SLOW

- Using CPU to Buffer

- Thread Blocking

- Array Gabage Collection

Process (JVM)

read()

arrayCopy()

99

Java.NIO

Buffer-oriented & Non-blocking I/O

Channel Buffer Thread

read data into buffer

fill data into buffer

check data in buffer

goto top

1010

Java.NIO —Work with data

ByteBuffer readBuffer = ByteBuffer.allocate(1024);

//prepare to read

readBuffer.clear();

SocketChannel channel = getChannel(); //reading the buffer

// ...............

channel.read(readBuffer); readBuffer.flip();

ByteBuffer writeBuffer = ByteBuffer.allocate(1024);

// prepare to put data

writeBuffer.clear();

// putting the data

// ...............

// prepare to write

writeBuffer.flip();

channel.write(writeBuffer);

1111

Process (JVM)

read()Kernel

Disk

ControllerBuffer DISK

Java NIO – Direct Buffer

DMA

Direct

- Not use CPU to Buffer

- Thread Non-Blocking

- Reference Gabage Collection

Process JVM

DIRECT

1212

New IO (Java NIO)

•Buffer-oriented

• Data is read from a stream into a buffer

• Data is written from a buffer to a stream

•Non-blocking

• read() only reads the data that is currently available

• write() only writes the data that can be currently

written

• No need for several threads dedicated to blocking

read/write operations on separate channels, a

single thread (or a limited set of threads) can be

used

1313

NIO Concepts

Handlers• Channels

Provide for data transfers between sockets and buffers

• Buffers

Contiguous extent of memory for processing data

• Selector

Allows to wait on several channels until one or more become available for data transfer

channels

buffers

Sele

cto

r

1414

NIO Buffers

• To hold data, either received or to be sent• Not a stream, a random-access buffer of a given size

• Non-blocking send (send what can be sent)

• Non-blocking receive (only receive what is currently available)

• Keep the memory of read / write indexes

• Provide optimized operations

• Use the physical memory of the under-laying operating system for

native IO operations

• No additional copies when transfers are processed

• Buffer attributes

• Capacity (number of bytes)

• Position (next index for read/write)

• Limit (maximum number of bytes that can be read / written)

1515

The java.nio.Buffer Hierarchy

Buffer

CharBuffer IntBuffer DoubleBuffer ShortBuffer LongBuffer FloatBuffer ByteBuffer

MappedByteBuffer

1616

The ByteBuffer Class

•The most important buffer class in practice is probably the ByteBuffer class. This represents a fixed-size vector of primitive bytes.

• Important methods on this class include:

byte get()

byte get(int index)

ByteBuffer get(byte [] dst)

ByteBuffer get(byte [] dst, int offset, int length)

ByteBuffer put(byte b)

ByteBuffer put(int index, byte b)

ByteBuffer put(byte [] src)

ByteBuffer put(byte [] src, int offset, int length)

ByteBuffer put(ByteBuffer src)

1717

File Position and Limit

• Apart from forms with an index parameter, all operations are relativeoperations. The position property is like the file pointer in sequential file access.

• The superclass Buffer has methods for explicitly manipulating the position and related properties of buffers, e.g:

int position()

Buffer position(int newPosition)

int limit()

Buffer limit(int newLimit)

• The ByteBuffer or Buffer references returned by these various methods are simply references to this buffer object, not new buffers. They are provided to support cryptic invocation chaining. Feel free to ignore them.

• The limit property defines either the last space available for writing, or how much data has been written to the file.

• After finishing writing a flip() method can be called to set limit to the current value of position, and reset position to zero, ready for reading.

• Various operations implicitly work on the data between position and limit.

1818

Buffer's Basic Attributes

Invariant: 0 <= mark <= position <= limit <= capacity

• Capacity

• Limit

• Position

• Mark

1919

Buffer Basic Operations

• Creating

• Filling andDraining

• Flipping andRewind

• Marking

• Comparing

• Duplicating

2020

Creating

public static XXXBuffer allocate (int capacity)

public static XXXBuffer wrap (XXX [] array)

public static XXXBuffer wrap (XXX [] array, int offset,int length)

For e.g.

CharBuffer charBuffer= CharBuffer.allocate(10);

char [] charArray = new char[10];

CharBuffer charbuffer = CharBuffer.wrap (charArray);

CharBuffer charbuffer = CharBuffer.wrap (charArray, 2, 7);

2121

Filling and Draining

XXXBuffer put (XXX b);

XXXBuffer put (int index, XXXb);

XXX get( );

XXX get(int index);

XXXBuffer put (XXX[]src);

XXXBuffer put(XXX [] src, int offset, intlength);

XXXBuffer get(XXX[] dest);

XXXBufferget(XXX [] dest, int offset, int length);

2222

Flipping and Rewind

Manually Flipping a Buffer:

buffer.limit(buffer.position( )).position(0)

API provides Flip and rewind method: Buffer flip()

Buffer rewind()

2323

Marking

Buffer mark()

Buffer reset()

For e.g

buffer.position(2).mark( ).position(4);

2424

Marking - cont.

After calling resetmethod

2525

Comparing

boolean equals (Object ob)

int compareTo (Object ob)

Two buffers are considered to be equal if and only if:

Both objects are the same type

Both buffers have the same number of remaining elements

[position, limit)

The sequence of remaining data elements, which would be returned from get( ), must be identical in each buffer.

2626

Two buffers considered to beequal

Comparing cont.

2727

Two buffers considered to be unequal

Comparing cont.

2828

Duplicating

CharBuffer duplicate( );

CharBuffer asReadOnlyBuffer( );

CharBuffer slice( );

For e.g CharBuffer buffer = CharBuffer.allocate (8);

buffer.position (3).limit (6).mark( ).position (5);

CharBuffer dupeBuffer = buffer.duplicate( );

buffer.clear( );

2929

Duplicating – cont.

3030

Duplicating – cont.

e.g. CharBuffer buffer = CharBuffer.allocate(8);

buffer.position (3).limit (5);

CharBuffer sliceBuffer = buffer.slice( );

3131

Creating Buffers

• Four interesting factory methods can be used to create a new ByteBuffer:

ByteBuffer allocate(int capacity)

ByteBuffer allocateDirect(int capacity)

ByteBuffer wrap(byte [] array)

ByteBuffer wrap(byte [] array, int offset, length)

These are all static methods of the ByteBuffer class.

• allocate() creates a ByteBuffer with an ordinary Java backing array

of size capacity.

• allocateDirect()—perhaps the most interesting case—creates a

direct ByteBuffer, backed by capacity bytes of system memory.

• The wrap() methods create ByteBuffer’s backed by all or part of an

array allocated by the user.

• The other typed buffer classes (CharBuffer, etc) have similar factory methods, except they don’t support the important allocateDirect() method.

3232

View Buffers: cont.

ByteBuffer byteBuffer =ByteBuffer.allocate(7);

CharBuffer charBuffer = byteBuffer.asCharBuffer();

3333

• A channel is a new abstraction in java.nio.

• In the package java.nio.channels.

• Channels are a high-level version of the file-descriptors familiar from POSIX-compliant operating systems.

• So a channel is a handle for performing I/O operations and various control operations on an open file or socket.

• For those familiar with conventional Java I/O, java.nioassociates a channel with any RandomAccessFile, FileInputStream, FileOutputStream, Socket, ServerSocket or DatagramSocket object.

• The channel becomes a peer to the conventional Java handle objects; the conventional objects still exist, and in general retain their role—the channel just provides extra NIO-specificfunctionality.

• NIO buffer objects can be written to or read from channels directly. Channels also play an essential role in readiness selection, discussed later on.

Channels

3434

Channels and Buffers

All IO in NIO starts with a Channel. A Channel is a bit like a stream.

From the Channel, data can be read into a Buffer. Data can also be

written from a Buffer into a Channel.

3535

Simplified Channel Hierarchy

Channel<<<Interface>>>

ByteChannel<<<interface>>> SelectableChannel

FileChannel SocketChannelDatagramChannel ServerSocketChannel

Some of the “inheritance” arcs here are indirect: we missed

out some interesting intervening classes and interfaces.

3636

Opening Channels

• Socket channel classes have static factory methods called open(), e.g.:

SocketChannel sc = SocketChannel.open() ;

Sc.connect(new InetSocketAddress(hostname, portnumber)) ;

• File channels cannot be created directly; first use conventional Java I/O mechanisms to create a FileInputStream, FileOutputStream, or RandomAccessFile, then apply the new getChannel() method to get an associated NIO channel, e.g.:

RandomAccessFile raf = new RandomAccessFile(filename, “r”) ;

FileChannel fc = raf.getChannel() ;

ByteBuffer buffer = ByteBuffer.allocate( 1024 );

fc.read(buffer); //notice we did not specify the amount to read.

3737

Using Channels

•Any channel that implements the ByteChannel interface—i.e. all channels except ServerSocketChannel—provide a read() and a write() instance method:

int read(ByteBuffer dst)

int write(ByteBuffer src)

• These may look reminiscent of the read() and write() system calls in UNIX:

int read(int fd, void* buf, int count)

int write(int fd, void* buf, int count)

• The Java read() attempts to read from the channel as many bytes as there are remaining to be written in the dst buffer. Returns number of bytes actually read, or -1 if end-of-stream. Also updates dst buffer position.

• Similarly write() attempts to write to the channel as many bytes as there are remaining in the src buffer. Returns number of bytes actually read, and updates src buffer position.

3838

• This example assumes a source channel src and a destination channel dest:

ByteBuffer buffer = ByteBuffer.allocateDirect(BUF_SIZE) ;

while(src.read(buffer) != -1) {

buffer.flip() ; // Prepare read buffer for “draining”

while(buffer.hasRemaining())

dest.write(buffer) ;

buffer.clear() ; // Empty buffer, ready to read next chunk.

}

• Note a write() call (or a read() call) may or may not succeed in transferring whole buffer in a single call. Hence need for inner while loop.

• Example introduces two new methods on Buffer: hasRemaining()returns true if position < limit; clear() sets position to 0 and limit to buffer’s capacity.

• Because copying is a common operation on files, FileChannel provides a couple of special methods to do just this:

long transferTo/From(long position, long count, WriteableByteChannel target)

Example: Copying one Channel to Another

3939

• In modern operating systems one can exploit the virtual memory system to map a physical file into a region of program memory.

• Once the file is mapped, accesses to the file can be extremely fast: one doesn’t have to go through read() and write() system calls.

• One application might be a Web Server, where you want to read a whole file quickly and send it to a socket.

• Problems arise if the file structure is changed while it is mapped—use this technique only for fixed-size files.

• This low-level optimization is now available in Java. FileChannel has a method:

MappedByteBuffer map(MapMode mode, long position, long size)

• mode should be one of MapMode.READ_ONLY, MapMode.READ_WRITE, MapMode.PRIVATE.

• The returned MappedByteBuffer can be used wherever an ordinary ByteBuffer can.

Memory-Mapped Files

4040

Memory-Mapped Files

MappedByteBuffer map (MapMode mode, long position,long size)

4141

Scatter/Gather

ScatteringByteChannel

read (ByteBuffer [] dsts)

read (ByteBuffer [] dsts, int offset, int length)

GatheringByteChannel

write(ByteBuffer[] srcs)

write(ByteBuffer[] srcs, int offset, int length)

ByteBuffer header = ByteBuffer.allocate(128);

ByteBuffer body = ByteBuffer.allocate(1024);

ByteBuffer[] bufferArray = { header, body };

channel.read(bufferArray);

4242

A Selector allows a single thread to handle multiple Channels. This is handy if

your application has many connections (Channels) open, but only has low

traffic on each connection. For instance, in a chat server.

● To use a Selector you register the Channel's with it. Then you call it's

select() method.

● This method will block until there is an event ready for one of the

registered channels.

● Once the method returns, the thread can then process these events.

Examples of events

are incoming

connection,

data received etc.

Selectors

4343

• Prior to New I/O, Java provided no standard way of selecting—from a set of possible socket operations—just the ones that are currently ready to proceed, so the ready operations can be immediately serviced.

• One application would be in implementing an MPI-like message passing system: in general incoming messages from multiple peers must be consumed as they arrive and fed into a message queue, until the user program is ready to handle them.

• Previously one could achieve equivalent effects in Java by doing blocking I/O operations in separate threads, then merging the results through Java thread synchronization. But this can be inefficient because thread context switching and synchronization is quite slow.

• One way of achieving the desired effect in New I/O would be set all the channels involved to non-blocking mode, and use a polling loop to wait until some are ready to proceed.

• A more structured—and potentially more efficient—approach is to use Selectors.

• In many flavors of UNIX this is achieved by using the select() system call.

Readiness Selection - Select

4444

Selectors

•Creating a Selector

Selector selector = Selector.open();

• Registering Channels with the Selector

channel.configureBlocking(false);

SelectionKey key = channel.register(selector,SelectionKey.OP_READ);

Rules

1. The Channel must be in non-blocking mode to be used with a Selector.

2. This means that you cannot use FileChannel's with a Selector since FileChannel's

cannot be switched into non-blocking mode ( > JSE4).

3. Socket channels will work fine though.

4545

Selectors

● There is an "interest set", meaning what events you are

interested in listening for in the Channel, via the Selector.

There are four different events you can listen for:

1. Connect

2. Accept

3. Read

4. Write

● These four events are represented by

the four SelectionKey constants:

1. SelectionKey.OP_CONNECT

2. SelectionKey.OP_ACCEPT

3. SelectionKey.OP_READ

4. SelectionKey.OP_WRITE

● If you are interested in more than one event, OR the

constants together, like this:

int interestSet = SelectionKey.OP_READ | SelectionKey.OP_WRITE;

SelectionKey key = channel.register(selector, interestSet);

This SelectionKey object contains

a few interesting properties:

The interest set

The ready set

The Channel

The Selector

An attached object (optional)

4646

Select Methods

•Here are the select() methods:

• int select(): blocks until at least one channel is ready for

the events you registered for;

• int select(long timeout): does the same as select()

except it blocks for a maximum of timeoutmilliseconds (the

parameter);

• int selectNow(): doesn't block at all. It returns immediately

with whatever channels are ready.

4747

Blocking IO

ServerSocket server = new ServerSocket(8080);

Socket sock = server.accept();

InputStream in = sock.getInputStream();

len = in.read(buf);

…

Blocking

Blocking

4848

Multi Thread IO

ServerSocket server = new ServerSocket(8080);

while(true){

Socket sock = server.accept();

new ServerProcess (sock).start();

}

class ServerProcess extends Thread{

public void run(){

InputStream in = sock.getInputStream();

len = in.read(buf);

…

}

}

Problems with this model• Gabage Collection

Thread Context Switch Overhead

• Real-time• Out Of Memory

4949

Another approach: Polling

public class ServerAcceptor extends Thread{

public void run(){

Socket sock = server.accept();

socketList.add(socket);

}

}

public class ServerProcess extends Thread{

while(true){

for(Socket sock : socketList){

InputStream in = sock.getInputStream();

len = in.read(buf);

…

}

}

}

5050

Polling Problem

- Waiting

Polling problem

•… waiting

5151

NIO Socket Channel

public SelectableChannel configureBlocking(boolean block)

throws IOException

ServerSocketChannel ssc = ServerSocketChannel.open();

ssc.socket().bind(new InetSocketAddress(port));

ssc.configureBlocking(false);

while(true){

SocketChannel sc = ssc.accept();

if(null != sc){

sc.read(buf);…

}

}

Non-blocking

Non-blocking

5252

Select

5353

ServerSocketChannel ssc = ServerSocketChannel.open();

ssc.socket().bind(new InetSocketAddress(port));

ssc.configureBlocking(false);

Selector selector = Selector.open();

ssc.register(selector, SelectionKey.OP_ACCEPT);

while(true){

selector.select();

Set<SelectionKey> keyset = selector.selectedKeys();

for (SelectionKey key : keyset) {

SocketChannel sc = (ServerSocketChannel)key.channel().accpept();

sc.register(selector, SelectionKey.OP_READ);

…

}

NIO Selector

Non-blocking

select : wait