Group PJ_IOQUEUE

group PJ_IOQUEUE

I/O Queue provides API for performing asynchronous I/O operations. It conforms to proactor pattern, which allows application to submit an asynchronous operation and to be notified later when the operation has completed.

The I/O Queue can work on both socket and file descriptors. For asynchronous file operations however, one must make sure that the correct file I/O back-end is used, because not all file I/O back-end can be used with the ioqueue. Please see File I/O for more details.

The framework works natively in platforms where asynchronous operation API exists, such as in Windows NT with IoCompletionPort/IOCP. In other platforms, the I/O queue abstracts the operating system’s event poll API to provide semantics similar to IoCompletionPort with minimal penalties (i.e. per ioqueue and per handle mutex protection).

The I/O queue provides more than just unified abstraction. It also:

• makes sure that the operation uses the most effective way to utilize the underlying mechanism, to achieve the maximum theoritical throughput possible on a given platform.

• choose the most efficient mechanism for event polling on a given platform.

Currently, the I/O Queue is implemented using:

• , as the common denominator, but the least efficient. Also the number of descriptor is limited to PJ_IOQUEUE_MAX_HANDLES (which by default is 64).

• on Linux (user mode and kernel mode), a much faster replacement for select() on Linux (and more importantly doesn’t have limitation on number of descriptors).

• I/O Completion ports on Windows NT/2000/XP, which is the most efficient way to dispatch events in Windows NT based OSes, and most importantly, it doesn’t have the limit on how many handles to monitor. And it works with files (not only sockets) as well.

Concurrency Rules

The ioqueue has been fine tuned to allow multiple threads to poll the handles simultaneously, to maximize scalability when the application is running on multiprocessor systems. When more than one threads are polling the ioqueue and there are more than one handles are signaled, more than one threads will execute the callback simultaneously to serve the events. These parallel executions are completely safe when the events happen for two different handles.

However, with multithreading, care must be taken when multiple events happen on the same handle, or when event is happening on a handle (and the callback is being executed) and application is performing unregistration to the handle at the same time.

The treatments of above scenario differ according to the concurrency setting that are applied to the handle.

Concurrency Settings for Handles

Concurrency can be set on per handle (key) basis, by using pj_ioqueue_set_concurrency() function. The default key concurrency value for the handle is inherited from the key concurrency setting of the ioqueue, and the key concurrency setting for the ioqueue can be changed by using pj_ioqueue_set_default_concurrency(). The default key concurrency setting for ioqueue itself is controlled by compile time setting PJ_IOQUEUE_DEFAULT_ALLOW_CONCURRENCY.

Note that this key concurrency setting only controls whether multiple threads are allowed to operate on the same key at the same time. The ioqueue itself always allows multiple threads to enter the ioqeuue at the same time, and also simultaneous callback calls to differrent keys is always allowed regardless to the key concurrency setting.

Parallel Callback Executions for the Same Handle

Note that when key concurrency is enabled (i.e. parallel callback calls on the same key is allowed; this is the default setting), the ioqueue will only perform simultaneous callback executions on the same key when the key has invoked multiple pending operations. This could be done for example by calling pj_ioqueue_recvfrom() more than once on the same key, each with the same key but different operation key (pj_ioqueue_op_key_t). With this scenario, when multiple packets arrive on the key at the same time, more than one threads may execute the callback simultaneously, each with the same key but different operation key.

When there is only one pending operation on the key (e.g. there is only one pj_ioqueue_recvfrom() invoked on the key), then events occuring to the same key will be queued by the ioqueue, thus no simultaneous callback calls will be performed.

Concurrency is Enabled (Default Value)

The default setting for the ioqueue is to allow multiple threads to execute callbacks for the same handle/key. This setting is selected to promote good performance and scalability for application.

However this setting has a major drawback with regard to synchronization, and application MUST carefully follow the following guidelines to ensure that parallel access to the key does not cause problems:

• Always note that callback may be called simultaneously for the same key.

• Care must be taken when unregistering a key from the ioqueue. Application must take care that when one thread is issuing an unregistration, other thread is not simultaneously invoking the callback to the same key

  .

  This happens because the ioqueue functions are working with a pointer to the key, and there is a possible race condition where the pointer has been rendered invalid by other threads before the ioqueue has a chance to acquire mutex on it.

Concurrency is Disabled

Alternatively, application may disable key concurrency to make synchronization easier. As noted above, there are three ways to control key concurrency setting:

• by controlling on per handle/key basis, with pj_ioqueue_set_concurrency().

• by changing default key concurrency setting on the ioqueue, with pj_ioqueue_set_default_concurrency().

• by changing the default concurrency on compile time, by declaring PJ_IOQUEUE_DEFAULT_ALLOW_CONCURRENCY macro to zero in your config_site.h

Examples

For some examples on how to use the I/O Queue, please see:

• I/O Queue TCP test: pjlib/src/pjlib-test/ioq_tcp.c

• I/O Queue UDP test: pjlib/src/pjlib-test/ioq_udp.c

• I/O Queue Performance test: pjlib/src/pjlib-test/ioq_perf.c

Defines

PJ_IOQUEUE_MAX_EVENTS_IN_SINGLE_POLL

This macro specifies the maximum number of events that can be processed by the ioqueue on a single poll cycle, on implementation that supports it. The value is only meaningfull when specified during PJLIB build.

PJ_IOQUEUE_MAX_CAND_EVENTS

This macro specifies the maximum event candidates collected by each polling thread to be able to reach maximum number of processed events (i.e: PJ_IOQUEUE_MAX_EVENTS_IN_SINGLE_POLL) in each poll cycle. An event candidate will be dispatched to application as event unless it is already being dispatched by other polling thread. So in order to anticipate such race condition, each poll operation should collects its event candidates more than PJ_IOQUEUE_MAX_EVENTS_IN_SINGLE_POLL, the recommended value is (PJ_IOQUEUE_MAX_EVENTS_IN_SINGLE_POLL * number of polling threads).

The value is only meaningfull when specified during PJLIB build and is only effective on multiple polling threads environment.

PJ_IOQUEUE_ALWAYS_ASYNC

When this flag is specified in ioqueue’s recv() or send() operations, the ioqueue will always mark the operation as asynchronous.

Enums

enum pj_ioqueue_operation_e

Types of pending I/O Queue operation. This enumeration is only used internally within the ioqueue.

Values:

enumerator PJ_IOQUEUE_OP_NONE

No operation.

enumerator PJ_IOQUEUE_OP_READ

read() operation.

enumerator PJ_IOQUEUE_OP_RECV

recv() operation.

enumerator PJ_IOQUEUE_OP_RECV_FROM

recvfrom() operation.

enumerator PJ_IOQUEUE_OP_WRITE

write() operation.

enumerator PJ_IOQUEUE_OP_SEND

send() operation.

enumerator PJ_IOQUEUE_OP_SEND_TO

sendto() operation.

enumerator PJ_IOQUEUE_OP_ACCEPT

accept() operation.

enumerator PJ_IOQUEUE_OP_CONNECT

connect() operation.

enum pj_ioqueue_epoll_flag

Epoll flags.

Values:

enumerator PJ_IOQUEUE_EPOLL_EXCLUSIVE

Use of EPOLLEXCLUSIVE.

enumerator PJ_IOQUEUE_EPOLL_ONESHOT

Use of EPOLLONESHOT.

enumerator PJ_IOQUEUE_EPOLL_AUTO

Default flag to specify which epoll type to use, which mean to use EPOLLEXCLUSIVE if available, otherwise EPOLLONESHOT, otherwise “bare” epoll when neither are available.

Functions

void pj_ioqueue_cfg_default(pj_ioqueue_cfg *cfg)

Initialize the ioqueue configuration with the default values.

Parameters:

cfg – The configuration to be initialized.

const char *pj_ioqueue_name(void)

Return the name of the ioqueue implementation.

Returns:

Implementation name.

pj_status_t pj_ioqueue_create(pj_pool_t *pool, pj_size_t max_fd, pj_ioqueue_t **ioqueue)

Create a new I/O Queue framework.

Parameters:

• pool – The pool to allocate the I/O queue structure.

• max_fd – The maximum number of handles to be supported, which should not exceed PJ_IOQUEUE_MAX_HANDLES.

• ioqueue – Pointer to hold the newly created I/O Queue.

Returns:

PJ_SUCCESS on success.

pj_status_t pj_ioqueue_create2(pj_pool_t *pool, pj_size_t max_fd, const pj_ioqueue_cfg *cfg, pj_ioqueue_t **ioqueue)

Create a new I/O Queue framework.

Parameters:

• pool – The pool to allocate the I/O queue structure.

• max_fd – The maximum number of handles to be supported, which should not exceed PJ_IOQUEUE_MAX_HANDLES.

• cfg – Optional ioqueue configuration. Application must initialize this structure with pj_ioqueue_cfg_default() first. If this is not specified, default config values as set pj_ioqueue_cfg_default() by will be used.

• ioqueue – Pointer to hold the newly created I/O Queue.

Returns:

PJ_SUCCESS on success.

pj_status_t pj_ioqueue_destroy(pj_ioqueue_t *ioque)

Destroy the I/O queue.

Parameters:

ioque – The I/O Queue to be destroyed.

Returns:

PJ_SUCCESS if success.

pj_status_t pj_ioqueue_set_lock(pj_ioqueue_t *ioque, pj_lock_t *lock, pj_bool_t auto_delete)

Set the lock object to be used by the I/O Queue. This function can only be called right after the I/O queue is created, before any handle is registered to the I/O queue.

Initially the I/O queue is created with non-recursive mutex protection. Applications can supply alternative lock to be used by calling this function.

Parameters:

• ioque – The ioqueue instance.

• lock – The lock to be used by the ioqueue.

• auto_delete – In non-zero, the lock will be deleted by the ioqueue.

Returns:

PJ_SUCCESS or the appropriate error code.

pj_status_t pj_ioqueue_set_default_concurrency(pj_ioqueue_t *ioqueue, pj_bool_t allow)

Set default concurrency policy for this ioqueue. If this function is not called, the default concurrency policy for the ioqueue is controlled by compile time setting PJ_IOQUEUE_DEFAULT_ALLOW_CONCURRENCY.

Note that changing the concurrency setting to the ioqueue will only affect subsequent key registrations. To modify the concurrency setting for individual key, use pj_ioqueue_set_concurrency().

Parameters:

• ioqueue – The ioqueue instance.

• allow – Non-zero to allow concurrent callback calls, or PJ_FALSE to disallow it.

Returns:

PJ_SUCCESS on success or the appropriate error code.

pj_status_t pj_ioqueue_register_sock(pj_pool_t *pool, pj_ioqueue_t *ioque, pj_sock_t sock, void *user_data, const pj_ioqueue_callback *cb, pj_ioqueue_key_t **key)

Register a socket to the I/O queue framework. When a socket is registered to the IOQueue, it may be modified to use non-blocking IO. If it is modified, there is no guarantee that this modification will be restored after the socket is unregistered.

Parameters:

• pool – To allocate the resource for the specified handle, which must be valid until the handle/key is unregistered from I/O Queue.

• ioque – The I/O Queue.

• sock – The socket.

• user_data – User data to be associated with the key, which can be retrieved later.

• cb – Callback to be called when I/O operation completes.

• key – Pointer to receive the key to be associated with this socket. Subsequent I/O queue operation will need this key.

Returns:

PJ_SUCCESS on success, or the error code.

pj_status_t pj_ioqueue_register_sock2(pj_pool_t *pool, pj_ioqueue_t *ioque, pj_sock_t sock, pj_grp_lock_t *grp_lock, void *user_data, const pj_ioqueue_callback *cb, pj_ioqueue_key_t **key)

Variant of pj_ioqueue_register_sock() with additional group lock parameter. If group lock is set for the key, the key will add the reference counter when the socket is registered and decrease it when it is destroyed.

pj_status_t pj_ioqueue_unregister(pj_ioqueue_key_t *key)

Unregister from the I/O Queue framework. Caller must make sure that the key doesn’t have any pending operations before calling this function, by calling pj_ioqueue_is_pending() for all previously submitted operations except asynchronous connect, and if necessary call pj_ioqueue_post_completion() to cancel the pending operations.

Note that asynchronous connect operation will automatically be cancelled during the unregistration.

Also note that when I/O Completion Port backend is used, application MUST close the handle immediately after unregistering the key. This is because there is no unregistering API for IOCP. The only way to unregister the handle from IOCP is to close the handle.

See also

pj_ioqueue_is_pending

Parameters:

key – The key that was previously obtained from registration.

Returns:

PJ_SUCCESS on success or the error code.

void *pj_ioqueue_get_user_data(pj_ioqueue_key_t *key)

Get user data associated with an ioqueue key.

Parameters:

key – The key that was previously obtained from registration.

Returns:

The user data associated with the descriptor, or NULL on error or if no data is associated with the key during registration.

pj_status_t pj_ioqueue_set_user_data(pj_ioqueue_key_t *key, void *user_data, void **old_data)

Set or change the user data to be associated with the file descriptor or handle or socket descriptor.

Parameters:

• key – The key that was previously obtained from registration.

• user_data – User data to be associated with the descriptor.

• old_data – Optional parameter to retrieve the old user data.

Returns:

PJ_SUCCESS on success or the error code.

pj_status_t pj_ioqueue_set_concurrency(pj_ioqueue_key_t *key, pj_bool_t allow)

Configure whether the ioqueue is allowed to call the key’s callback concurrently/in parallel. The default concurrency setting for the key is controlled by ioqueue’s default concurrency value, which can be changed by calling pj_ioqueue_set_default_concurrency().

If concurrency is allowed for the key, it means that if there are more than one pending operations complete simultaneously, more than one threads may call the key’s callback at the same time. This generally would promote good scalability for application, at the expense of more complexity to manage the concurrent accesses in application’s code.

Alternatively application may disable the concurrent access by setting the allow flag to false. With concurrency disabled, only one thread can call the key’s callback at one time.

Parameters:

• key – The key that was previously obtained from registration.

• allow – Set this to non-zero to allow concurrent callback calls and zero (PJ_FALSE) to disallow it.

Returns:

PJ_SUCCESS on success or the appropriate error code.

pj_status_t pj_ioqueue_lock_key(pj_ioqueue_key_t *key)

Acquire the key’s mutex. When the key’s concurrency is disabled, application may call this function to synchronize its operation with the key’s callback (i.e. this function will block until the key’s callback returns).

Parameters:

key – The key that was previously obtained from registration.

Returns:

PJ_SUCCESS on success or the appropriate error code.

pj_status_t pj_ioqueue_trylock_key(pj_ioqueue_key_t *key)

Try to acquire the key’s mutex. When the key’s concurrency is disabled, application may call this function to synchronize its operation with the key’s callback.

Parameters:

key – The key that was previously obtained from registration.

Returns:

PJ_SUCCESS on success or the appropriate error code.

pj_status_t pj_ioqueue_unlock_key(pj_ioqueue_key_t *key)

Release the lock previously acquired with pj_ioqueue_lock_key().

Parameters:

key – The key that was previously obtained from registration.

Returns:

PJ_SUCCESS on success or the appropriate error code.

void pj_ioqueue_op_key_init(pj_ioqueue_op_key_t *op_key, pj_size_t size)

Initialize operation key.

Parameters:

• op_key – The operation key to be initialied.

• size – The size of the operation key.

pj_bool_t pj_ioqueue_is_pending(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key)

Check if operation is pending on the specified operation key. The op_key must have been initialized with pj_ioqueue_op_key_init() or submitted as pending operation before, or otherwise the result is undefined.

Parameters:

• key – The key.

• op_key – The operation key, previously submitted to any of the I/O functions and has returned PJ_EPENDING.

Returns:

Non-zero if operation is still pending.

pj_status_t pj_ioqueue_post_completion(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, pj_ssize_t bytes_status)

Post completion status to the specified operation key and call the appropriate callback. When the callback is called, the number of bytes received in read/write callback or the status in accept/connect callback will be set from the bytes_status parameter.

Parameters:

• key – The key.

• op_key – Pending operation key.

• bytes_status – Number of bytes or status to be set. A good value to put here is -PJ_ECANCELLED.

Returns:

PJ_SUCCESS if completion status has been successfully sent.

pj_status_t pj_ioqueue_clear_key(pj_ioqueue_key_t *key)

Clear ioqueue key states. This function will cancel any outstanding operations on that key, without invoking any completion callback. After calling this function, application should reinit its all operation keys, i.e: using pj_ioqueue_op_key_init(), before reusing them.

Parameters:

key – The key.

Returns:

PJ_SUCCESS on success or the appropriate error code.

pj_status_t pj_ioqueue_accept(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, pj_sock_t *new_sock, pj_sockaddr_t *local, pj_sockaddr_t *remote, int *addrlen)

Instruct I/O Queue to accept incoming connection on the specified listening socket. This function will return immediately (i.e. non-blocking) regardless whether a connection is immediately available. If the function can’t complete immediately, the caller will be notified about the incoming connection when it calls pj_ioqueue_poll(). If a new connection is immediately available, the function returns PJ_SUCCESS with the new connection; in this case, the callback WILL NOT be called.

Parameters:

• key – The key which registered to the server socket.

• op_key – An operation specific key to be associated with the pending operation, so that application can keep track of which operation has been completed when the callback is called.

• new_sock – Argument which contain pointer to receive the new socket for the incoming connection.

• local – Optional argument which contain pointer to variable to receive local address.

• remote – Optional argument which contain pointer to variable to receive the remote address.

• addrlen – On input, contains the length of the buffer for the address, and on output, contains the actual length of the address. This argument is optional.

Returns:

• PJ_SUCCESS When connection is available immediately, and the parameters will be updated to contain information about the new connection. In this case, a completion callback WILL NOT be called.

• PJ_EPENDING If no connection is available immediately. When a new connection arrives, the callback will be called.

• non-zero which indicates the appropriate error code.

pj_status_t pj_ioqueue_connect(pj_ioqueue_key_t *key, const pj_sockaddr_t *addr, int addrlen)

Initiate non-blocking socket connect. If the socket can NOT be connected immediately, asynchronous connect() will be scheduled and caller will be notified via completion callback when it calls pj_ioqueue_poll(). If socket is connected immediately, the function returns PJ_SUCCESS and completion callback WILL NOT be called.

Parameters:

• key – The key associated with TCP socket

• addr – The remote address.

• addrlen – The remote address length.

Returns:

• PJ_SUCCESS If socket is connected immediately. In this case, the completion callback WILL NOT be called.

• PJ_EPENDING If operation is queued, or

• non-zero Indicates the error code.

int pj_ioqueue_poll(pj_ioqueue_t *ioque, const pj_time_val *timeout)

Poll the I/O Queue for completed events.

Note: polling the ioqueue is not necessary in Symbian. Please see Symbian OS Specific for more info.

Parameters:

• ioque – the I/O Queue.

• timeout – polling timeout, or NULL if the thread wishes to wait indefinetely for the event.

Returns:

• zero if timed out (no event).

• (<0) if error occured during polling. Callback will NOT be called.

• (>1) to indicate numbers of events. Callbacks have been called.

pj_status_t pj_ioqueue_recv(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, void *buffer, pj_ssize_t *length, pj_uint32_t flags)

Instruct the I/O Queue to read from the specified handle. This function returns immediately (i.e. non-blocking) regardless whether some data has been transferred. If the operation can’t complete immediately, caller will be notified about the completion when it calls pj_ioqueue_poll(). If data is immediately available, the function will return PJ_SUCCESS and the callback WILL NOT be called.

Parameters:

• key – The key that uniquely identifies the handle.

• op_key – An operation specific key to be associated with the pending operation, so that application can keep track of which operation has been completed when the callback is called. Caller must make sure that this key remains valid until the function completes.

• buffer – The buffer to hold the read data. The caller MUST make sure that this buffer remain valid until the framework completes reading the handle.

• length – On input, it specifies the size of the buffer. If data is available to be read immediately, the function returns PJ_SUCCESS and this argument will be filled with the amount of data read. If the function is pending, caller will be notified about the amount of data read in the callback. This parameter can point to local variable in caller’s stack and doesn’t have to remain valid for the duration of pending operation.

• flags – Recv flag. If flags has PJ_IOQUEUE_ALWAYS_ASYNC then the function will never return PJ_SUCCESS.

Returns:

• PJ_SUCCESS If immediate data has been received in the buffer. In this case, the callback WILL NOT be called.

• PJ_EPENDING If the operation has been queued, and the callback will be called when data has been received.

• non-zero The return value indicates the error code.

pj_status_t pj_ioqueue_recvfrom(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, void *buffer, pj_ssize_t *length, pj_uint32_t flags, pj_sockaddr_t *addr, int *addrlen)

This function behaves similarly as pj_ioqueue_recv(), except that it is normally called for socket, and the remote address will also be returned along with the data. Caller MUST make sure that both buffer and addr remain valid until the framework completes reading the data.

Parameters:

• key – The key that uniquely identifies the handle.

• op_key – An operation specific key to be associated with the pending operation, so that application can keep track of which operation has been completed when the callback is called.

• buffer – The buffer to hold the read data. The caller MUST make sure that this buffer remain valid until the framework completes reading the handle.

• length – On input, it specifies the size of the buffer. If data is available to be read immediately, the function returns PJ_SUCCESS and this argument will be filled with the amount of data read. If the function is pending, caller will be notified about the amount of data read in the callback. This parameter can point to local variable in caller’s stack and doesn’t have to remain valid for the duration of pending operation.

• flags – Recv flag. If flags has PJ_IOQUEUE_ALWAYS_ASYNC then the function will never return PJ_SUCCESS.

• addr – Optional Pointer to buffer to receive the address.

• addrlen – On input, specifies the length of the address buffer. On output, it will be filled with the actual length of the address. This argument can be NULL if addr is not specified.

Returns:

• PJ_SUCCESS If immediate data has been received. In this case, the callback must have been called before this function returns, and no pending operation is scheduled.

• PJ_EPENDING If the operation has been queued.

• non-zero The return value indicates the error code.

pj_status_t pj_ioqueue_send(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, const void *data, pj_ssize_t *length, pj_uint32_t flags)

Instruct the I/O Queue to write to the handle. This function will return immediately (i.e. non-blocking) regardless whether some data has been transferred. If the function can’t complete immediately, the caller will be notified about the completion when it calls pj_ioqueue_poll(). If operation completes immediately and data has been transferred, the function returns PJ_SUCCESS and the callback will NOT be called.

Parameters:

• key – The key that identifies the handle.

• op_key – An operation specific key to be associated with the pending operation, so that application can keep track of which operation has been completed when the callback is called.

• data – The data to send. Caller MUST make sure that this buffer remains valid until the write operation completes.

• length – On input, it specifies the length of data to send. When data was sent immediately, this function returns PJ_SUCCESS and this parameter contains the length of data sent. If data can not be sent immediately, an asynchronous operation is scheduled and caller will be notified via callback the number of bytes sent. This parameter can point to local variable on caller’s stack and doesn’t have to remain valid until the operation has completed.

• flags – Send flags. If flags has PJ_IOQUEUE_ALWAYS_ASYNC then the function will never return PJ_SUCCESS.

Returns:

• PJ_SUCCESS If data was immediately transferred. In this case, no pending operation has been scheduled and the callback WILL NOT be called.

• PJ_EPENDING If the operation has been queued. Once data base been transferred, the callback will be called.

• non-zero The return value indicates the error code.

pj_status_t pj_ioqueue_sendto(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, const void *data, pj_ssize_t *length, pj_uint32_t flags, const pj_sockaddr_t *addr, int addrlen)

Instruct the I/O Queue to write to the handle. This function will return immediately (i.e. non-blocking) regardless whether some data has been transferred. If the function can’t complete immediately, the caller will be notified about the completion when it calls pj_ioqueue_poll(). If operation completes immediately and data has been transferred, the function returns PJ_SUCCESS and the callback will NOT be called.

Parameters:

• key – the key that identifies the handle.

• op_key – An operation specific key to be associated with the pending operation, so that application can keep track of which operation has been completed when the callback is called.

• data – the data to send. Caller MUST make sure that this buffer remains valid until the write operation completes.

• length – On input, it specifies the length of data to send. When data was sent immediately, this function returns PJ_SUCCESS and this parameter contains the length of data sent. If data can not be sent immediately, an asynchronous operation is scheduled and caller will be notified via callback the number of bytes sent. This parameter can point to local variable on caller’s stack and doesn’t have to remain valid until the operation has completed.

• flags – send flags. If flags has PJ_IOQUEUE_ALWAYS_ASYNC then the function will never return PJ_SUCCESS.

• addr – Optional remote address.

• addrlen – Remote address length, addr is specified.

Returns:

• PJ_SUCCESS If data was immediately written.

• PJ_EPENDING If the operation has been queued.

• non-zero The return value indicates the error code.

pj_oshandle_t pj_ioqueue_get_os_handle(pj_ioqueue_t *ioqueue)

Get the underlying OS handle associated with an ioqueue instance.

Parameters:

ioqueue – The ioqueue instance.

Returns:

The OS handle associated with the instance. For epoll/kqueue this will be a pointer to the file descriptor. For all other platforms, this will be a pointer to a platform-specific handle. If no handle is available, NULL will be returned.

struct pj_ioqueue_op_key_t

#include <ioqueue.h>

This structure describes operation specific key to be submitted to I/O Queue when performing the asynchronous operation. This key will be returned to the application when completion callback is called.

Application normally wants to attach it’s specific data in the user_data field so that it can keep track of which operation has completed when the callback is called. Alternatively, application can also extend this struct to include its data, because the pointer that is returned in the completion callback will be exactly the same as the pointer supplied when the asynchronous function is called.

Public Members

void *internal__[32]

Internal I/O Queue data.

void *activesock_data

Active socket data.

void *user_data

Application data.

struct pj_ioqueue_callback

#include <ioqueue.h>

This structure describes the callbacks to be called when I/O operation completes.

Public Members

void (*on_read_complete)(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, pj_ssize_t bytes_read)

This callback is called when pj_ioqueue_recv or pj_ioqueue_recvfrom completes.

Param key:

The key.

Param op_key:

Operation key.

Param bytes_read:

>= 0 to indicate the amount of data read, otherwise negative value containing the error code. To obtain the pj_status_t error code, use (pj_status_t code = -bytes_read).

void (*on_write_complete)(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, pj_ssize_t bytes_sent)

This callback is called when pj_ioqueue_send or pj_ioqueue_sendto completes.

Param key:

The key.

Param op_key:

Operation key.

Param bytes_sent:

>= 0 to indicate the amount of data written, otherwise negative value containing the error code. To obtain the pj_status_t error code, use (pj_status_t code = -bytes_sent).

void (*on_accept_complete)(pj_ioqueue_key_t *key, pj_ioqueue_op_key_t *op_key, pj_sock_t sock, pj_status_t status)

This callback is called when pj_ioqueue_accept completes.

Param key:

The key.

Param op_key:

Operation key.

Param sock:

Newly connected socket.

Param status:

Zero if the operation completes successfully.

void (*on_connect_complete)(pj_ioqueue_key_t *key, pj_status_t status)

This callback is called when pj_ioqueue_connect completes.

Param key:

The key.

Param status:

PJ_SUCCESS if the operation completes successfully.

struct pj_ioqueue_cfg

#include <ioqueue.h>

Additional settings that can be given during ioqueue creation. Application MUST initialize this structure with pj_ioqueue_cfg_default().

Public Members

unsigned epoll_flags

Specify flags to control e.g. how events are handled when epoll backend is used on Linux. The values are combination of pj_ioqueue_epoll_flag. The default value is PJ_IOQUEUE_DEFAULT_EPOLL_FLAGS, which by default is set to PJ_IOQUEUE_EPOLL_AUTO. This setting will be ignored for other ioqueue backends.

pj_bool_t default_concurrency

Default concurrency for the handles registered to this ioqueue. Setting this to non-zero enables a handle to process more than one operations at the same time using different threads. Default is PJ_IOQUEUE_DEFAULT_ALLOW_CONCURRENCY. This setting is equivalent to calling pj_ioqueue_set_default_concurrency() after creating the ioqueue.