Comments for page "Getting Rid of ZeroMQ-style Contexts"

(no title)

Fri, 17 May 2013 19:44:07 +0000

Wouldn't rigorously setting CLOEXEC on all fds do the job?

(no title)

Fri, 17 May 2013 19:20:12 +0000

Forking doesn't work well in 0mq. When fork()'ing, sockets are dup'ed, But the new (child) can't "close" them… because the context can't be reused. Then, if the parent tries to exit and reconnect, the dup'ed sockets block the bind() call.

After a fork, user should call nm_postfork(MODE)

Check to see if the pid associated with the socket is the same as the pid that opened it. if not, duplicated queues are cleared, and "file descriptor based" (tcp, unix) sockets are closed. All sockets are still valid in the child, but they are all "closed".

Especially when there are no "contexts" you need a way of handling fork()

(no title)

Tue, 14 May 2013 20:11:38 +0000

Shut down is done when the last socket is closed. It requires some synchronisation kung-fu but it's doable.

You are right about the title. I've changed it to "ZeroMQ-style contexts".

(no title)

Tue, 14 May 2013 20:09:20 +0000

1) Same behaviour is exhibited by zmq_term(). How do you handle it there? This is just moving it to close() function. That being said, the only non-half-assed (full-assed?) solution is to move the entire messaging layer into kernel space and thus allow it to send pending data even after the process it terminated.

2) Global state is simply global, guarded by a mutex. However, the mutex is locked only when sockets are sreated or closed. The critical path (send/recv) doesn't touch the global state and thus requires no locking.

Re: Shutdown...

Tue, 14 May 2013 19:58:03 +0000

The term() function still exists, but it is not longer a required step in the process shutdown. It is an optional utility that unblocks any blocking calls and causes nanomsg functions to return ETERM from that point on.

Actual deallocation of the global resources happens when last socket is closed, irrespectively of whether nn_term() was used or not.

(no title)

Tue, 14 May 2013 17:32:11 +0000

Contexts were a design mistake, yes. There are use cases for them but they should not be part of the basic API. However you don't need blocking as far as I can tell. What most apps do, and what the API should do, is maintain one context by default. A static thread safe object. Making this visible to the user and asking the user to take control was too complex (especially when we still asked the user to specify how many i/o threads to use…)

Still, with no contexts how do you shut-down the whole engine safely?

BTW the title of your post is misleading, you are not getting rid of ZeroMQ contexts. :)

(no title)

Tue, 14 May 2013 16:52:49 +0000

Two comments:

1)
Every API calls needs to provide a non-blocking variant
e.g. nn_close() always blocking will greatly complicate event-driven
code that assumes no blocking…

2)
To avoid locking between threads, are you planning to use
an implicit per-thread context (i.e. thread-local variable) in conjunction
with the shared/locked global state?

If the core of the library makes use of a context, then many options are possible:

1) Provide a shared implicit context for the entire process
+ the simple API you outlined above that uses the global context

2) An implicit per-thread context (totally independent or shared between some threads)

3) Multiple independent contexts under full programmer control

Shutdown...

Tue, 14 May 2013 15:34:30 +0000

How do you plan on shutdown?

In ZeroMQ for example, I often setup different threads to handle various sockets. The threads will just listen for data and process it (think something like a worker thread). In ZeroMQ, we currently have a context.term(). This interrupts any threads or pollers to throw a ZMQException (in Java). I can then catch the exception and call socket.close() in each thread and close out the loops.

A possible solution is for the socket.close() to be thread-safe. I can then close the sockets from my main thread.