What I did was to extend the STD to do more. I did this by including incoming events, transitions, and outgoing events. This allows a lot more expression in the story. In fact it nearly tells the whole story. The transitions can be compiled into routines (or "Events" in your definition).

In my definition of STD an 'event' is something happening on the underlying socket (like a msg received), a user API routine call, or an 'internal event' being signalled.

Internal events are usually part of the of a transition on Object A, issuing an event on Object B.

I know this sounds weird, but think about how much code you could remove from nanomsg if your state diagrams looked like: http://westrick.com/Jerry/CubixSystem.pdf .

This is the method I invented to get a handle on the inherent complexity of async systems.

Be interested in what you think…

Oh, and yes, in this method, a socket can be assigned an STD per protocol, instead of one STD for all… Which seams to be the core of your problem.

I wonder how the stuff even got so widespread in the first place. We had a perfectly viable model (processes&pipes) even before threads so there was no obvious reason… except maybe, when you are coding a boring crud application, using threads generates a lot of fun. Not even speaking of job security :)

I was fascinated by the discussion of events verses 'safe' callback handling. The former puts event (handlers) in a (single) queue, the later puts the event handlers in two distinct queues - the LIFO and the stack. They are both fundamentally the same paradigm, they differ only in how the queue(s) is(are) managed.

I would argue that the 'safe' callback method is actually more dangerous for two reasons;
a) because it splits event handlers between the LIFO and the stack &
b) because it depends on a high level of dev discipline to keep track.

b) violates a fundamental rule that, despite all best intentions everyone is fallible.

I have not reviewed code, but you will find that the 'cleaner' (i.e. cleanly abstracted) your event paradigm, the easier it will be for devs to use the pattern and thus, they will be less likely to spend energy implementing something else. 'Clean' abstraction means easy setup, simple APIs, transparent architecture, event queue management (typically at least 3 levels) and queue visibility (debugging and system handling).

I have since taken the (event) paradigm to the ultimate conclusion of eliminating the need for threads (RTOS) altogether in many systems.

fwiw…

The only thing different is that EVENTS have no EVENT prefix. I.e. NN_USOCK_CONNECTED rather than NN_USOCK_EVENT_CONNECTED. The reason is that outgoing events are the only entities visible to the user of the object — states and actions are private to the state machine. Thus, as the state machine API goes, EVENT prefix would serve no meaningful purpose and just make the identifiers longer.

STATE:
The internal state of the machine
NN_USOCK_STATE_CONNECTED

ACTION:
A command placing the machine in a new state and possibly raising an event
NN_USOCK_ACTION_CONNECT

EVENT:
Raised when the machine enters a new state
NN_USOCK_EVENT_CONNECTED

The distinction between ACTION and EVENT may or may not be necessary, I don't yet know enough about the internals and you might be just using STATE vs having an EVENT fire and event I believe is your up-stream function call.

I hope this makes sense.

Can a source object ever have more than one owner?

No. Not allowed.

It's actually pretty crucial requirement. The idea is to have objects arranged in a tree (thus one owner for each one). Then we have to ways of communication:

1. Up the tree (direction from root to leaves): this is implemented as simple function calls
2. Down the tree (direction from leaves to root): this is implemented as state machine events

That makes interactions between the components relatively easy to grasp. With mulitple owners (i.e. graph instead of tree) it would be much harder.

I'm not familiar enough with the code base and how initialization works yet, so put another way, the requirement could be that the first four bytes of any object participating in the state machine belong to the owner so I think we are on the same page.

Can a source object ever have more than one owner?

"Reserving the few bytes" thing seems more resonable. I thought of just reserving a single integer. You would initialise it when intialising the source and get it back once the source fires an event.

For example:

nn_timer_init (&self->retry_timer, NN_REQ_RETRY_TIMER);

And then, in the handler:

if (source_type == NN_REQ_RETRY_TIMER && event_type == NN_TIMER_TIMEOUT) {
timer = (struct nn_timer*) source;
….
}

If you have different handler functions for different events the code will be structured (presumably) like this: event_type=>source=>state. That kind of code is extremely hard to follow.

Reserve a few bytes at the beginning of the source object, the owner could then attach the type once before the state machine executes. This could also be used for other data such as the owner setting a flag on the source.

Create a state machine for a single type in the O(n) cases and have the source call into the handler for that machine, for example nn_cipc_handler_spic(…). If possible it would be nice eliminate the need for "if (source == &cipc->sipc)" without over complicating the state machine.

If source_type cannot be avoided and the owner cannot write to the source object even one time during the wiring/initialization phase, consider some type of context object that the owner hands to the source that the source must pass with each handler call, that context object could contain the source_type, source_ptr etc.

Reserving a few bytes in the source seems the most appealing, you could have an issue where a source object might have more than one owner, then you would have to consider how many owners a source object could have, but I could still see this being workable.

If you can explain why some of these options won't work it might provide some more insight.

Even more, I would use different handlers for different event types, instead of using just one handler per event source. This way you can add event-specific arguments to the handler.

As for handing of the control from cipc to sipc it's done here:

nn_sipc_start (&cipc->sipc, &cipc->usock);

What happens is that sipc object takes ownership of the usock object and redirects any events from it to itself. When the connection breaks, it hands ownership of the usock back to cipc object.

One unrelated thought: I am considering passing adding one parameter to the handler function. Currently it is (target,source_ptr, event_type). I am thinking of extending it to (target, source_type, source_ptr, event_type). The problem it solves is when the state machine owns an unlimited number of source objects, for example, a list of sockets. In such case checking that source_ptr is one of the objects in the list would require list traversal (O(n) operation). Any thoughts about that?

If this is correct you might consider a convention such as NN_USOCK_EVENT_CONNECTED, and it might even be useful distinguish between an action and event where an action ACTION_CONNECT is initiated and EVENT_CONNECTED moves the machine into STATE_CONNECTED, action and event would still reside at the same level in the state machine.

  case NN_CIPC_STATE_CONNECTING: if (source == &cipc->usock) { switch (type) { case NN_USOCK_CONNECTED: nn_sipc_start (&cipc->sipc, &cipc->usock); cipc->state = NN_CIPC_STATE_ACTIVE; return; case NN_USOCK_ERROR: nn_usock_stop (&cipc->usock); cipc->state = NN_CIPC_STATE_STOPPING_USOCK; return; default: nn_assert (0); } } nn_assert (0); 

Regarding the nested state machines:

For example, when cipc state machine is in ACTIVE state it hands the execution to sipc sub-state-machine.

I could not find the hand off in cipc.cs NN_CIPC_STATE_ACTIVE, can you point me to src?

I don't feel good about name 'type' myself. It's not another state. It's the type of the event being processed. So, a single source object can emit different kind of events, say a socket can emit SENT and RECEIVED events. 'type' argument is used to distinguish between the two. Suggestions for better fitting name are welcome.

As for state transition tables I have deliberately not used them. The rationale is the same as the rationale for not using code generation: State transitions tables make the code very hard to follow. Instead, I've opted for a single transition function with three nested levels of switches (in this order): the state, the source of the event, the type of the event. That makes it relatively easy to find your way through the state machine be simply scrolling the source code up and down.

As for nested state machines, I actually use them. For example, when cipc state machine is in ACTIVE state it handles the execution to sipc sub-state-machine. When sipc state machine terminates it hands the execution back to the cipc state machine. The same trick can be applied recursively, getting an arbitrary stack depth.

The debug log is a pretty neat idea. It would be worth of implementing, I guess.

Great to hear you don't see any barrier to entry. That's what I was trying to achieve. Typically, when code generation or macros are used, people feel there's an barrier to overcome.

One thing I did find a little confusing was the nested "switch (type)" it looks like type is the state of another state machine, "switch (sock->state)".

There is also the possibility of using a state transition table and/or function lookup tables to remove some of the boiler plate. With this approach your default: handles all the common cases, like make a call and sets the next state and more complex cases get their own case statement "case NN_COMPLICATED_STATE:". If you can make the lookup arrays easy to read and maintain this could be a win. You could even case the boilerplate states and let them all fall through to the boilerplate lookup->call->set case and still maintain the default state for trapping invalid states. Lookup tables can also eliminate a lot of branching.

In even more complex state transitions a stack can be used to pass information between nested states, not sure that applies here, but it's useful in lex/parse.

Another nice advantage to the state machine approach is debugging, just log the states or even keep the last n states in a ring buffer.

I don't see any barrier to entry here, all the states and transitions are clearly defined in a hundred lines of very readable code. If you need any clarification, email or reply. I've have built quite a few DFSMs.

Although it means a lot of boilerplate code in the codebase (yuck!), on the other hand it allows any developer to peek directly at the source code and understand what's going on without to learn a new language.

Are you considering a code generator for nanomsg or a spec/api for the state machine?