OpenSIPS Geo-Distributed Clustering & Data-Exchange

OpenSIPS Clusterer Module

Summary:

OpenSIPS 2.4 now allows users to exchange user data between clustered proxy nodes via proto_bin module. As a result the receiving Proxy can perform an action.

In this blog post I'll try to show how we can exchange that data from the opensips script.

USE CASE: Multi-Proxy cluster sharing User Locations and Querying Cluster to Find Users.

The Slideshow above shows a simple setup where multiple proxies can "Talk" to each other without using any other application as the communication platform.

Previously we could share data as a one-way pipe to SQL, or NoSQL applications. By one-way pipe I mean that OpenSIPS server can deposit data but won't do anything as a recipient. since there is no way to tell each opensips box that you are the recipient and need to do something.

OpenSIPS introduced the Event_route and EBR (Event Based Routing) some time ago but the custom events can not trigger on their own w/o developing some MI script. Obviously, the concept of adding more scripts and tools to make a fully functional distributed OpenSIPS environment is complicated to develop and maintain.

Clusterer module in older versions of OpenSIPS (pre-devel/2.4) was only capable to exchange data from certain module internally i.e dialog, usrloc - user has no control over what data to be sent and how to perform an action when such a data is received.

WHATS NEW

1 - OpenSIPS cluster can broadcast script based Queries and wait for replies, perform action based on reply!

• Cluster can detect and share IP of a hacker with other nodes to perform some action
• Update cluster about Location of a user, inquire about location of a user and send call to relevant proxy
• Cluster can update about an ongoing conference call, other boxes can route calls to the hosting proxy
• A Central OpenSIPS talking to Database on behalf of rest of the OpenSIPS cluster. SQL statements only reside there rest of the cluster only sends small text to expect a small reply. Useful incase of depositing CDRs.

2 - FreeSWITCH advance scripting:

     If you've multiple FreeSWITCH boxes behind an OpenSIPS and multiple such deployments needing integration then this is going to be perfect to collect an ESL event and Broadcast it from local OpenSIPS to rest of the regions. This could be a game changer for multi-region deployments.

TIME FOR ACTION

Lets see how this all can be done from inside the opensips.cfg

A complete sample config file is available here:
https://github.com/goharahmed/opensips-clusterer

Loading Modules

The "db_mode" param is set to 0 so we don't depend on DB to find other Cluster nodes, its my personal preference you can load all cluster nodes from MySQL DB and run mi-fifo command to reload the new node data.
What I've done is added two core nodes in the config so any new node just needs to point to any one of the core servers and everyone will be aware of the whole cluster topology.

Event_routing module are critical for this to work, they invoke config file script when a certain event is triggered.

Sharing Data

First we need to identify what data needs to be shared across the cluster. In the code snippet below I figure I need to tell other boxes if some hacker is trying to make Bruteforce Registration to users which are not defined in my DB. Also, I share the successfully registered user info.




Here are the exact routes responsible for Broadcasting the message string to the cluster.



Infact these Lines are the real deal here:

        $var(cl_id) = 1;
        $var(msg_str) = "I_LOVE_OPENSIPS";
        cluster_broadcast_req("$var(cl_id)", "$var(msg_str)", "$avp(tag)");

Since we are sharing Data so we need to carefully fill the details as to which cluster this info needs to be routed to and what message to be delivered. The Tag is filled in automatically and is very important for retrieval of any replies.

Remote Proxy Data Retrieval

Once this is executed all Online nodes of the cluster will receive this message in an Event Route "E_CLUSTERER_REQ_RECEIVED"

Remote Proxy Replying Back

The AVP $avp(rcv_msg) will contain the sent string and we can script what to do further with that. Similarly the $avp(source_id) variable will contain the NodeID of the sender proxy so if we need to reply back we can send a reply using the same TAG. It is NOT mandatory for the cluster nodes to reply to each and every broadcasted message btw. They can remain silent if there is nothing to be replied with.

Receiving Reply

Once remote proxy replies with the same tag (OpenSIPS internaly differentiates between a Request and a Reply) an event is raised "E_CLUSTERER_RPL_RECEIVED" -

In this Event Route we don't need to do much, this is just collection of replied data. The real work is done elsewhere.

 Finding Location of a User in Cluster

This is approximately how the code will look like for a call where user was not found registered and query was dispatched to whole cluster. The Querying OpenSIPS waits for an event to occur in async mode (wait w/o blocking resources for as long as needed)

That is pretty much all that is required to create a Talking-Cluster of OpenSIPS servers.

WARNING: This is a work in progress from OpenSIPS developers and should be tested heavily before deploying in production. Version 2.4 is expected to be released as stable production release by May-2018 or around.