Tuesday, June 19, 2012

Multi-Site OpenVPN Interconnect [Server-Client mode] + GRE tunnel over OpenVPN

Last time I connected two sites, with one site behind NAT router and had no direct Public access, so then I continued with the setup and wanted to add another node into my network and this time it was my another cousin's home I invaded, setup Vyatta at their home and created a 3-Site OpenVPN network in Server-Client fashion.This can grow from 3 to N sites if I need.

Here's the network topology I worked on.
Server-Client 3 Site OpenVPN network diagram
Now Starting from Main-Site Vyatta, create the required certificates and keys files for Server as well as the clients.

vyatta@Main-Vyatta:~$ sudo su -
root@Main-Vyatta:~# cd  /usr/share/doc/openvpn/examples/easy-rsa/2.0
root@Main-Vyatta:# cp * /etc/openvpn/
root@Main-Vyatta:# cd /etc/openvpn/
root@Main-Vyatta:# nano vars  

Edit the parameter
export KEY_DIR="/config/auth"

These parameters should be modified too.
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"

Save and Exit

Now execute
root@Main-Vyatta:# source ./vars
now Build CA

root@Main-Vyatta:# ./build-ca

Enter Data as inquired.

After this completes you should be able to have an output like this
root@Main-Vyatta:# ls /config/auth/
ca.crt  ca.key
Now time to Build Server side CA

root@Main-Vyatta:# touch /config/auth/index.txt
root@Main-Vyatta:# echo 01 > /config/auth/serial

root@Main-Vyatta:# ./build-key-server Main-Vyatta

This will ask at the end

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated

Next generate DH parameters file
root@Main-Vyatta:# ./build-dh

Now all set. Time to generate files for the Remote end points/Clients.

I've genrates Client files in password protected mode. This password will be asked whenever any client tries to use these files. If this matches then the setup will proceed on remote end.

root@Main-Vyatta:# ./build-key-pass site1

Again questions will be asked, type as you like.

Buliding another set of files for the second remote-site;

root@Main-Vyatta:# ./build-key-pass site2

At this point these are the contents of my /config/auth directory

root@Main-Vyatta:# ls /config/auth/
01.pem  03.pem           Main-Vyatta.csr  ca.crt  dh1024.pem  index.txt.attr      index.txt.old  serial.old  site1.csr  site2.crt  site2.key
02.pem  Main-Vyatta.crt  Main-Vyatta.key  ca.key  index.txt   index.txt.attr.old  serial         site1.crt   site1.key  site2.csr
root@Main-Vyatta:# exit

Do the last step of generating site certificates and keys for as many sites as could be in the network i.e "./build-key-pass siteN".I only had access to two distinct locations.

Now time to get back to Main-Router's Vyatta Console.

Explaining the lines below before it gets too complicated. I declared a server mode OpenVPN interface vtun0 on Main Vyatta. Set static IPs of each site into the node so they don't change their IPs on reboots and mess up with my static routes(see at the end). Also I declared the subnets on each site so vtun0 knows which subnet is located on which site.
Then I configured vtun0 to use the TLS keys and certificates for Main-Vyatta. Thats it. Server is done.

vyatta@Main-Vyatta:~$ configure
vyatta@Main-Vyatta# set interfaces openvpn vtun0 mode server
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server subnet
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server topology point-to-point
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server client site1 ip
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server client site1 subnet
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server client site2 ip
vyatta@Main-Vyatta# set interfaces openvpn vtun0 server client site2 subnet
vyatta@Main-Vyatta# set interfaces openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
vyatta@Main-Vyatta# set interfaces openvpn vtun0 tls cert-file /config/auth/Main-Vyatta.crt
vyatta@Main-Vyatta# set interfaces openvpn vtun0 tls dh-file /config/auth/dh1024.pem
vyatta@Main-Vyatta# set interfaces openvpn vtun0 tls key-file /config/auth/Main-Vyatta.key
vyatta@Main-Vyatta# commit

Now time to export the files to the Site-1 and Site-2 so they use their files accordingly.
This should be done in a more secure way, make sure we don't atleast loose the ca.crt file to any hacker.

vyatta@Main-Vyatta# sudo cp /config/auth/site* /tmp/
vyatta@Main-Vyatta# sudo chown vyatta /tmp/site*
vyatta@Main-Vyatta# sudo cp /config/auth/ca.crt /tmp/
vyatta@Main-Vyatta# sudo chown vyatta /tmp/ca.crt

Copy-pasted the files in /tmp directory and changed the ownership of files so that they can be fetched by remote ends via SCP.
Goto Site1 Router and fetch the files from the Main-Site:

Goto Linux root console

root@Site1-Vyatta~:# scp vyatta@* /tmp/
root@Site1-Vyatta~:# scp vyatta@ /tmp/
root@Site1-Vyatta~:# mv /tmp/* /config/auth/
root@Site1-Vyatta~:# exit

Back to Vyatta console

vyatta@Site1-Vyatta:#set interface openvpn vtun0 mode client
vyatta@Site1-Vyatta:#set interface openvpn vtun0 remote-host
vyatta@Site1-Vyatta:#set interface openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
vyatta@Site1-Vyatta:#set interface openvpn vtun0 tls cert-file /config/auth/site1.crt
vyatta@Site1-Vyatta:#set interface openvpn vtun0 tls key-file /config/auth/site1.key

In above steps I've only defined that the remote server is on IP and the mode of this OpenVPN interface vtun0 is client then certificates and keys are configured.

Once you commit it , it'll ask about the passphrase that we entered on server while creating it.


vyatta@Site1-Vyatta:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0                     u/u
eth1                     u/u
lo                                   u/u
vtun0                       u/u

Repeat the same steps of Site1 on Site2.

After this just add Static Routes in Main-Router for each subnets and Viola !!
vyatta@Main-Vyatta# set protocols static route next-hop
vyatta@Main-Vyatta# set protocols static route next-hop

vyatta@Main-Vyatta# run show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

S>* [1/0] via, eth0
S>* [1/0] via , vtun0
C>* is directly connected, lo
C>* is directly connected, eth0
K>* via, vtun0
C>* is directly connected, vtun0
C>* is directly connected, eth1
S>* [1/0] via, vtun0

vyatta@Main-Vyatta# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_req=1 ttl=64 time=2.25 ms
--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.250/2.250/2.250/0.000 ms

vyatta@Main-Vyatta# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_req=1 ttl=64 time=21.9 ms
64 bytes from icmp_req=2 ttl=64 time=3.29 ms
--- ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 3.297/12.603/21.909/9.306 ms

Thats it, we are done here. I did something interesting while doing it as well. Like Creating a GRE tunnel over OpenVPN.

GRE over OpenVPN

This has nothing to do with the static Routes. Once the OpenVPN is set and I was able to ping between the subnet I went to Main-Vyatta created a tun0 GRE interface and set the remote and local IPs to use for this tunnel.

vyatta@Main-Vyatta# set interfaces tunnel tun0 encapsulation gre
vyatta@Main-Vyatta# set interfaces tunnel tun0 remote-ip
vyatta@Main-Vyatta# set interfaces tunnel tun0 local-ip
vyatta@Main-Vyatta# set interfaces tunnel tun0 address

vyatta@Main-Vyatta# set interfaces tunnel tun1 address
vyatta@Main-Vyatta# set interfaces tunnel tun1 remote-ip
vyatta@Main-Vyatta# set interfaces tunnel tun1 local-ip
vyatta@Main-Vyatta# set interfaces tunnel tun1 encapsulation gre
vyatta@Main-Vyatta# commit

Then on the I ran the following commands

vyatta@Site1-Vyatta:~$ configure
vyatta@Site1-Vyatta#set interfaces tunnel tun0 address
vyatta@Site1-Vyatta#set interfaces tunnel tun0 local-ip
vyatta@Site1-Vyatta#set interfaces tunnel tun0 remote-ip
vyatta@Site1-Vyatta#set interfaces tunnel tun0 encapsulation gre

Then on the I ran the following commands

vyatta@Site2-Vyatta:~$ configure
vyatta@Site2-Vyatta#set interfaces tunnel tun0 address
vyatta@Site2-Vyatta#set interfaces tunnel tun0 local-ip
vyatta@Site2-Vyatta#set interfaces tunnel tun0 remote-ip
vyatta@Site2-Vyatta#set interfaces tunnel tun0 encapsulation gre

That's it I was able to ping between Main-Site IP: and Site-1, similarly site-2 was able to ping Main-Site tunnel interface :)

I've couple more ideas in mind to play with VPNs on Vyatta and then I'll revert back to VoIP Stuff.

Monday, June 18, 2012

OpenVPN with Vyatta [Site Behind NAT | Firewall]

Yesterday I was at my cousin's place and suddenly I remembered that I forgot to bring some documents from my desktop at home, going back home wasn't an option. I knew I've Static IP provisioned at home and at cousin's place its Dynamic IP modem.

It was a good thing that Vyatta controls my home internet traffic so I quickly accessed my vyata-router and from there fetching documents was a piece of cake.
Later I decided to create a VPN between me and my cousin's home so we could always access our "Shared Documents" without involving any hi-fi router/linux commands.

So here is the scenario:
Main Site/My Home: It listen for incoming OpenVPN requests and then listen for RIP messages.
Site-B/Cousin's Home: This initiates a VPN to main-site and shares the network routing table via RIP protocol.

Note that only Main Site haS static IP and other site has dynamic IP which changes on every reset maybe. So after this setup I will be able to access their home network without caring for their Public IPs.

Here is a network diagram for better understanding.
Site-to-Site OpenVPN tunnel topology

Main-Site Router:
Here's the break down of the commands you're about to see. Assign the Public IP, set the gateway address, set the public DNS server, start the SSH service (This is optional - don't do this to risk your router by making it's SSH accessible over the internet)

Then the LAN interface was configured, I verified that router's internet connectivity is good by resolving www.msn.com and then later I pinged it as well. All perfect at this point.

Main-Vyatta#set interfaces ethernet eth0 address
Main-Vyatta#set system gateway-address
Main-Vyatta#set system name-server
Main-Vyatta#set service ssh
Main-Vyatta#set interfaces ethernet eth1 address
Main-Vyatta#sudo nslookup www.msn.com
Then I created NAT rules for my LAN stations to reach out internet by NAT'd to Public IP.  Main-Vyatta#set nat source rule 5 outbound-interface eth0 Main-Vyatta#set nat source rule 5 source address Main-Vyatta#set nat source rule 5 translation address masquerade Main-Vyatta#commit Main-Vyatta#exit

Uptil here, my Home network is all set. Now Starting the OpenVPN Setup.
Create openvpn key , copy it it temporary directory, change its user ownership.

Main-Vyatta:~$generate openvpn key /etc/openvpn/key.psk
Main-Vyatta:~$sudo cp /etc/openvpn/key.psk /tmp/
Main-Vyatta:~$sudo chown vyatta /tmp/key.psk
Main-Vyatta:~$ls -l /etc/openvpn/key.psk

Create openVPN interface vtun0 with a Local-IP to assign and a remote IP.

Main-Vyatta#set interfaces openvpn vtun0 mode site-to-site
Main-Vyatta#set interfaces openvpn vtun0 local-address
Main-Vyatta#set interfaces openvpn vtun0 remote-address
Main-Vyatta#set interfaces openvpn vtun0 shared-secret-key-file /etc/openvpn/key.psk

In last five lines above I've created a "vtun0" interface of type openvpn. Its mode is set to site-to-site. Main site is addressed as, other site have this as its remote site address, and very important step to use the key file.

Start RIP interface on Main router's vtun0 interface to accept other side routes.

Main-Vyatta#set protocols rip interface vtun0
Main-Vyatta#set protocols rip neighbor

Now, Site-B Router.

Site1-Vyatta#set interfaces ethernet eth0 address
Site1-Vyatta#set interfaces ethernet eth1
Site1-Vyatta#set system gateway-address
Site1-Vyatta#set service ssh

Configure the Site1 router's IP address, gateway is the DSL modem, name server, and NAT rules and interface for this LAN should be set as well, Im skipping those.

Fetch the OpenVPN Key for this site to use.

Site1-Vyatta#sudo scp vyatta@ /etc/openvpn/key.psk
Site1-Vyatta#sudo chown root:root /etc/openvpn/key.psk
Now, time to configure the "vtun0" interface. Once we commit this this router will try connecting to the Main-Site router.

Site1-Vyatta#set interfaces openvpn vtun0 mode site-to-site
Site1-Vyatta#set interfaces openvpn vtun0 local-address
Site1-Vyatta#set interfaces openvpn vtun0 remote-address
Site1-Vyatta#set interfaces openvpn vtun0 remote-host
Site1-Vyatta#set interfaces openvpn vtun0 shared-secret-key-file /etc/openvpn/key.psk
Site1-Vyatta$show interfaces

Executing the show interfaces will show the new interface.

Now, Time to advertise my local network to the Main-Site, this way the Main-Site router will get to know which next-hop to take for reaching to network.

Site1-Vyatta#set protocols rip interface eth0
Site1-Vyatta#set protocols rip network
Site1-Vyatta#set protocols rip neighbor
Site1-Vyatta#set protocols rip interface vtun0

Thats all, everything is set and should be working fine. Ping from Main-Site to Site-B is flowing smoothly.

I'm thinking of exploring different ways and types of creating VPN between two or more Vyatta routers. Hope to publish those soon too.

Saturday, June 16, 2012

Getting to know ICE

First things first, these slides belong to a very senior VoIP Guru - I am one of his big fan  and really admire his contribution to the world of VoIP, Mr. Saúl Ibarra Corretgé. He is a great helper and a huge resource on open user's mailing lists - addressing other's issues and keep on helping others until the issue is resolved.

Now back to the original topic. I know one day or another I will be working on ICE to overcome NAT issues so I viewed the blogs and books of the VoIP Experts and found these slides perfect for my tiny mind to grasp this complicated thing quite easily.

So I thought to spread the knowledge, after all that's exactly why these slides were created at the first place.
Very helpful for me atleast, I wish I find some video recording of this presentation and know all that's not in these slides. A very good point at slide-20 to take ICE into consideration before trying to change the SDPs via any other functions.

I always appreciate people who share their knowledge. This was just an intro for me. I know I'll be working with NAT problems soon enough so why not take a head start. I hope I will be posting some more things about NAT issues in VoIP and their solutions sometime.

Sunday, June 10, 2012

A Glass Painting - After a long break.

Just for the sake of change of mood and colours here. Finally finished this one today. I think I could've done better than this, but two years is a long time to rust one's skills. I just do drawing or paintings to keep myself busy and to keep the creative side alive ( And obviously, for someone very very special  )

The painting above is not my design, I just re-did it in glass paints. The very last glass painting, before this one, I  made was back in 2009. Here it is.

This one is totally my own - I used to have better brain back then . I've so many more drawings and paintings but I'll publish some more colours here when it gets dry here.

Thats all for now. Hope you guys liked it :)

P.S: IMHO, this is equal or more tougher than doing any of the VoIP stuff.

Monday, June 4, 2012

OpenSIPS as Load-Balancer for FreeSWITCH

With reference to my older posts in which I talked about increasing VoIP services capacity (with failover for load-balanced media-servers), then I tested the whole scenario using Kamailio and RTPproxy.This post, however, is replica of the above scenario but using OpenSIPS and RTPproxy.

I wanted to use MediaProxy  instead of RTPprxoy but media-Proxy unfortunately is not used for bridging RTPs from Public Internet to Private subnets and vice-versa [For details: See this forum discussion]

Following is the topology I wanted to achieve.

OpenSIPS+RTPproxy in Action
I used FreeSWITCH as my Media-Server layer. Asterisk servers can be used alternatively as well.
Now moving quickly to the opensips.cfg file changes required in order to make this work.

First add,


in Global Parameters, This will tell opensips that server has multiple interfaces to send/receive and relay traffic in-between subnets, so modify headers wisely and accordingly.

Then following modules needs to be declared.

(click on underlined modules to read their documentation)

loadmodule "rtpproxy.so"
loadmodule "nathelper.so"
loadmodule "dialog.so"
loadmodule "load_balancer.so"
loadmodule "avpops.so"

Then parameters for each modules:

# ------ RTP-proxy Parameters
#RTPproxy instance to be used: 9901 
modparam("rtpproxy", "rtpproxy_sock", "udp:") 

In my case, since using Virt.Environment, I consider to be my Public IP and transform into the internal Private subnet 192.168.30.x.

NOTE[1]: My assumption  of as Public IP has serious complications, as I understand, because the NAT-HELPER functions will always consider traffic from this subnet as Private rather public and this may've effect on OpenSIPS behavior, if used in production.
NOTE[2]: Please reply back in case if there is any discrepancy or suggestion in the configurations. This is a work in progress, so I apologize in case if you find anything incorrect. Changes and Suggestions are always welcome.

# ------ AVPOPS params --------
modparam("avpops","db_url", "mysql://opensips:opensipsrw@localhost/opensips")

# ------ NAT-Helper params ------
modparam("nathelper", "ping_nated_only", 1)
modparam("nathelper", "received_avp", "$avp(i:42)")
modparam("nathelper", "sipping_bflag", 7)
modparam("nathelper", "sipping_from", "sip:pinger@saevolgo.com")

modparam("load_balancer", "probing_interval", 30) 
modparam("load_balancer", "probing_reply_codes", "501, 404") 
modparam("load_balancer", "probing_from","sip:opensips@saevolgo.com")
modparam("load_balancer", "db_url","mysql://opensips:opensipsrw@localhost/opensips")

Now main routing logic:

Add Highlighted NAT test just at the start of main route. I put that just under the Max-FWD header validation check.

      if (!mf_process_maxfwd_header("10")) {
  sl_send_reply("483","Too Many Hops");
      if (nat_uac_test("19")) {
xlog("L_INFO","[$pr:$fU@$si:$sp]: NAT Detection Test-19 Passed Now Fixing-Nat for '$rm' \n");
  if (is_method("REGISTER")) {
  } else {
xlog("L_INFO","[$pr:$fU@$si:$sp]: NAT Fixed for '$rm' \n");
Soon after this I could see code Logic for "BYE" method. There I need to tell (on call hangup) RTPproxy to dis-engage its ports for this particular call. if (loose_route()) {
 if (is_method("BYE")) {
  setflag(1); # do accounting ...
  setflag(3); # ... even if the transaction fails

Then scrolling down at the end of the main route I could see this piece of code.
# do lookup with method filtering
xlog("L_NOTICE","[$pr:$fU@$si:$sp]: Call from '$fu' to '$ru' LOOKUP in 'Location' table\n");
 if (!lookup("location","m")) {
  switch ($retcode) {
   case -1:
   case -3:
    #t_reply("404", "Not Found");
xlog("L_NOTICE","[$pr:$fU@$si:$sp]: Call from '$fu' to '$ru' User Not Found - try Dialing Media-Server\n");
   case -2:
    sl_send_reply("405", "Method Not Allowed");
I added the highlighted lines. Basically at this point the opensips default configuration searches its online sip users and if the dialed destination string is an online user it sends the call out to that user.

If that user is offline, or the dialed string don't matches any user it sends "404 Not Found" back to the caller.

Now, What I'm doing over here is that once I find out that a user is offline,or is not a defined user then route the call to route(3) which is where I'll call in Load-Balancer and select a LB'd destination Media-Server and record a VoiceMail or do some other call routing.

 Just after route(3); I've called route(1);. Route[1] is the default route already defined which only relays the call towards the destination. So here are these routes: Add these lines just after your main route ends.
# Media-Server Route, Engage Load-Balancer and Select Server here.
 xlog("L_NOTICE","[$pr:$fU@$si:$sp]: This is Media-Server Route Use Load-balancer NOW!!\n");
 if (!load_balance("1","calls")) {
  sl_send_reply("500","Service full");
 xlog("L_NOTICE","[$pr:$fU@$si:$sp]: Selected destination Media-Server : $du\n");
See as simple as this to use Load-Balancer. But the real game for RTPproxy is handled in route[1] where I test if the selected destination of this call is one of my Media-Server defined in loadbalancer DB-table or not.

If results in True I engage RTPproxy using flags sequence IE, if not and call is coming in from my Media-Servers to outside use RTPproxy flags in EI sequence.

See this post for more information on making RTPproxy work for you.

Now, in route[1] I added the following Highlighted code and RTPproxy started working for me.

# for INVITEs enable some additional helper routes
 xlog("L_INFO","[$pr:$fU@$si:$sp]: SET flags for '$rm' & Relay the Message\n");
 if (is_method("INVITE")) {
  if (isflagset(5)) {
xlog("L_NOTICE","[$pr:$fU@$si:$sp]: Engage Media-Proxy here\n");
xlog("L_NOTICE","[$pr:$fU@$si:$sp]: InBound-OR-OutBound Call Check!!\n");
   if(avp_db_query("select dst_uri from load_balancer where dst_uri like '%$dd%'")){
   else if(avp_db_query("select dst_uri from load_balancer where dst_uri='sip:$si:$sp'")){



Now opensips.cfg is all done.  Don;t forget to add FreeSWITCH server IPs in the loadbalancer table.

# mysql -uopensips -popensipsrw opensips
mysql>INSERT INTO load_balancer (group_id,dst_uri,resources,probe_mode,description) VALUES (1,'sip:','calls=50',2,'FreeSWITCH-B Server');
mysql>INSERT INTO load_balancer (group_id,dst_uri,resources,probe_mode,description) VALUES (1,'sip:','calls=100',2,'FreeSWITCH-B Server');

Quit MySQL, now you can use the following commands to reload Load-balancer servers list in opensips on the fly and see the status of each FS server in opensips.

#opensipsctl fifo lb_reload
#opensipsctl fifo lb_list
This is not all done!!


You need to define a gateway for OpenSIPS in your FreeSWITCH external or itnernal profile per your settings.

FreeSWITCH-A:~# cd /usr/local/freeswitch/conf/sip_profiles/external/
FreeSWITCH-A:~# vim opensips.xml
Insert these Lines in this file:
  <gateway name="OpenSIPS">
  <param name="username" value=""/>
  <param name="from-user" value=""/>
  <param name="password" value="2007"/>
  <param name="proxy" value=""/>
  <param name="register" value="false"/>
  <param name="retry-seconds" value="10"/>
  <param name="caller-id-in-from" value="true"/>
  <param name="extension-in-contact" value="true"/>
  <param name="ping" value="25"/>
  <param name="inbound-late-negotiation" value="true"/>
  <param name="context" value="default"/>
Reload Sofia files in FreeSWITCH. I did a whole sofia module reload.

freeswitch@internal> reload mod_sofia

I also had to tell FreeSWITCH to allow my LAN IPs in acl.conf.xml located at 

<list name="lan" default="allow">
      <node type="deny" cidr=""/>

Save changes in that file and reload acl.

freeswitch@internal> reloadacl

Now make calls and see that calls are landing in your Public context, there take control of your incoming calls and land to your own context/xml files.

Thats all folks ! Hope you guys were really bored uptill here ;)