EEP (formely HEP) Extensible Encapsulation Protocol with HOMER

EEP duplicates and IP datagram and encapsulates and sends for remote relatime monitoring for SIP specific alerts and notifications . HEP is popular among many SIP servers including Freeswitch , Opensips , kamailio , RTP engine as an external module .

  • intended for passive duplicated for remote collection
  • can be used for audit storage and analysis
  • does not alter the orignal datagram or headers

HOMER is Packet and Event capture system popular fpr VOIP/RTC Monitoring based on HEP/EEP (Extensible Encapsulation protocol)

SIP Server Integration

Homer and homer encapsulation protocl (HEP) integration with sip server brings the capabilities to SIP/SDP payload retention with precise timestamping better monitor and detect anomilies in call tarffic and events correlation of session ,logs , reports also the power to bring charts and statictics for SIP and RTP/RTCP packets etc. We read about sipcapture and sip trace modules in project sipcapture_siptrace_hep.

Both Kamailio and Opensips HEP Integration are structurally simmilar. In kamailio SIPCAPTURE [2] module enables support for –

● Monitoring/mirroring port
● IPIP encapsulation (ETHHDR+IPHDR+IPHDR+UDPHDR)
● HEP encapsulation protocol mode (HEP v1, v2, v3)

Figure Opensips Capturing ( credits http://www.opensips.org)

Figure showing Opensips integartion with external capturing agent via proxy agent ( which can be HOMER)

To achieve that, load and configure the SipCapture module in the routing script.

Snippets fro Kamailio Homer docker installation as a collector

git clone https://github.com/sipcapture/homer-docker.git
cd homer-docker
docker-compose build
docker-compose up

Outsnippets from screen while the installation takes place

Creating network "homer-docker_default" with the default driver
Creating volume "homer-docker_homer-data-semaphore" with default driver
Creating volume "homer-docker_homer-data-mysql" with default driver
Creating volume "homer-docker_homer-data-dashboard" with default driver
Pulling mysql (mysql:5.6)...
5.6: Pulling from library/mysql
...
Creating mysql ... done
Creating homer-webapp   ... done
Creating homer-cron      ... done
Creating homer-kamailio  ... done
Creating bootstrap-mysql ... done
Attaching to mysql, homer-webapp, bootstrap-mysql, homer-cron, homer-kamailio
....
homer-webapp | Homer web app, waiting for MySQL
homer-cron   | Homer cron container, waiting for MySQL
homer-kamailio | Kamailio, waiting for MySQL
bootstrap-mysql | Mysql is now running.
bootstrap-mysql | Beginning initial data load....
bootstrap-mysql | Creating Databases...
bootstrap-mysql | Creating Tables...
.....
omer-kamailio | Kamailio container detected MySQL is running & bootstrapped
homer-kamailio |  0(22) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-kamailio |  0(22) WARNING: <core> [core/socket_info.c:1315]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio | config file ok, exiting...
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on 
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases: 
homer-kamailio | 
homer-kamailio |  0(23) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-kamailio |  0(23) WARNING: <core> [core/socket_info.c:1315]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on 
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases: 
homer-kamailio | 
homer-kamailio |  0(23) INFO: sipcapture [sipcapture.c:480]: parse_table_names(): INFO: table name:sip_capture
...
homer-webapp | Homer web app container detected MySQL is running & bootstrapped
homer-webapp | Module php5 already enabled

Capture tools

Dialoge module

storing dialogs in mysql DB , requires initialising mysql

#!define WITH_MYSQL
...
#!ifdef WITH_MYSQL
loadmodule "db_mysql.so"
#!endif
...
#!ifdef WITH_MYSQL
# - database URL - used to connect to database server by modules such
#       as: auth_db, acc, usrloc, a.s.o.
#!ifndef DBURL
#!define DBURL "mysql://root:kamailio@localhost/kamailio"
#!endif
#!endif
loadmodule "dialog.so"
# ----- dialog params ------
modparam("dialog", "dlg_flag", 10)
modparam("dialog", "track_cseq_updates", 0)
modparam("dialog", "dlg_match_mode", 2)
modparam("dialog", "timeout_avp", "$avp(i:10)")
modparam("dialog", "enable_stats", 1)
modparam("dialog", "db_url", DBURL)
modparam("dialog", "db_mode", 1)
modparam("dialog", "db_update_period", 120)
modparam("dialog", "table_name", "dialog")

seting db_mode – synchronisation of dialog information from memory to an underlying database has following options
0 – NO_DB – the memory content is not flushed into DB;
1 – REALTIME – any dialog information changes will be reflected into the database immediately.
2 – DELAYED – the dialog information changes will be flushed into DB periodically, based on a timer routine.
3 – SHUTDOWN – the dialog information will be flushed into DB only at shutdown – no runtime updates.

note :

  • use the same hash_size while using diff kamailio to restore dialogs

database table for dialogue

  1. install mysql
  2. define root ( with db create permissions ) and user ( with database read wrote ) permission in kamctlrc
vi /usr/local/etc/kamailio/kamctlrc
  • Dialogue table schema *
name type size default null key extra attributes description
id unsigned int 10 no primary autoincrement unique ID
hash_entry unsigned int 10 no Number of the hash entry in the dialog hash table
hash_id unsigned int 10 no The ID on the hash entry
callid string 255 no Call-ID of the dialog
from_uri string 128 no URI of the FROM header (as per INVITE)
from_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog.
to_uri string 128 no URI of the TO header (as per INVITE)
to_tag string 64 no identify a dialog, which is the combination of the Call-ID along with two tags, one from participant in the dialog.
caller_cseq string 20 no Last Cseq number on the caller side.
callee_cseq string 20 no Last Cseq number on the caller side.
caller_route_set string 512 yes Route set on the caller side.
callee_route_set string 512 yes Route set on on the caller side.
caller_contact string 128 no Caller's contact uri.
callee_contact string 128 no Callee's contact uri.
caller_sock string 64 no Local socket used to communicate with caller
callee_sock string 64 no Local socket used to communicate with callee
state unsigned int 10 no The state of the dialog.
start_time unsigned int 10 no The timestamp (unix time) when the dialog was confirmed.
timeout unsigned int 10 0 no The timestamp (unix time) when the dialog will expire.
sflags unsigned int 10 0 no The flags to set for dialog and accesible from config file.
iflags unsigned int 10 0 no The internal flags for dialog.
toroute_name string 32 yes The name of route to be executed at dialog timeout.
req_uri string 128 no The URI of initial request in dialog
xdata string 512 yes Extra data associated to the dialog (e.g., serialized profiles).

Siptrace module

SIPtrace module offer a possibility to store incoming and outgoing SIP messages in a database and/or duplicate to the capturing server (using HEP, the Homer encapsulation protocol, or plain SIP mode).

loadmodule "siptrace.so"
modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060")
modparam("siptrace", "hep_mode_on", 1)
modparam("siptrace", "trace_to_database", 0)
modparam("siptrace", "trace_flag", 22)
modparam("siptrace", "trace_on", 1)

integrating iut with request route to start duplicating the sip messages

sip_trace();
setflag(22);

  • trace_mode * 1 – uses core events triggered when receiving or sending SIP traffic to mirror traffic to a SIP capture server using HEP 0 – no automatic mirroring of SIP traffic via HEP.

duplicate

address in form of a SIP URI where to send a duplicate of traced message. It uses UDP all the time.

modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060")

to check the duplicate messages arriving

ngrep -W byline -d any port 9060 -q

RPC commands

Can ruen sip trace on or off

kamcmd> siptrace.status on   
Enabled

and to check

kamcmd> siptrace.status check
Enabled

Store sip_trace in database

modparam("siptrace", "trace_to_database", 1)
modparam("siptrace", "db_url", DBURL)
modparam("siptrace", "table", "sip_trace")

where the sip_trace tabel description is

+-------------+------------------+------+-----+---------------------+----------------+
| Field       | Type             | Null | Key | Default             | Extra          |
+-------------+------------------+------+-----+---------------------+----------------+
| id          | int(10) unsigned | NO   | PRI | NULL                | auto_increment |
| time_stamp  | datetime         | NO   | MUL | 2000-01-01 00:00:01 |                |
| time_us     | int(10) unsigned | NO   |     | 0                   |                |
| callid      | varchar(255)     | NO   | MUL |                     |                |
| traced_user | varchar(128)     | NO   | MUL |                     |                |
| msg         | mediumtext       | NO   |     | NULL                |                |
| method      | varchar(50)      | NO   |     |                     |                |
| status      | varchar(128)     | NO   |     |                     |                |
| fromip      | varchar(50)      | NO   | MUL |                     |                |
| toip        | varchar(50)      | NO   |     |                     |                |
| fromtag     | varchar(64)      | NO   |     |                     |                |
| totag       | varchar(64)      | NO   |     |                     |                |
| direction   | varchar(4)       | NO   |     |                     |                |
+-------------+------------------+------+-----+---------------------+----------------+

sample databse storage for sip traces

select * from sip_trace;

| id | time_stamp          | time_us | callid  | traced_user | msg         | method | status | fromip                   | toip                     | fromtag  | totag    | direction |
+----+---------------------+---------+---------------------------------------------+-------------+-----------------------------------
|  1 | 2019-07-18 09:00:18 |  417484 | MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM |             | INVITE sip:altanai@sip_addr;transport=udp SIP/2.0
Via: SIP/2.0/UDP local_addr:25584;branch=z9hG4bK-d8754z-1f5a337092a84122-1---d8754z-;rport
Max-Forwards: 70
Contact: <sip:derek@call_addr:7086;transport=udp>
To: <sip:altanai@sip_addr>
From: <sip:derek@sip_addr>;tag=de523549
Call-ID: MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM
CSeq: 1 INVITE
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, NOTIFY, MESSAGE, SUBSCRIBE, INFO
Content-Type: application/sdp
Supported: replaces
User-Agent: Bria 3 release 3.5.5 stamp 71243
Content-Length: 214

v=0
o=- 1563440415743829 1 IN IP4 local_addr
s=Bria 3 release 3.5.5 stamp 71243
c=IN IP4 local_addr
t=0 0
m=audio 59814 RTP/AVP 9 8 0 101
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv                                                                                                                                                                                      | INVITE |        | udp:caller_addr:27982 | udp:sip_pvt_addr:5060   | de523549 |          | in        |

|  2 | 2019-07-18 09:00:18 |  421675 | MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM |             | SIP/2.0 100 trying -- your call is important to us
Via: SIP/2.0/UDP local_addr:25584;branch=z9hG4bK-d8754z-1f5a337092a84122-1---d8754z-;rport=27982;received=caller_addr
To: <sip:altanai@sip_addr>
From: <sip:derek@sip_addr>;tag=de523549
Call-ID: MTlhY2VmNDdjN2QxZGM5ZDFhMWRhZThhZDU4YjE0MGM
CSeq: 1 INVITE
Server: kamailio (5.2.3 (x86_64/linux))
Content-Length: 0                                                                                                                                                                                                                                                                                                                                                                                                                                                           | ACK    |        | udp:caller_addr:27982 | udp:local_addr:5060   | de523549 | b2d8ad3f | in       |
...
+----+---------------------+---------+---------------------------------------------+-------------+-----------------------------------

Heplify

Multi-Protocol Go HEP Capture Agent made   https://github.com/sipcapture/heplify

wget https://dl.google.com/go/go1.11.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.11.2.linux-amd64.tar.gz

move package to /usr/local/go

mv go 

Either add go bin to ~/.profile

export PATH=$PATH:/usr/local/go/bin

and apply

source ~/.profile

or set GO ROOT , and GOPATH

export GOROOT=/usr/local/go
export GOPATH=$HOME/heplify
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

installation of dependencies

go get

clone heplify repo and make

make 

CAPTAGENT

New OSS Capture-Agent framework with capture suitable for SIP, XMPP and more. With internal method filtering , encryption and authetication this does look very promising howevr since I have perosnally not tried it yet , I will leave this space TBD for future

sngrep

https://github.com/irontec/sngrep

Other include Sipgrep , HEPipe and nProbe

HEPop

Multi-Protocol HEP Server & Switch in NodeJS. stand-alone HEP Capture Server designed for HOMER7 capable of emitting indexed datasets and tagged timeseries to multiple backends

https://github.com/sipcapture/HEPop

node hepop.js -c /app/myconfig.js

PCAP monitoring -> Homer Server -> Notification and Fraud Prevention

A realtime monitoring and alerting setup fom homer can best safeguard on VoIP specific attacks and suspecious activity by early warning . Some list of attacks such as DDOS , SIP SQL injections , parser , remote manipulation hijacking as cell as resource enumeration are common ifor a cloud telephony provider.

Adiitionally homer provide session quality using varables that include [1]

SD = Session Defects
[SUM(500,503,504)]

ISA = Ineffective Session Attempts
[SUM(408,500,503)]

AHR = Average HOP Requests

ASR = Answer Seizure Ratio
[(‘200’ / (INVITES – AUTH – SUM(3XX))) * 100]

NER = Network Efficiency Ratio
[(‘200’ + (‘486′,’487′,’603’) / (INVITES -AUTH-(SUM(30x)) * 100]

HOMER Web Interface or Custom Dashboard

Some more visualization for inter team communication such as NOC team can include

Homer Integration with influx DB

time series Reltiem DB install

wget https://dl.influxdata.com/influxdb/releases/influxdb_1.7.7_amd64.deb
sudo dpkg -i influxdb_1.7.7_amd64.deb

start

 >influxd
 8888888           .d888 888                   8888888b.  888888b.
   888            d88P"  888                   888  "Y88b 888  "88b
   888            888    888                   888    888 888  .88P
   888   88888b.  888888 888 888  888 888  888 888    888 8888888K.
   888   888 "88b 888    888 888  888  Y8bd8P' 888    888 888  "Y88b
   888   888  888 888    888 888  888   X88K   888    888 888    888
   888   888  888 888    888 Y88b 888 .d8""8b. 888  .d88P 888   d88P
 8888888 888  888 888    888  "Y88888 888  888 8888888P"  8888888P"

2019-07-19T07:03:04.603494Z	info	InfluxDB starting	{"log_id": "0GjGVvbW000", "version": "1.7.7", "branch": "1.7", "commit": "f8fdf652f348fc9980997fe1c972e2b79ddd13b0"}
2019-07-19T07:03:04.603756Z	info	Go runtime	{"log_id": "0GjGVvbW000", "version": "go1.11", "maxprocs": 1}
2019-07-19T07:03:04.707567Z	info	Using data dir	{"log_id": "0GjGVvbW000", "service": "store", "path": "/var/lib/influxdb/data"}

Home Integration with grafana

-tbd

For Kamailio integration follow github instructions on https://github.com/altanai/kamailioexamples

References :

[1] https://www.kamailio.org/events/2013-KamailioWorld/13-Alexandr.Dubovikov-Homer-SIP-Capture.pdf

[2] HEP/EEP – https://github.com/sipcapture/hep

[3] kamailio sipdump module – https://www.kamailio.org/docs/modules/devel/modules/sipdump.html

[4] https://github.com/sipcapture/HEPop

[5] HOMER Big Data – https://github.com/sipcapture/homer/wiki/Homer-Bigdata

General Data Protection Regulation (GDPR) in VoIP

GDPR, Europe’s digital privacy legislation passed in 2018, replaces the 1995 EU Data Protection Directive. It is rules designed to give EU citizens more control over their personal data & strengthen privacy rights. It aims to simplify the regulatory environment for business and citizens.

To read about other Certificates , compliances and Security in VoIP which summaries

  • HIPAA (Health Insurance Portability and Accountability Act) ,
  • SOX( Sarbanes Oxley Act of 2002),
  • Privacy Related Compliance certificates like COPPA (Children’s Online Privacy Protection Act ) of 1998,
  • CPNI (Customer Proprietary Network Information) 2007,
  • GDPR (General Data Protection Regulation)  in European Union 2018,
  • California Consumer Privacy Act (CCPA) 2019,
  • Personal Data Protection Bill (PDP) – India 2018 and
  • also specifications against Robocalls and SPIT ( SPAM over Internet Telephony) among others

Multinational companies will predominantly be regulated by the supervisory authority where they have their “main establishment” or headquarter. However, the issue concerning GDPR is that it not only applies to any organisation operating within the EU, but also to any organisations outside of the EU which offer goods or services to customers or businesses in the EU.

Key Principles of GDPR are

  • Lawfulness, fairness and transparency
  • Purpose limitation
  • Data minimisation
  • Accuracy
  • Storage limitation
  • Integrity and confidentiality (security)
  • Accountability

GDPR consists of 7 projects (DPO, Impact assessment, Portability, Notification of violations, Consent, Profiling, Certification and Lead authority) that will strengthen the control of personal data throughout the European Union.

Stakeholders

stakeholders of data protection regulation are
Data Subject – an individual, a resident of the European Union, whose personal data are to be protected

Data Controller – an institution, business or a person processing the personal data e.g. e-commerce website.

Data Protection Officer – a person appointed by the Data Controller responsible for overseeing data protection practices.

Data Processor – a subject (company, institution) processing a data on behalf of the controller. It can be an online CRM app or company storing data in the cloud.

Data Authority – a public institution monitoring implementation of the regulations in the specific EU member country.

Extra-Territorial Scope

Any VoIP service provider may feel that since they are not based out of EU such as officially headquartered in the Asia Pacific or US region they may not be legally binding to GDPR. However, GDPR expands the territorial and material scope of EU data protection law.  It applies to both controllers and processors established in the EU, and those outside the EU, who offer goods or services to or monitor EU data subject.

VoIP service providers as Data Processors

A processor is a “person, public authority, agency or other body which processes personal data on behalf of the controller”.
Most VoIP service providers are multinational in nature with services offered directly or indirectly to all regions. The GDPR imposes direct statutory obligations on data processors, which means they will be subject to direct enforcement by supervisory authorities, fines, and compensation claims by data subjects. However, a processor’s liability will be limited to the extent that it has not complied with it’s statutory and contractual obligations.

Data minimization – It is now a good practise to store and process as less user’s personal data as necessary to render our services effectively. Also to maintain data for only a stipulated time ( approx 90 days of CDR for call details and logs )

Record Keeping, Accountability and governance

To show compliance with GDPR, a service provider maintain detailed records of processing activities. Also, they must implement technological and organisational measures to ensure, and be able to demonstrate, that processing is performed in accordance with the GDPR. Some ways to apply these are :

  • Contracts: putting written contracts in place with organisations that process personal data on your behalf
  • maintaining documentation of your processing activities
  • Organisational policies focus on Data protection by design and default – two-factor auth, strong passwords to guard against brute-force, encryption, focus on security in architecture
  • Risk analysis and impact assessments: for uses of personal data that are likely to result in a high risk to individuals’ interests
  • Audit by Data protection officer
  • Clear Codes of conduct
  • Certifications

As for a VOIP landscape thankfully every call or message session is followed by a CDR ( Calld Detail Record ) or MDR ( Message Detail Record).

Additionally, assign a unique signature to every data-access client the VoIP system and log every read/write operation carried out on data stores whether persistent datastores or system caches.

Privacy Notices to Subjects

User profile data such as :

  • Basic identity information, name, address and ID numbers
  • Web data such as location, IP address, cookie data and RFID tags
  • Health and genetic data
  • Bio-metric data
  • Racial or ethnic data
  • Political opinions
  • Sexual orientation

is protected strictly under GDPR rules

A service provider should provide indepth information to data subjects when collecting their personal data, to ensure fairness and transparency. They must provide the information in an easily accessible form, using clear and plain language.

Consent

The GDPR introduces a higher bar for relying on consent , requiring clear affirmative action. Silence, pre ticked boxes or inactivity will not be sufficient to constitute consent. Data subjects can withdraw their consent at any time, and it must be easy for them to do so.

Lawful basis for processing Data now include

In Article 6 of the GDPR , there are six available lawful bases for processing.

(a) Consent: the individual has given clear consent for you to process their personal data for a specific purpose.

(b) Contract: the processing is necessary for a contract you have with the individual, or because they have asked you to take specific steps before entering into a contract.

(c) Legal obligation: the processing is necessary for you to comply with the law (not including contractual obligations).

(d) Vital interests: the processing is necessary to protect someone’s life.

(e) Public task: the processing is necessary for you to perform a task in the public interest or for your official functions, and the task or function has a clear basis in law.

(f) Legitimate interests: the processing is necessary for your legitimate interests or the legitimate interests of a third party, unless there is a good reason to protect the individual’s personal data which overrides those legitimate interests.

File such as PCAPS , Recordings and transcripts of calls hold sensitive information from end users , these should be encryoted and inaccssible to even the dev teams within the org without explicit consent of end user .

Individuals’ Rights

The GDPR provides individuals with new and enhanced rights to Data subjects who will have more control over the processing of their personal data. A data subject access request can only be refused if it is manifestly unfounded or excessive, in particular because of its repetitive character.

Rights of Data Subjets include

  • Right of Access
  • Right to Rectification
  • Right to Be Forgotten
  • Right to Restriction of Processing
  • Right to Data Portability
  • Right to Object
  • Right to Object to Automated Decisionmaking

For a VoIP service provider if a user opts for redaction then none of his calls or messages should be traced in logs . Also replace distinguishable end user identifier such as phone number and sip uri with *** charecters

Provide option for “Account Deletion” and purge account – If a user wished to close his/her account , his/her detaisl should be deleted form the sustem except for the bare bones detaisl which are otherwise required for legal , taxation and accounting requirnments

Breach Notification

A controller is a “person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of processing of personal data”,

A controller will have a mandatory obligation to notify his supervisory authority of a data breach within 72 hours unless the breach is unlikely to result in a risk to the rights of data subjects. Will also have to notify affected data subjects where the breach is likely to result in a “high risk” to their rights. A processor, however, will only be obliged to report data breaches to controllers

International Data Transfers

Data transfers to countries outside the EEA(European Economic Area) continue to be prohibited unless that country ensures an adequate level of protection. The GDPR retains existing transfer mechanisms and provides for additional mechanisms, including approved codes of conduct and certification schemes.

The GDPR prohibits any non-EU court, tribunal or regulator from ordering the disclosure of personal data from EU companies unless it requests such disclosure under an international agreement, such as a mutual legal assistance treaty.

One of the biggest challenges for a service provider is the identification & categorization of GDPR impacted data sets in disparate locations across the enterprise. A dev team must flag tables, attributes and other data objects that are categorically covered under GDPR regulations and then ensure that they are not transferred to a server outside of EU.

In the present age of Virtual shared server instance, cloud computing and VoIP protocol it is operational a very tough task for a communication service provider to ensure that data is not transferred outside of EU such as a VoIP call from origination in US and destination in EU will require information exchanges via SDP, vcard , RTP stream via media proxies etc.

Sanctions

The GDPR provides supervisory authorities with wide-ranging powers to enforce compliance, including the power to impose significant fines. You will face fines of up to €20m or 4% of your total worldwide annual turnover of the preceding financial year. In addition, data subjects can sue you for pecuniary or non-pecuniary damages (i.e. distress). Supervisory authorities will have a discretion as to whether to impose a fine and the level of that fine.

Data Protection officer (DPO)

Under the terms of GDPR, an organisation must appoint a Data Protection Officer (DPO) if it carries out large-scale processing of special categories of data, carries out large scale monitoring of individuals such as behaviour tracking or is a public authority.

Reference :

Attacks on SIP Networks

Major standards bodies including 3GPP, ITU-T, and ETSI have all adopted SIP as the core signalling protocol for services such as LTE , VoIP, conferencing, Video on Demand (VoD), IPTV (Internet Television), presence, and Instant Messaging (IM) etc. With the continous evolution of SIP as the defacto VoIP protocol , we need to underatdn the risk mitigartion practices around it .

I have written about VoIP and security in these blogs before

For Security around web browser based calling via webrtc i have written

  • Webrtc Security , which describes browser threat modal , access to local resource , Same Orogin Policy (SOP) and Cross Resource Sharing ( CORS) as well as Location sharing , ICE , TUEN and threats to privacy with screen sharing , microgone camera long term access and probable mid call attacks .
  • Genric secrutity of web Application build around hosting platform of webrtc. Includs concepts like Identity management , browser security – cross site security amd clickjacking , Authetication of devices and applications , Media Encryption and regex checking.

Also Written about VoIP security at protocol level with SRTP /DTLS using TLS and specifically using available pre added security modules on kamailio SIP server. It describes Sanity checks , ACL lists with permissions , hiding topology details , countering Flood using pike and Fail2Ban as well as Traffic monitoring and detection .

In this article we will cover types of attacks on SIP systems

Types of attacks on SIP based systems

Registration Hijacking

malicious registrations on registrar by a third party who modifies From header field of a SIP request.

exmaple implementation :
attacker de-registers all existing contacts for a URI
attacker can also register their own device as the appropriate contact address, thereby directing all requests for the affected user to him

solution – Autheticaion of user

Impersonating a Server

attacker impersonates the remote server
user’s request can now be intercepted by some other party
user’s request may be forwarded to insecure locations

Solution –

confidentiality, integrity, and authentication of proxy servers

Proxy/redirect sever, and registrars SHOULD possess a site certificate issued by CA which could be validated by UA

Temparing Message bodies

If users are relying on SIP message bodies to communicate either of

  • session encryption keys for a media session
  • MIME bodies
  • SDP
  • encapsulated telephony signals
    Then the atackers on proxy server can modify the session key or can act as a man-in-the-middle and do eaves droppng

exmaple implementation :
attacker can point RTP media streams to a wiretapping device
can changes Subject header field to appear to users as spam

solution – end to end ecryption over TLS + Digest Authorization

mid-session threats like tearing down session

Request forging
attacker learns the params of the session like To , From tags etc then he can alter ongoing session parameters and even bring it down

example implementation :
attacker inserts a BYE in a ongoing session thereby tearing it down
can insert re INVITE and redierct the stream to wiretaping device

solution – authetication on every request
signing and encrypting of MIME bodies, and transference of credentials with S/MIME

Denial of Service and Amplification

DOS attacks – rendering a particular network element unavailable, usually by directing an excessive amount of network traffic at its interfaces.
dDOS – multiple network hosts to flood a target host with a large amount of network traffic.

Can be created by sending falsified sip requests to other parties such that numerous transactions originating in the backwards direction comes to the target server created congestion.

exmaple implementation :
attackers creates a falsified source IP address and a corresponding Via header field that identify a targeted host as the originator of the request. Then send this to large number of SIP network element . This geneerates DOS aimed at target.

attackers uses falsified Route header field values in a request that identify the target host and then send such messages to forking proxies that will amplify messaging sent to the target.

Flooding with register attacks can deplete available memory and disk resources of a registrar by registering huge numbers of bindings.
Flooding a stateful proxy server causes it to consume computational expense associated with processing a SIP transaction

Solution –
detect flooding and pike in traffic and use ipban to block
challenge questionable requests with only a single 401 (Unauthorized) or 407 (Proxy Authentication Required), forgoing the normal response retransmission algorithm, and thus behaving statelessly towards unauthenticated requests.

Security mchanisms

Full encryption vs hop by hop encrption

SIP mssages cannot be encrypted end-to-end in their entirety since
message fields such as the Request-URI, Route, and Via need to be visible to proxies in most network architectures
so that SIP requests are routed correctly.
proxy servers need to also update the message with via headers

Thus SIP uses low level security along with hop by hop encrption and auth headers to verify the identity of proxy servers

Transport and Network Layer Security

IPsec – used where set of hosts or administrative domains have an existing trust relationship with one another.

TLS – used where hop-by-hop security is required between hosts with no pre-existing trust association.

SIPS URI Scheme

Used as an address-of-record for a particular user, signifies that each hop over which the request is forwarded, must be secured with TLS

HTTP Authentication

Reuse of the HTTP Digest authentication via 401 and 407 response codes that implement challenge for autehtication
provides replay protection and one-way authentication.

S/MIME

allows SIP UAs to encrypt MIME bodies within SIP, securing these bodies end-to-end without affecting message headers.
provides end-to-end confidentiality and integrity for message bodies

nonce-count

provides replay protection

SIP over TLS

SIP messages can be secured using TLS. There is also TLS for Datagrams called DTLS.

Security of SIP signalling is different from security of protocols used in concert with SIP like RTP , RTCP. and that will be covered in later topics of this article.

TLS operation consists of two phases: handshake phase and bulk data encryption phase

Handshake phase

Prepare algorithm to be used during TLS session

Server Authentication

server sends its certificate to the client, which then verifies the certificate using a certificate authority’s (CA’s) public key.

Client Authentication

Server sends an additional CertificateRequest message to request the client’s certificate. The client responds with

  1. Certificate message containing the client certificate with the client public key and
  2. CertificateVerify message containing a digest signature of the handshake messages signed by clients private key

Server authenticates client by client’s public key , since only client holding correct private key can sign the message.

prepare the shared secret for bulk data encryption

client generate a pre_master_secret, and encrypt it using the server’s public key obtained from the server’s certificate. The server decrypts the pre_master_secret using its own private key.
Both the server and client then compute a master_secret they share based on the same pre_master_secret. The master_secret is further used to generate the shared symmetric keys for bulk data encryption and message authentication

Public key cryptographic operations such as RSA are much more expensive than shared key cryptography. This is why TLS uses public key cryptography to establish the shared secret key in the handshake phase, and then uses symmetric key cryptography with the negotiated shared secret as the data encryption key.

Stateless proxy servers do not maintain state information about the SIP session and therefore tend to be more scalable. However, many standard application functionalities, such as authentication, authorization, accounting, and call forking require the proxy server to operate in a stateful
mode by keeping different levels of session state information.

Steps :

  1. The SIP proxy server enforces proxy authentication with
    407 Proxy Authentication Required challenge.
  2. UAC provides credentials that verify its claimed identity (e.g., based on MD5 [34] digest algorithm) and retransmits in authorization header

Security of RTP

confidentiality protection of the RTP session and integrity protection of the RTP/RTCP packets requires source authentication of all the packets to ensure no man-in-the-middle (MITM) attack is taking place.

end to end media encryption – SRTP ( Secure RTP )

encodes the voice into encrypted IP packages and transport those via the internet from the transmitter  to receive

References

  • The Impact of TLS on SIP Server Performance – Charles Shen† Erich Nahum‡ Henning Schulzrinne† Charles Wright , Department of Computer Science, Columbia University,IBM T.J. Watson Research Center

Certificates, compliances and Security in VoIP

This article describes various Certificates and compliances, Bill and Acts on data privacy, Security and prevention of Robocalls as adopted by countries around the world pertaining to Interconnected VoIP providers, telecommunications services, wireless telephone companies etc

Compliance certificates by Industry types

HIPAA (Health Insurance Portability and Accountability Act)

Deals with privacy and security of personal medical records and electronic health care transaction

Applicability  : If voip company handles medical information

Includes : 

  • Not allowed Voice mail transcription
  • Should have End-to-End Encryption
  • Restrict  using unsecured WiFi networks to prevent Snooping
  • User security , strong password rules  and mandatory monthly change
  • Secure Firmware on VoIP phones
  • Maintaining Call and Access Logs

SOX( Sarbanes Oxley Act of 2002)

Also known as SOX, SarbOX or Public Company Accounting Reform and Investor Protection Act

Applicability : if managing the communications operations of a regulated, publicly traded company 

Includes : 

  • Retain records which include financial and other sensitive data
  • ways employees are provided or denied access to records or data based on their roles and responsibilities
  • do information audit by a trusted third party. 
  • Retention and deletion of files such as audio files like voicemails, text messages, video clips, declared paper records, storage, and logs of communications activities
  • Physical and digital security controls around cloud-based VoIP applications and the networks

Privacy Related Compliance certificates

COPPA (Children’s Online Privacy Protection Act ) of 1998 

prohibits deceptive marketing to children under the age of 13, or collecting personal information without disclosure to their parents. 

any information is to be passed on to a third party, must be easy for the child’s guardian to review and/or protect

2011 amendment  requires that the data collected was erased after a period of time,

2014 FTC issued guidelines that apps and app stores require “verifiable parental consent.”

CPNI (Customer Proprietary Network Information) 2007

CPNI (Customer Proprietary Network Information) in united states is the information that communication providers  acquire about their subscribers. This Individually identifiable information that is created by a customer’s relationship with a provider, such as data about the frequency, duration, and timing of calls, the information on a customer’s bill, and call identifying information. This processing information is governed strictly by FCC and certification should be renewed on an annual basis

Provider can pass along that information to marketers to sell other services, as long as the customer is notified

In 2007, the FCC explicitly extended the application of the Commission’s CPNI rules of the Telecommunications Act of 1996 to providers of interconnected VoIP service.

CALEA

Communications Assistance for Law Enforcement Act (CALEA) conduct electronic surveillance by imposing specific obligations on “telecommunications carriers” for assisting law enforcement, including delivering call interception and call identification functionality to the government with a minimum of interference to customer service and privacy.

Read more about CALEA and its roles in VoIP here Regulatory and Legal Considerations with WebRTC development

GDPR (General Data Protection Regulation)  in European Union 2018

Supersedes the 1995 Data Protection Directive

Establishes requirements of organizations that process data, defines the rights of individuals to manage their data, and outlines penalties for those who violate these rights.

No personal data may be processed unless this processing is done under one of six lawful bases specified by the regulation (consent, contract, public task, vital interest, legitimate interest or legal requirement). When the processing is based on consent the data subject has the right to revoke it at any time.

Controllers must notify Supervising Authorities (SA)s of a personal data breach within 72 hours of learning of the breach.

California Consumer Privacy Act (CCPA) 2019

consumer rights relating to the access to, deletion of, and sharing of personal information that is collected by businesses. 

Allows consumers to know whether their personal data is sold or disclosed , to whom .

Allows opt-out right for sales of personal information

Right to deletion – to request a business to delete any personal information about a consumer collected from that consumer

Personal Data Protection Bill (PDP) – India 2018

This bill introduces various private and sensitive protection frameworks  like restriction on retention of personal data, Right to correction and erasure (such as right to be forgotten) , Prohibition and transparency of processing of personal data. It also classifies data fiduciaries  including certain social media intermediaries. 

The Bill amends the Information Technology Act, 2000 to delete the provisions related to compensation payable by companies for failure to protect personal data.

Other data privacy acts similar to GDPR 

  • South Korea’s Personal Information Protection Act  2011
  • Brazil’s Lei Geral de Proteçao de Dados (LGPD)  2020
  • Privacy Amendment (Notifiable Data Breaches) to Australia’s Privacy Act 2018
  • Japan’s Act on Protection of Personal Information 2017
  • Thailand Personal Data Protection Act (PDPA) 2020

Features offered by VOIP companies for Data privacy 

  • Access Control & Logging
  • Auto Data Redaction / Account Deletion policy 
  • SIEM (Security information and event management) alerts 
  • Information security , Encrypted Storage For Recordings & Transcripts
  • Disclosing all third party services that are involved in data processing too
  • Role Based Access Control and 2 Factor Authentication
  • Data Security Audits and appointing  data protection officer to oversee GDPR compliance

Against Robocalls and SPIT ( SPAM over Internet Telephony)

 2009 Truth in Caller ID Act 

Telephone Consumer Protection Act of 1991

Implementation of Do not call registry against use of robocalls, automatic dialers, and other methods of communication

Do-Not-Call Implementation Act of 2003

if a business has an established relationship with a customer, it can continue to call them for up to 18 months. If a consumer calls the company, say, to ask for information about the product or service, the company has three months to get back to him.

if the customer asks to not receive calls, the company must stop calling, or be subject to fines.

Exemptions – Calls from a not-for-profit B organisation , informational messages as flight cancellations , Calls from sales and debt collectors etc

Personal Data Privacy and Security Act 2009

Implemented to curb  identity theft and computer hacking. Sensitive personal identifiable information includes : victim’s name, social security number, home address, fingerprint/biometrics data, date of birth, and bank account numbers.

Any company that is breached must notify the affected individuals by mail, telephone, or email, and the message must include information on the company and how to get in touch with credit reporting agencies

If the breach involves government or national security , company must also contact the Secret Service within fourteen days 

TRACED Act (Telephone Robocall Abuse Criminal Enforcement and Deterrence) 2019

Canadian Radio-television and Telecommunications Commission (CRTC) 2018 -32

A solution mechanism has already been standardised and active in adoption called STIR / SHAKEN ( Secure Telephony Identity Revisited / Signature-based Handling of Asserted information using toKENs) described in another article here.

Emergency services 

FCC E911 E911 / VoIP E911 rules

Unlike traditional telephone connections, which are tied to a physical location, VOIP’s packet switched technology allows a particular number to be anywhere making it more difficult for it to reach localised services like emergency numbers of Public Safety Answering Points (PSAPs) . Thus FCC regulations as well as the New and Emerging Technologies 911 Improvement Act of 2008 (NET 911 Act), interconnected VoIP providers are required to provide 911 and E911 service. 

Ref : 

WebRTC APIs

Contents

  • Media Capture and streams
  • Peer to peer Connection
    • RTCPeerConnection, RTCConfiguration, ICE, Offer/Answer, states
  • RTP Media API
    • RTCRtpSender
    • RTCRtpReceiver
    • RTCRtpTransreciver
    • SDP Semantics
  • RTCDTLS Transport
  • RTCIceCandidate
    • ICE gathering
  • RTCIceTransport Interface
  • Peer-to-peer Data API
  • Peer-to-peer DTMF
  • Statistics

Peer-to-peer connections

creates p2p communication channel

RTCConfiguration Dictionary

dictionary RTCConfiguration {
  sequence<RTCIceServer> iceServers;
  RTCIceTransportPolicy iceTransportPolicy;
  RTCBundlePolicy bundlePolicy;
  RTCRtcpMuxPolicy rtcpMuxPolicy;
  DOMString peerIdentity;
  sequence<RTCCertificate> certificates;
  [EnforceRange] octet iceCandidatePoolSize = 0;
};

RTCIceCredentialType Enum

enum RTCIceCredentialType {
  "password",
  "oauth"
};

supports OAuth 2.0 based authentication. The application, acting as the OAuth Client, is responsible for refreshing the credential information and updating the ICE Agent with fresh new credentials before the accessToken expires. The OAuth Client can use the RTCPeerConnection setConfiguration method to periodically refresh the TURN credentials.

RTCOAuthCredential Dictionary

Describes the OAuth auth credential information which is used by the STUN/TURN client (inside the ICE Agent) to authenticate against a STUN/TURN server

dictionary RTCOAuthCredential {
  required DOMString macKey;
  required DOMString accessToken;
};

RTCIceServer Dictionary

Describes the STUN and TURN servers that can be used by the ICE Agent to establish a connection with a peer.

dictionary RTCIceServer { required (DOMString or sequence<DOMString>) urls; DOMString username; (DOMString or RTCOAuthCredential) credential; RTCIceCredentialType credentialType = "password"; };

Example :

 [{urls: 'stun:stun1.example.net'},
  {urls: ['turns:turn.example.org', 'turn:turn.example.net'],
    username: 'user',
    credential: 'myPassword',
    credentialType: 'password'},
  {urls: 'turns:turn2.example.net',
    username: '22BIjxU93h/IgwEb',
    credential: {
      macKey: 'WmtzanB3ZW9peFhtdm42NzUzNG0=',
      accessToken: 'AAwg3kPHWPfvk9bDFL936wYvkoctMADzQ5VhNDgeMR3+ZlZ35byg972fW8QjpEl7bx91YLBPFsIhsxloWcXPhA=='
    },
    credentialType: 'oauth'}
];

RTCIceTransportPolicy Enum

ICE candidate policy [JSEP] to select candidates for the ICE connectivity checks

  • relay – use only media relay candidates such as candidates passing through a TURN server. It prevents the remote endpoint/unknown caller from learning the user’s IP addresses
  • all – ICE Agent can use any type of candidate when this value is specified.

RTCBundlePolicy Enum

  • balanced – Gather ICE candidates for each media type (audio, video, and data). If the remote endpoint is not bundle-aware, negotiate only one audio and video track on separate transports.
  • max-compat – Gather ICE candidates for each track. If the remote endpoint is not bundle-aware, negotiate all media tracks on separate transports.
  • max-bundle – Gather ICE candidates for only one track. If the remote endpoint is not bundle-aware, negotiate only one media track. If the remote endpoint is bundle-aware, all media tracks and data channels are bundled onto the same transport.

If the value of configuration.bundlePolicy is set and its value differs from the connection’s bundle policy, throw an InvalidModificationError.

RTCRtcpMuxPolicy Enum

what ICE candidates are gathered to support non-multiplexed RTCP.

  • negotiate – Gather ICE candidates for both RTP and RTCP candidates. If the remote-endpoint is capable of multiplexing RTCP, multiplex RTCP on the RTP candidates. If it is not, use both the RTP and RTCP candidates separately.
  • require – Gather ICE candidates only for RTP and multiplex RTCP on the RTP candidates. If the remote endpoint is not capable of rtcp-mux, session negotiation will fail.

If the value of configuration.rtcpMuxPolicy is set and its value differs from the connection’s rtcpMux policy, throw an InvalidModificationError. If the value is “negotiate” and the user agent does not implement non-muxed RTCP, throw a NotSupportedError.

Offer/Answer Options – VoiceActivityDetection

dictionary RTCOfferAnswerOptions {
  boolean voiceActivityDetection = true;
};

capable of detecting “silence”

dictionary RTCOfferOptions : RTCOfferAnswerOptions {
  boolean iceRestart = false;
};
dictionary RTCAnswerOptions : RTCOfferAnswerOptions {};

An RTCPeerConnection object has a signaling state, a connection state, an ICE gathering state, and an ICE connection state.

RTCSignalingState Enum

“stable”,
“have-local-offer”,
“have-remote-offer”,
“have-local-pranswer”,
“have-remote-pranswer”,
“closed”

RTCIceGatheringState Enum

“closed”,
“failed”,
“disconnected”,
“new”,
“connecting”,
“connected”

RTCIceConnectionState Enum

“closed”,
“failed”,
“disconnected”,
“new”,
“checking”,
“completed”,
“connected”

An RTCPeerConnection object has an operations chain which ensures that only one asynchronous operation in the chain executes concurrently.

Also an RTCPeerConnection object MUST not be garbage collected as long as any event can cause an event handler to be triggered on the object. When the object’s internal slot is true ie closed, no such event handler can be triggered and it is therefore safe to garbage collect the object.

RTCPeerConnection Interface

interface RTCPeerConnection : EventTarget {

Promise<RTCSessionDescriptionInit> createOffer(optional RTCOfferOptions options);

Promise<RTCSessionDescriptionInit> createAnswer(optional RTCAnswerOptions options);

Promise<void> setLocalDescription(optional RTCSessionDescriptionInit description);
  
readonly attribute RTCSessionDescription? localDescription;
readonly attribute RTCSessionDescription? currentLocalDescription;
readonly attribute RTCSessionDescription? pendingLocalDescription;
  
Promise<void> setRemoteDescription(optional RTCSessionDescriptionInit description);
  
readonly attribute RTCSessionDescription? remoteDescription;
readonly attribute RTCSessionDescription? currentRemoteDescription;
readonly attribute RTCSessionDescription? pendingRemoteDescription;

Promise<void> addIceCandidate(optional RTCIceCandidateInit candidate);
readonly attribute RTCSignalingState signalingState;
readonly attribute RTCIceGatheringState iceGatheringState;
readonly attribute RTCIceConnectionState iceConnectionState;
readonly attribute RTCPeerConnectionState connectionState;
readonly attribute boolean? canTrickleIceCandidates;
void restartIce();
static sequence<RTCIceServer> getDefaultIceServers();

RTCConfiguration getConfiguration();
void setConfiguration(RTCConfiguration configuration);
void close();

attribute EventHandler onnegotiationneeded;
attribute EventHandler onicecandidate;
attribute EventHandler onicecandidateerror;
attribute EventHandler onsignalingstatechange;
attribute EventHandler oniceconnectionstatechange;
attribute EventHandler onicegatheringstatechange;
attribute EventHandler onconnectionstatechange;
};

CreateOffer() – generates a blob of SDP that contains an RFC 3264 offer with the supported configurations for the session, including

  • descriptions of the local MediaStreamTracks attached to this RTCPeerConnection,
  • codec/RTP/RTCP capabilities
  • ICE agent (usernameFragment, password , local candiadtes etc )
  • DTLS connection
var pc = new RTCPeerConnection();
pc.createOffer({
     mandatory: {
        OfferToReceiveAudio: true,
        OfferToReceiveVideo: true
    },
    optional: [{
        VoiceActivityDetection: false
    }]
}).then(function(offer) {
	return pc.setLocalDescription(offer);
})
.then(function() {
// Send the offer to the remote through signaling server
})
.catch(handleError);

CreateAnswer() – generates an SDPanswer with the supported configuration for the session that is compatible with the parameters in the remote configuration

var pc = new RTCPeerConnection();
pc.createAnswer({
  OfferToReceiveAudio: true
  OfferToReceiveVideo: true
})
.then(function(answer) {
  return pc.setLocalDescription(answer);
})
.then(function() {
  // Send the answer to the remote through signaling server
})
.catch(handleError);

Codec preferences of an m= section’s associated transceiver is said to be the value of the RTCRtpTranceiver with the following filtering applied

  • If direction is “sendrecv”, exclude any codecs not included in the intersection of RTCRtpSender.getCapabilities(kind).codecs and RTCRtpReceiver.getCapabilities(kind).codecs.
  • If direction is “sendonly”, exclude any codecs not included in RTCRtpSender.getCapabilities(kind).codecs.
  • If direction is “recvonly”, exclude any codecs not included in RTCRtpReceiver.getCapabilities(kind).codecs.

Legacy Interface Extensions

partial interface RTCPeerConnection {
 
Promise<void> createOffer(
 RTCSessionDescriptionCallback successCallback,
 RTCPeerConnectionErrorCallback failureCallback,
 optional RTCOfferOptions options);
 
Promise<void> setLocalDescription(
optional RTCSessionDescriptionInit description,
VoidFunction successCallback,                                  RTCPeerConnectionErrorCallback failureCallback);

Promise<void> createAnswer(
RTCSessionDescriptionCallback successCallback,
RTCPeerConnectionErrorCallback failureCallback);
  
Promise<void> setRemoteDescription(
optional RTCSessionDescriptionInit description,
VoidFunction successCallback,
                                     RTCPeerConnectionErrorCallback failureCallback);
  
Promise<void> addIceCandidate(
RTCIceCandidateInit candidate,
VoidFunction successCallback,
RTCPeerConnectionErrorCallback failureCallback);
};

Session Description Model

enum RTCSdpType {
  "offer",
  "pranswer",
  "answer",
  "rollback"
};
interface RTCSessionDescription {
  readonly attribute RTCSdpType type;
  readonly attribute DOMString sdp;
  [Default] object toJSON();
};
dictionary RTCSessionDescriptionInit {
  RTCSdpType type;
  DOMString sdp = "";
};

RTCPriorityType
Priority and QoS Model which can be

“very-low”,
“low”,
“medium”,
“high”

RTP Media API

Send and receive MediaStreamTracks over a peer-to-peer connection.
Tracks, when added to an RTCPeerConnection, result in signaling; when this signaling is forwarded to a remote peer, it causes corresponding tracks to be created on the remote side.

RTCRtpSenders objects manages encoding and transmission of MediaStreamTracks .

RTCRtpReceivers objects manage the reception and decoding of MediaStreamTracks. These are associated with one track.

RTCRtpTransceivers interface describes a permanent pairing of an RTCRtpSender and an RTCRtpReceiver.

Each transceiver is uniquely identified using its mid ( media id) property from the corresponding m-line.

They are created implicitly when the application attaches a MediaStreamTrack to an RTCPeerConnection via the addTrack(), or explicitly when the application uses the addTransceiver(). They are also created when a remote description is applied that includes a new media description.

rtpTransceiver = RTCPeerConnection.addTransceiver(trackOrKind, init);

trackOrKind should be either audio or video othereise a TypeError is thrown

init is optional – can contain direction , sendEncodings , streams

RTCPeerConnection Interface

partial interface RTCPeerConnection {
  
sequence<RTCRtpSender> getSenders();
sequence<RTCRtpReceiver> getReceivers();
sequence<RTCRtpTransceiver> getTransceivers();

RTCRtpSender addTrack(MediaStreamTrack track, MediaStream... streams);
void removeTrack(RTCRtpSender sender);
RTCRtpTransceiver addTransceiver((MediaStreamTrack or DOMString) trackOrKind, optional RTCRtpTransceiverInit init);
attribute EventHandler ontrack;
};

RTCRtpTransceiverInit

dictionary RTCRtpTransceiverInit {
  RTCRtpTransceiverDirection direction = "sendrecv";
  sequence<MediaStream> streams = [];
  sequence<RTCRtpEncodingParameters> sendEncodings = [];
};

RTCRtpTransceiverDirection can be either of

“sendrecv”,
“sendonly”,
“recvonly”,
“inactive”,
“stopped”

RTCRtpSender Interface

Allows an application to control how a given MediaStreamTrack is encoded and transmitted to a remote peer.

interface RTCRtpSender {
readonly attribute MediaStreamTrack? track;
readonly attribute RTCDtlsTransport? transport;
readonly attribute RTCDtlsTransport? rtcpTransport;
static RTCRtpCapabilities? getCapabilities(DOMString kind);
Promise<void> setParameters(RTCRtpSendParameters parameters);
RTCRtpSendParameters getParameters();
Promise<void> replaceTrack(MediaStreamTrack? withTrack);
void setStreams(MediaStream... streams);
Promise<RTCStatsReport> getStats();
};

RTCRtpParameters Dictionary

dictionary RTCRtpParameters {
  required sequence<RTCRtpHeaderExtensionParameters> headerExtensions;
  required RTCRtcpParameters rtcp;
  required sequence<RTCRtpCodecParameters> codecs;
};

RTCRtpSendParameters Dictionary

dictionary RTCRtpSendParameters : RTCRtpParameters {
  required DOMString transactionId;
  required sequence<RTCRtpEncodingParameters> encodings;
  RTCDegradationPreference degradationPreference = "balanced";
  RTCPriorityType priority = "low";
};

RTCRtpReceiveParameters Dictionary

dictionary RTCRtpReceiveParameters : RTCRtpParameters {
  required sequence<RTCRtpDecodingParameters> encodings;
};

RTCRtpCodingParameters Dictionary

dictionary RTCRtpCodingParameters {
  DOMString rid;
};

RTCRtpDecodingParameters Dictionary

dictionary RTCRtpDecodingParameters : RTCRtpCodingParameters {};

RTCRtpEncodingParameters Dictionary

dictionary RTCRtpEncodingParameters : RTCRtpCodingParameters {
  octet codecPayloadType;
  RTCDtxStatus dtx;
  boolean active = true;
  unsigned long ptime;
  unsigned long maxBitrate;
  double maxFramerate;
  double scaleResolutionDownBy;
};

RTCDtxStatus Enum

disabled- Discontinuous transmission is disabled.
enabled- Discontinuous transmission is enabled if negotiated.

RTCDegradationPreference Enum

enum RTCDegradationPreference {
  "maintain-framerate",
  "maintain-resolution",
  "balanced"
};

RTCRtcpParameters Dictionary

dictionary RTCRtcpParameters {
  DOMString cname;
  boolean reducedSize;
};

RTCRtpHeaderExtensionParameters Dictionary

dictionary RTCRtpHeaderExtensionParameters {
  required DOMString uri;
  required unsigned short id;
  boolean encrypted = false;
};

RTCRtpCodecParameters Dictionary

dictionary RTCRtpCodecParameters {
  required octet payloadType;
  required DOMString mimeType;
  required unsigned long clockRate;
  unsigned short channels;
  DOMString sdpFmtpLine;
};

payloadType – identify this codec.
mimeType – codec MIME media type/subtype. Valid media types and subtypes are listed in [IANA-RTP-2]
clockRate – expressed in Hertz
channels – number of channels (mono=1, stereo=2).
sdpFmtpLine – “format specific parameters” field from the “a=fmtp” line in the SDP corresponding to the codec

RTCRtpCapabilities Dictionary

dictionary RTCRtpCapabilities {
  required sequence<RTCRtpCodecCapability> codecs;
  required sequence<RTCRtpHeaderExtensionCapability> headerExtensions;
};

RTCRtpCodecCapability Dictionary

dictionary RTCRtpCodecCapability {
  required DOMString mimeType;
  required unsigned long clockRate;
  unsigned short channels;
  DOMString sdpFmtpLine;
};

RTCRtpHeaderExtensionCapability Dictionary

dictionary RTCRtpHeaderExtensionCapability {
  DOMString uri;
};

Example JS code to RTCRtpCapabilities

const pc = new RTCPeerConnection();
const transceiver = pc.addTransceiver('audio');
const capabilities = RTCRtpSender.getCapabilities('audio');

Output :

codecs: Array(13)
0: {channels: 2, clockRate: 48000, mimeType: "audio/opus", sdpFmtpLine: "minptime=10;useinbandfec=1"}
1: {channels: 1, clockRate: 16000, mimeType: "audio/ISAC"}
2: {channels: 1, clockRate: 32000, mimeType: "audio/ISAC"}
3: {channels: 1, clockRate: 8000, mimeType: "audio/G722"}
4: {channels: 1, clockRate: 8000, mimeType: "audio/PCMU"}
5: {channels: 1, clockRate: 8000, mimeType: "audio/PCMA"}
6: {channels: 1, clockRate: 32000, mimeType: "audio/CN"}
7: {channels: 1, clockRate: 16000, mimeType: "audio/CN"}
8: {channels: 1, clockRate: 8000, mimeType: "audio/CN"}
9: {channels: 1, clockRate: 48000, mimeType: "audio/telephone-event"}
10: {channels: 1, clockRate: 32000, mimeType: "audio/telephone-event"}
11: {channels: 1, clockRate: 16000, mimeType: "audio/telephone-event"}
12: {channels: 1, clockRate: 8000, mimeType: "audio/telephone-event"}
length: 13

headerExtensions: Array(6)
0: {uri: "urn:ietf:params:rtp-hdrext:ssrc-audio-level"}
1: {uri: "http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time"}
2: {uri: "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01"}
3: {uri: "urn:ietf:params:rtp-hdrext:sdes:mid"}
4: {uri: "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id"}
5: {uri: "urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id"}
length: 6

RTCRtpReceiver Interface

allows an application to inspect the receipt of a MediaStreamTrack.

interface RTCRtpReceiver {
  readonly attribute MediaStreamTrack track;
  readonly attribute RTCDtlsTransport? transport;
  readonly attribute RTCDtlsTransport? rtcpTransport;
  static RTCRtpCapabilities? getCapabilities(DOMString kind);
  RTCRtpReceiveParameters getParameters();
  sequence<RTCRtpContributingSource> getContributingSources();
  sequence<RTCRtpSynchronizationSource> getSynchronizationSources();
  Promise<RTCStatsReport> getStats();
};

dictionary RTCRtpContributingSource

dictionary RTCRtpContributingSource {
  required DOMHighResTimeStamp timestamp;
  required unsigned long source;
  double audioLevel;
  required unsigned long rtpTimestamp;
};

dictionary RTCRtpSynchronizationSource

dictionary RTCRtpSynchronizationSource : RTCRtpContributingSource {
  boolean voiceActivityFlag;
};

voiceActivityFlag of type boolean – Only present for audio receivers. Whether the last RTP packet, delivered from this source, contains voice activity (true) or not (false).

RTCRtpTransceiver Interface

Each SDP media section describes one bidirectional SRTP (“Secure Real Time Protocol”) stream. RTCRtpTransceiver describes this permanent pairing of an RTCRtpSender and an RTCRtpReceiver, along with some shared state. It is uniquely identified using its mid property.

Thus it is combination of an RTCRtpSender and an RTCRtpReceiver that share a common mid. An associated transceiver( with mid) is one that’s represented in the last applied session description.

interface RTCRtpTransceiver {

readonly attribute DOMString? mid;
[SameObject] readonly attribute RTCRtpSender sender;
[SameObject] readonly attribute RTCRtpReceiver receiver;

attribute RTCRtpTransceiverDirection direction;
readonly attribute RTCRtpTransceiverDirection? currentDirection;

void stop();

void setCodecPreferences(sequence<RTCRtpCodecCapability> codecs);
};

Method stop() – Irreversibly marks the transceiver as stopping, unless it is already stopped. This will immediately cause the transceiver’s sender to no longer send, and its receiver to no longer receive.
stopping transceiver will cause future calls to createOffer to generate a zero port in the media description for the corresponding transceiver and stopped transceiver will cause future calls to createOffer or createAnswer to generate a zero port in the media description for the corresponding transceiver

Methods setCodecPreferences() – overrides the default codec preferences used by the user agent.

Example setting codec Preferebec for OPUS in audio

peer = new RTCPeerConnection();    
const transceiver = peer.addTransceiver('audio');
const audiocapabilities = RTCRtpSender.getCapabilities('audio');
let codec = [];
codec.push(audiocapabilities.codecs[0]);
transceiver.setCodecPreferences(codec);

Before setting codec preference for OPUS

m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0

a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000

After setting codec preference for OPUS audio

m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0

a=sendrecv
a=msid:hcgvWcGG7WhdzboWk79q39NiO8xkh4ArWhbM f15d77bb-7a6f-4f41-80cd-51a3c40de7b7
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1

RTCDtlsTransport

Access to information about the Datagram Transport Layer Security (DTLS) transport over which RTP and RTCP packets are sent and received by RTCRtpSender and RTCRtpReceiver objects, as well other data such as SCTP packets sent and received by data channels.
Each RTCDtlsTransport object represents the DTLS transport layer for the RTP or RTCP component of a specific RTCRtpTransceiver, or a group of RTCRtpTransceivers if such a group has been negotiated via [BUNDLE].

interface RTCDtlsTransport : EventTarget {
  [SameObject] readonly attribute RTCIceTransport iceTransport;
  readonly attribute RTCDtlsTransportState state;
  sequence<ArrayBuffer> getRemoteCertificates();
  attribute EventHandler onstatechange;
  attribute EventHandler onerror;
};

RTCDtlsTransportState Enum

“new”- DTLS has not started negotiating yet.
“connecting” – DTLS is in the process of negotiating a secure connection and verifying the remote fingerprint.
“connected”- DTLS has completed negotiation of a secure connection and verified the remote fingerprint.
“closed” – transport has been closed intentionally like close_notify alert, or calling close().
“failed” – transport has failed as the result of an error like failure to validate the remote fingerprint

RTCDtlsFingerprint dictionary

dictionary RTCDtlsFingerprint {
  DOMString algorithm;
  DOMString value;
};

Protocols multiplexed with RTP (e.g. data channel) share its component ID. This represents the component-id value 1 when encoded in candidate-attribute while ICE candadte for RTCP has component-id value 2 when encoded in candidate-attribute.

RTCTrackEvent

The track event uses the RTCTrackEvent interface.

interface RTCTrackEvent : Event {
  readonly attribute RTCRtpReceiver receiver;
  readonly attribute MediaStreamTrack track;
  [SameObject] readonly attribute FrozenArray<MediaStream> streams;
  readonly attribute RTCRtpTransceiver transceiver;
};

dictionary RTCTrackEventInit

dictionary RTCTrackEventInit : EventInit {
  required RTCRtpReceiver receiver;
  required MediaStreamTrack track;
  sequence<MediaStream> streams = [];
  required RTCRtpTransceiver transceiver;
};

RTCIceCandidate

This interface candidate Internet Connectivity Establishment (ICE) configuration used to setup RTCPeerconnection. To facilitate routing of media on given peer connection, both endpoints exchange several candidates and then one candidate out of the lot is chosen which will be then used to initiate the connection.

  • candidate – transport address for the candidate that can be used for connectivity checks.
  • component – candidate is an RTP or an RTCP candidate
  • foundation – unique identifier that is the same for any candidates of the same type , helps optimize ICE performance while prioritizing and correlating candidates that appear on multiple RTCIceTransport objects.
  • ip , port
  • priority
  • protocol – tcp/udp
  • relatedAddress , relatedPort
  • sdpMid – candidate’s media stream identification tag
  • sdpMLineIndex

usernameFragment – randomly-generated username fragment (“ice-ufrag”) which ICE uses for message integrity along with a randomly-generated password (“ice-pwd”).

Interfaces for Connectivity Establishment

describes ICE candidates

interface RTCIceCandidate {
  readonly attribute DOMString candidate;
  readonly attribute DOMString? sdpMid;
  readonly attribute unsigned short? sdpMLineIndex;
  readonly attribute DOMString? foundation;
  readonly attribute RTCIceComponent? component;
  readonly attribute unsigned long? priority;
  readonly attribute DOMString? address;
  readonly attribute RTCIceProtocol? protocol;
  readonly attribute unsigned short? port;
  readonly attribute RTCIceCandidateType? type;
  readonly attribute RTCIceTcpCandidateType? tcpType;
  readonly attribute DOMString? relatedAddress;
  readonly attribute unsigned short? relatedPort;
  readonly attribute DOMString? usernameFragment;
  RTCIceCandidateInit toJSON();
};

RTCIceProtocol can be either tcp or udp

TCP candidate type which can be either of

  • active – An active TCP candidate is one for which the transport will attempt to open an outbound connection but will not receive incoming connection requests.
  • passive – A passive TCP candidate is one for which the transport will receive incoming connection attempts but not attempt a connection.
  • so – An so candidate is one for which the transport will attempt to open a connection simultaneously with its peer.

UDP candidate type

  • host – actual direct IP address of the remote peer
  • srflx – server reflexive ,  generated by a STUN/TURN server
  • prflx – peer reflexive ,IP address comes from a symmetric NAT between the two peers, usually as an additional candidate during trickle ICE
  • relay – generated using TURN

ICE Candidate UDP Host

sdpMid: 0, sdpMLineIndex: 0, candidate: candidate:27784895 1 udp 2122260223 192.168.1.114 51577 typ host generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 1, sdpMLineIndex: 1, candidate: candidate:27784895 1 udp 2122260223 192.168.1.114 51382 typ host generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 2, sdpMLineIndex: 2, candidate: candidate:27784895 1 udp 2122260223 192.168.1.114 53600 typ host generation 0 ufrag muSq network-id 1 network-cost 10

ICE Candidate TCP Host

Notice TCP host camdidates for mid 0 , 1 and 3 for video , audio and data media types

sdpMid: 0, sdpMLineIndex: 0, candidate: candidate:1327761999 1 tcp 1518280447 192.168.1.114 9 typ host tcptype active generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 1, sdpMLineIndex: 1, candidate: candidate:1327761999 1 tcp 1518280447 192.168.1.114 9 typ host tcptype active generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 2, sdpMLineIndex: 2, candidate: candidate:1327761999 1 tcp 1518280447 192.168.1.114 9 typ host tcptype active generation 0 ufrag muSq network-id 1 network-cost 10

ICE Candidate UDP Srflx

Notice 3 candidates for 3 streams sdpMid 0,1 and 2

sdpMid: 2, sdpMLineIndex: 2, candidate: candidate:2163208203 1 udp 1686052607 117.201.90.218 27177 typ srflx raddr 192.168.1.114 rport 53600 generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 1, sdpMLineIndex: 1, candidate: candidate:2163208203 1 udp 1686052607 117.201.90.218 27176 typ srflx raddr 192.168.1.114 rport 51382 generation 0 ufrag muSq network-id 1 network-cost 10
sdpMid: 0, sdpMLineIndex: 0, candidate: candidate:2163208203 1 udp 1686052607 117.201.90.218 27175 typ srflx raddr 192.168.1.114 rport 51577 generation 0 ufrag muSq network-id 1 network-cost 10

ICE Candidate (host)

sdpMid: 0, sdpMLineIndex: 0, candidate: candidate:2880323124 1 udp 2122260223 192.168.1.116 61622 typ host generation 0 ufrag jsPO network-id 1 network-cost 10

usernameFragment – randomly-generated username fragment (“ice-ufrag”) which ICE uses for message integrity along with a randomly-generated password (“ice-pwd”).

a=ice-ufrag:D+yg
a=ice-pwd:2ep6CXO0FP/JfS4ue/dfjeQM
a=ice-options:trickle

RTCPeerConnectionIceEvent

interface RTCPeerConnectionIceEvent : Event {
  readonly attribute RTCIceCandidate? candidate;
  readonly attribute DOMString? url;
};

RTCPeerConnectionIceErrorEvent

interface RTCPeerConnectionIceErrorEvent : Event {
  readonly attribute DOMString hostCandidate;
  readonly attribute DOMString url;
  readonly attribute unsigned short errorCode;
  readonly attribute USVString errorText;
};

RTCIceTransport Interface

Access to information about the ICE transport over which packets are sent and received. Each RTCIceTransport object represents the ICE transport layer for the RTP or RTCP component of a specific RTCRtpTransceiver, or a group of RTCRtpTransceivers if such a group has been negotiated via [BUNDLE].

interface RTCIceTransport : EventTarget {
  readonly attribute RTCIceRole role;
  readonly attribute RTCIceComponent component;
  readonly attribute RTCIceTransportState state;
  readonly attribute RTCIceGathererState gatheringState;
  sequence<RTCIceCandidate> getLocalCandidates();
  sequence<RTCIceCandidate> getRemoteCandidates();
  RTCIceCandidatePair? getSelectedCandidatePair();
  RTCIceParameters? getLocalParameters();
  RTCIceParameters? getRemoteParameters();
  attribute EventHandler onstatechange;
  attribute EventHandler ongatheringstatechange;
  attribute EventHandler onselectedcandidatepairchange;
};

RTCIceParameters Dictionary

dictionary RTCIceParameters {
  DOMString usernameFragment;
  DOMString password;
};

RTCIceCandidatePair Dictionary

dictionary RTCIceCandidatePair {
  RTCIceCandidate local;
  RTCIceCandidate remote;
};

RTCIceGathererState Enum

“new”,
“gathering”,
“complete”

RTCIceTransportState Enum

  • “new” – ICE agent is gathering addresses or is waiting to be given remote candidates 
  • “checking” –
  • “connected” – Found a working candidate pair, but still performing connectivity checks to find a better one.
  • “completed” – Found a working candidate pair and done performing connectivity checks.
  • “disconnected”,
  • “failed”,
  • “closed”

RTCIceRole Enum

“unknown”, // agent who role is not yet defined
“controlling”, // controlling agent
“controlled” // controlled agent

RTCIceComponent Enum

“rtp”, // ICE Transport is used for RTP (or RTCP multiplexing)
“rtcp” // ICE Transport is used for RTCP

Peer-to-peer Data API

-tbd

Peer-to-peer DTMF

-tbd

Statistics Model

The browser maintains a set of statistics for monitored objects, in the form of stats objects.
A group of related objects may be referenced by a selector( like MediaStreamTrack that is sent or received by the RTCPeerConnection).

Statistics API extends the RTCPeerConnection interface

partial interface RTCPeerConnection {
  Promise<RTCStatsReport> getStats(optional MediaStreamTrack? selector = null);
  attribute EventHandler onstatsended;
};

Method getStats()- Gathers stats for the given selector and reports the result asynchronously.

RTCStatsReport Object

map between strings that identify the inspected objects (id attribute in RTCStats instances), and their corresponding RTCStats-derived dictionaries.

interface RTCStatsReport {
  readonly maplike<DOMString, object>;
};

RTCStats Dictionary

stats object constructed by inspecting a specific monitored object.

dictionary RTCStats {
  required DOMHighResTimeStamp timestamp;
  required RTCStatsType type;
  required DOMString id;
};

RTCStatsEvent

Constructor (DOMString type, RTCStatsEventInit eventInitDict)]

interface RTCStatsEvent : Event {
  readonly attribute RTCStatsReport report;
};

dictionary RTCStatsEventInit

dictionary RTCStatsEventInit : EventInit {
  required RTCStatsReport report;
};

Stats

  • RTCRTPStreamStats, attributes ssrc, kind, transportId, codecId
  • RTCReceivedRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes packetsReceived, packetsLost, jitter, packetsDiscarded
  • RTCInboundRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes trackId, receiverId, remoteId, framesDecoded, nackCount
  • RTCRemoteInboundRTPStreamStats, all required attributes from its inherited dictionaries, and also attributes localId, bytesReceived, roundTripTime
  • RTCSentRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes packetsSent, bytesSent
  • RTCOutboundRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes trackId, senderId, remoteId, framesEncoded, nackCount
  • RTCRemoteOutboundRTPStreamStats, with all required attributes from its inherited dictionaries, and also attributes localId, remoteTimestamp
  • RTCPeerConnectionStats, with attributes dataChannelsOpened, dataChannelsClosed
  • RTCDataChannelStats, with attributes label, protocol, dataChannelIdentifier, state, messagesSent, bytesSent, messagesReceived, bytesReceived
  • RTCMediaStreamStats, with attributes streamIdentifer, trackIds
  • RTCMediaHandlerStats with attributes trackIdentifier, ended
  • RTCSenderVideoTrackAttachmentStats, with all required attributes from its inherited dictionaries
  • RTCSenderAudioTrackAttachmentStats, with all required attributes from its inherited dictionaries
  • RTCAudioHandlerStats with attribute audioLevel
  • RTCVideoHandlerStats with attributes frameWidth, frameHeight, framesPerSecond
  • RTCVideoSenderStats with attribute framesSent
  • RTCVideoReceiverStats with attributes framesReceived, framesDecoded, framesDropped, partialFramesLost
  • RTCCodecStats, with attributes payloadType, codecType, mimeType, clockRate, channels, sdpFmtpLine
  • RTCTransportStats, with attributes bytesSent, bytesReceived, rtcpTransportStatsId, selectedCandidatePairId, localCertificateId, remoteCertificateId
  • RTCIceCandidatePairStats, with attributes transportId, localCandidateId, remoteCandidateId, state, priority, nominated, bytesSent, bytesReceived, totalRoundTripTime, currentRoundTripTime
  • RTCIceCandidateStats, with attributes address, port, protocol, candidateType, url
  • RTCCertificateStats, with attributes fingerprint, fingerprintAlgorithm, base64Certificate, issuerCertificateId

Ref :

CLI/NCLI, Robocalls and STIR/SHAKEN

To understand the need for implementing an identification verification technique in Internet protocol based network to network communication system , we need to evaluate the existing problem plaguing the VoIP setup .

What is Call ID spoofing ? 

Vulnerability of existing interconnection phone system which is used by robo-callers to mask their identity or to make it appear the call is from a legitimate source, usually originates from voice-over-IP (VOIP) systems.

In this context understand the Caller Line identification CLI/ NCLI techniques used by VoIP and OTT( over the top) providers today.

CLI (Caller Line Identification)

If call goes out on a CLI route ( White Route ) the received party will likely see your callerID information

  • Lawful – Termination is legal on the remote end ie abiding country’s telco infrastructure and stable
  • Expensive – usually with direct or via leased line (TDM) interconnections with the tier-1 carriers.

Non-CLI (Non-Caller Line Identification)

The Caller ID is not visible at the call
If call goes out on a Non-CLI route (Grey Route) goes out on a non-CLI routes they will see either a blocked call or some generic number.

  • Unlawful – questionable legality or maybe violating some providers AUP(Acceptable Use Policy ) on the remote end.
  • Cheaper – low quality , usually via VoIP-GSM gateways

Example include robocalls , tele-marketting / spam etc which are unwilling to share their Caller Id for call receiver, to not be blocked or cancelled.

To overcome the problem of non-verifiable spam , robocalls a suite of protocols and procedures are proposed that can combat caller ID spoofing on VOIP and connected public telephone networks.

STIR/SHAKEN

Secure Telephony Identity Revisited / Signature-based Handling of Asserted information using toKENs

Used by robocallers to mask their identity or to make it appear the call is from a legitimate source
usually orignates from voice-over-IP (VOIP) systems

STIR

Standards developed by the Internet Engineering Task Force (IETF) 

For telecommunication service providers implement  certificate management system to create and manage the public and private keys, digital certificates used to sign and verify Caller ID details. 

Adds information to the SIP headers that allow the endpoints along the system to positively identify the origin of the data , such as JSON web tokens encrypted with the provider’s private key, encoded using Base64,

There are three levels of verification, or “attestation”

  • A : Full Attestation
    indicates that the provider recognizes the entire phone number as being registered with the originating subscriber.
  • B : Partial Attestation
    call originated with a known customer but the entire number cannot be verified,
  • C : Gateway Attestation
    call can only be verified as coming from a known gateway

How can the Public Key Infrastructure be used ? 

In an interconnection network , each telephone service provider will obtain its digital certificate from a certificate authority (CA)  that is trusted by other telephone service providers. Calling party signs the SIP Header  caller ID as legitimate . The called party verifies that the calling number is authentic

STIR

Originating service provider’s encrypted SIP Identity Header includes the following data:

  1. Attestation level
  2. Date and Time
  3. Calling and Called Numbers
  4. Orig ID for analytics and/or traceback purposes among others
  5. Location of certificate repository
  6. Signature
  7. Encryption algorithm

FCC has also assigned the role of a Secure Telephone Identity Policy Administrator (STI-PA) which oversees that CAs do not provide certificate to spoofing robocallers and enforce the framework for STIR /SHAKEN .

Sample Identity header in SIP requst

INVITE sip:bob@biloxi.example.org SIP/2.0
Via: SIP/2.0/TLS pc33.atlanta.example.com;branch=z9hG4bKnashds8
To: Bob
From: Alice ;tag=1928301774
Call-ID: a84b4c76e66710
CSeq: 314159 INVITE
Max-Forwards: 70
Date: Thu, 21 Feb 2002 13:02:03 GMT
Contact:
Identity:
"ZYNBbHC00VMZr2kZt6VmCvPonWJMGvQTBDqghoWeLxJfzB2a1pxAr3VgrB0SsSAaifsRdiOPoQZYOy2wrVghuhcsMbHWUSFxI6p6q5TOQXHMmz6uEo3svJsSH49thyGnFVcnyaZ++yRlBYYQTLqWzJ+KVhPKbfU/pryhVn9Yc6U="
Identity-Info: https://atlanta.example.com/atlanta.cer;alg=rsa-sha1
Content-Type: application/sdp
Content-Length: 147

v=0
o=UserA 2890844526 2890844526 IN IP4 pc33.atlanta.example.com
s=Session SDP
c=IN IP4 pc33.atlanta.example.com
t=0 0
m=audio 49172 RTP/AVP 0
a=rtpmap:0 PCMU/8000

SHAKEN

STIR is based on the SIP protocol and is designed to work with calls being routed through a VOIP network. Since traditional endpoints like POTS and SS7 networks also should be covered under this call authenticity framework , SHAKEN was developed to manage call via IP-to-telephone gateways .

Developed by the Alliance of Telecommunications Industry Solutions (ATIS)

Working Steps  :

  1. When a call is initiated, a SIP INVITE is received by the originating service provider.
  2. Originating service provider verifies the call source and number to determine how to confirm validity.
    1. Full Attestation (A) — The service provider authenticates the calling party AND confirms they are authorized to use this number. An example would be a registered subscriber.
    2. Partial Attestation (B) — The service provider verifies the call origination but cannot confirm that the call source is authorized to use the calling number. An example would be a calling number from behind an enterprise PBX.
    3. Gateway Attestation (C) — The service provider authenticates the call’s origin but cannot verify the source. An example would be a call received from an international gateway.
  3. Create a SIP Identity header that contains information on the calling number, called number, attestation level, and call origination, along with the certificate thus caller ID “signed” as legitimate
  4. SIP INVITE with the SIP Identity header with the certificate is sent to the destination service provider.
  5. Destination service provider verifies the identity of the header and certificate.

Diagrammatic depiction of flow of how Telecom carriers to digitally validates authenticity before receiving or handoff through their network

SHAKEN

References

Hosted IP-PBX and its SBC

SBC ( Session Borde Controllers ) are basically gateways that provide interconnectivity between the hosted IP-PBX of the enterprise to the outside world endpoints such as telco service provider, PSTN/ TDM , SIP trunking providers or even third party OTT provider apps like skype for business etc.

If you have a hosted IPPBX or PBX in your data-centre or on premise and you need controlled but heavy outflowing traffic, it is a good idea to integrate a resilient and efficient SBC to provide seamless interconnectivity.

Hosted PBX

For an enterprises such as an Trading floor or warehouse with multiple phone types , softphones , hardphones , turrets etc distributed across various geographies and zones a device agnostic architectural setup is prime . Listing the essentials for setting up such a system. Note supplementary services are data-services , logging , licensing etc are important but kept out of scope to keep focus on functional aspects .

An enterprise application usually is structured in tiers or layers

  • Client tier – the networks clients communication to the central java programs . Runs on client machines
  • web tier – state full communication between client and business tier . Runs in server machine.
  • business tier- handles the logic of the application. The business tier uses the Enterprise Java Bean (EJB) container, which manages the execution of the beans
  • data tier – encompasses DB drivers . Runs on separate machines for database storage

Event services for Line status notifications

providers lines status notification across enterprise for inter zone and softphone to hardphone .

Routing services

routing calls within enterprise and hardphone sites read more about resource zones later in the article

Call Control Manager (CCM)

consolidated set of all service and component that make up the VOIP platform besides media handlers . It includes SIP adapters , bridge managers , call processing frameworks , API frameworks , healthchecks etc .

Call processing framework ( CPF)

signalling and call routing logic , mostly in SIP and trunks . Manages identities such as Call Line information , Called Party Information , line status etc in shared memory.

Multiple shared Lines and their statuses

Incases where there is a need to process multiple calls from a single User agent device such as a softphone or hardphone ( common scenario for a turret phone) , the design involves assigning it multiple sip uris and each sip uri will establish a line.

When caller calls callee , the line is said to be BUSY , otherwise said to be IDLE. Transition of a shared sip line from IDLE to BUSY is transmitted to others via SIP PUBLISH as other UAs holding the same sip

Similarly any other event like transfer is propagated to other via SIP UPDATE

Clustering Call control managers (CCM)

A Call Communication manager (CCM) from various zones should be able to cowork on call and session management and advanced features such as routing from home guest zone to home zone , call transfer , refer , barge etc. Designing a clustered setup will also provide elasticity , fail-over and high availability. Can use clustered , HA compliant framework such as Oracle Communication Application Server , suited for enterprise level deployments.

Call Replication and distributed memory management

A node will store two types of data: active sessions and passive sessions. The active sessions are used by the node and stored in cache. The passive sessions are the replicas from the other nodes’ active sessions. The passives sessions are stored on a persistent storage.

Controlling Line Calls using AOR and Resource Zones

When dealing with many SIP endpoints , now referred to as resource, it is best to assign the resources to their respective zones. Thus a resource’s status updates will be only updated by its active resource zone while can be read by any resource zone.

Incoming request Zone vs Active Resource Zone

For an Incoming request such a INVITE , check whether the zone sending the request is its active resource zone or not .If the Active Resource Zone is the same zone on which the INVITE came in, then the call is handled by that zone. If the Active Resource Zone is a different zone, then the call needs to be forwarded to the Active Resource Zone.

Bridges for Local Media connections

Although call signalling is handled by a resources active resource zone only, we can still create media bridges in local zone of the resource .

Local MM bridges are used to auto answer an incoming sip line call and create trunk , especially from hardphones which do not support provisional responses.

Interzone proxy Handler

proxies call control messages between active and non active resource zones. Primarily mapping the sip messages with all custom headers inbetween the communication device interfaces.

Dial Trunk using multiple dedicated sip lines and connect via Media Bridge

To save up on call routing /connection time and to support te ability to add as many users on call at runtime , a dedicated media bridge is established for every call.

  • A sip line activated is auto-answered by MM , creates a trunk and waits for other endpoint to join the bridge. The flow is as follows :
  • As INVITE arrives for an IDLE sip line , it is connected to a trunk and auto answered by a local MM bridge .
  • Since the call is already answered , when caller dials number for callee , collect the DTMF digits over RTP using RFC 2833 DTMF events.
  • Run inter-digit timer for digit collection and detect end of dialing on timeout.
  • The dialed trunk connection is made and call is added to media bridge
  • When provisional responses are received on the trunk connection, generate in-band call progress tones (ringing, proceeding etc) via the MM
  • When the line answers, the progress tones have to be stopped and the called party gets bridged to the calling party via the media bridge.

Call Diversion involves forwarding calls from zone to another zone. joinjed parties get call UPDATE status and forward response .

Call barge is the processing of joining an ongoing call . The barge event is usually propagated to joined parities via SIP INFO. Private lines do not allow barge in and are exclusively reserved for only few users.

Interconnectivity provided by an SBC ( Session Border Controller)

Hold-Resume and Music on Hold in multi-line evironment

While a regular p2p call involves simple reinvite based hold and resume with varrying SDP, the scenario is slightly more detailed for hold resume on bridged trunk connection , as explained below.

As the calls made are on bridge , a hold signal involves a RE-INIVITE with held-SDP to media manager (MM). If hold status on trunk is 200 OK the hold status will be sent to other call interfaces connected on the trunk. Else if hold is denied ,403 is sent back to hold-initiates.

Music on hold is an one way RTP mostly from media server.

For a bridged scenarios , separate Music on hold bridges are kept on Media Managers. When an UA has to hold , it is removed from original bridge and place on music on hold bridge . To be unhold/ resume it is placed back into the orignal bridge from music on hold bridge .

Conference

user initiates conference, the conference feature can execute on the zone where the user was logged on, irrespective of zones where the other conference attendees join from . The Call processing framework of originators zone completes the SDP exchange to establish two-way speech path among all the parties.

Incases there are multiple connections from a zone , a local MM conference bridge can be created for them which would connect back to originators MM conf bridge . this two part conf bridge will be transparent to the sip line sand users .

For provisioning inputs and settings setup a Diagnostics , Administration and Configuration platform which can process APIs for data services , licences , alarms or do remote device control such as using SNMP

Session Border Controllers (SBC)

At network level SBC operations include

  • bridging multiple interfaces in different networks even between the IPv4 and IPv6 networks
  • auto NAT discovery and STUN
  • protocol conversion such as TLS to UDP etc
  • Flood detection and IP filtering

For SIP specific functionalities , SBC does

  • SIP validation involving checks on syntax and message contents also consistency checks are performed.
  • stateful and call aware. tracing, monitoring and checking for validitya and health of all the SIP messages
  • Topology hiding
  • Traffic filtering
  • Codec filtering , reordering , media pinning, transcoding, or call recording
  • Data replication brings High Availability (HA) with hot backups or even Active-Active solutions.

Traffic sharing and routing roles of SBC can include

  • IP-based and Digest-based authentication
  • limiting traffic by number of concurrent calls or calling rate.
  • Dialplan and/or Custom routing
  • Dispatching/Load-balancing to a backend cluster of servers

SBC’s can be physical hardware boxes or software based applications, as the name suggests their purpose is to control the session at border between the enterprise and external service provider.

SIP to PSTN – SIP is an IP protocol whereas PSTN is a TDM one , achieving interoperability is also the KRA of an SBC

SIP trunking – SBC provide a secure sip connectivity to connect calls to sip trunks which provide bulk calls functionality at a flat pricing.

support for various fixed or mobile endpoints – SBC ensure they are RFC compliant and can extend SIP to any kind of telecom endpoint like PSTN , GSM, fax , Skype , sipphone , IP phones etc.

NAT / Network address translator – To meet the packet routing challenges across a firewall or even during private -public mapping. A combo of DHCP servers and NAT provider comes very handy to reroute or perform hole punching such that signalling and media packets are not dropped and meet the required endpoint. More about NAT here – NAT traversal using STUN and TURN.

Load balancing – Reverse proxies and Load balancers is a much adopted industry practise to mask the inner IPs of the VoIP platform and also route traffic appropriately between control and media server .

Security , QoS and Regulatory compliance – since SBCs are required to typically support a large array of clients they adhere to regulatory and industry accepted standards ,which also involves security features like AAA, TLS/SSL and other means for quality of assurance like logging and fault detection, preventing DDoS etc . In many cases SBC can also encrypt / decrypt RTP streams for probing , tapping or lawful inspection .

Terminating at carriers , PSTN and IP gateways

Additional SBC features

Inaddition to above it is good to have if an SBC provides extra features like forking , emergency number dialing ( 911 ) or active directory integration . Real Time Analysis and monitoring of call and metrics are also expected from a SBC since they reside on edge of the network and are more vulnerable to threats . For example Dialogic Mediant SBC’s and gateways , Audio Codes SBCs

With the shift from on premise PBXs to cloud based VM or microservice architecture , SBC vendors adopt a lager umbrella of services also including automation scripts for checks , reporting tools / consoles , developer friendly APIs to manage sessions via SBC and even WebRTC gateways to connect browser endpoints .

Usage Scenarios

Any VOIP dependant system which deals with bulksome voice / video traffic from external endpoints is a usages scenarios. Listing few

  • Contact Call centres
  • Remote work / offsite monitoring
  • CRM solution for sales/marketing
  • Connecting webrtc click to dial from webpage to enterprise representatives
  • connecting enterprise UCC clients to PSTN endpoints

There are many more.

VoIP system DevOps, operations and Infrastructure management, Automation

This article is focussed around various tools required to operate and mantain a growing large scale VoIP Platform .

Read about VoIP/ OTT / Telecom Solution startup’s strategy for Building a scalable flexible SIP platform which includes :

  • Scalable and Flexible SIP platform building
  • Cluster SIP telephony Server for High Availability
  • Failure Recovery
  • Multi-tier cluster architecture
  • Role Abstraction / Micro-Service based architecture
  • Distributed Event management and Event-Driven architecture
  • Containerization
  • Autoscaling Cloud Servers
  • Open standards and Data Privacy
  • Flexibility for inter-working – NextGen911 , IMS , PSTN
  • security and Operational Efficiencies

Read more about SIP VoIP system Architecture which includes

  • Infrastructure Requirements
  • Integral Components of a VOIP SIP-based architecture
  • RTP ( Real-Time Transport Protocol ) / RtCP
  • SIP gateways, registrar, proxy, redirect, application
  • Developing SIP-based applications – basic call routing, media management
  • SIP platform Development – NAt and DNS , Cross-platform and integration to External Telecommunication provider landscape , Databases

The contents of this article are

  • PCAP Collections
  • CICD on Jenkins pipeline
  • Configuration management using chef cookbooks
  • virtualization and containerization using Docker
  • Infrastructure management using terraform / Kubernetes
  • Logs Analysis and Alarming
Overview of VoIP platform DevOPS tools

PCAP Collection

Packet Capture (PCAP) is an API that captures live network packets. Besides tracking, audit and RTC visualizers, PCAP is widely used for debugging faults such as during production alarm on high failure occurrences.

Example usecase: Production alert on 503 SIP response or log entry from a gateway is not as helpful as PCAP tracking of the session ID of call across various endpoints in and out of the network to determine the point of failure.Debugging involves :

  1. Pre-specified SIP / RTP and related protocols capture 

Capture pcaps examples

tcpdump -i any -w alltraffic.pcap
rtpbreak -P2 -t100 -T100 -d logz -r alltraffic.pcap

2. Call SessionId to uniquely identify failed calls among tens of thousands of the packet 

3. Analyzer such as wireshark or tshark to track the packet

TShark inspection examples

brew cask install wireshark
tshark -r alltraffic.pcap -R "sip.CSeq.method eq INVITE"

Some of the useful call specs captured from PCAP

  • DTMF – Both in-band and out of band DTMF for every call, along with the time stamp.
  • Codec negotiations –  Extracting codecs from PCAP lets us 
    1. Validate later whether there were codec changes without prior SIP message,
    2. If the call has been hung up with 488 error code then it was due to which  codec 
  • SIP errors – track deviations from standard SIP messaging. 
    1. Identify known erroneous SIP messaging scenarios such as for MITM or replay attacks
  • RTCP Media stats – extract Jitter, Loss, RTT with RTCP reports for both the incoming and outgoing stream.
  • Identify Media or ACK Timeouts 
    1. Check whether a party has not sent any media packet for > 60 s (media time out threshold duration)
    2. When a call is hung up due to ACK time out.
  • Audio stream – After GDPR, take explicit permission from users before storing audio streams.
PCAP file analyzed in Wireshark ( PCAP source : https://wiki.wireshark.org/SampleCaptures#Sample_Captures)

Continuous Integration and Delivery Automation using Jenkins

CICD provides continous delivery hub , distribute work across multiple machines, helping drive builds, tests and deployments across multiple platforms .

Jenkins jobs is a self-contained Java-based program extensible using plugins.

Jenkins pieline– orchestrates and automates building project in Jenkins

Configuration management using chef cookbooks

Alternatives like puppet and Ansible, which are also a cross-platform configuration management platform

Compute virtualization and containerization using Docker

Docker containers can be used instead of virtual machines such as VirtualBox , to isolates applications and be OS and platform independent
Makes distributed development possible and automates the deployment possible

  • stop Stop one or more running containers
  • top Display the running processes of a container
> docker top 4417600169e8
UID PID PPID C STIME TTY TIME CMD
root 9913 9888 0 08:50 ? 00:00:00 bash /point.sh
root 10083 9913 0 08:50 ? 00:00:01 /usr/sbin/worker
root 10092 10083 0 08:50 ? 00:00:02 /micro-service
  • unpause Unpause all processes within one or more containers
  • update Update configuration of one or more containers
  • wait Block until one or more containers stop, then print their exit codes

see all iamges

> docker images
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
sipcapture/homer-cron       latest              fb2243f90cde        3 hours ago         476MB
sipcapture/homer-kamailio   latest              f159d46a22f3        3 hours ago         338MB
sipcapture/heplify          latest              9f5280306809        21 hours ago        9.61MB
<none>                      <none>              edaa5c708b3a        

See all stats

>  docker stats
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
f42c71741107        homer-cron          0.00%               52KiB / 994.6MiB      0.01%               2.3kB / 0B          602MB / 0B          0
0111765091ae        mysql               0.04%               452.2MiB / 994.6MiB   45.46%              1.35kB / 0B         2.06GB / 49.2kB     22

Run command from within a docker instnace

docker exec -it  bash

First see all processes

docker ps

select a process and enter its bash

docker exec -it 0472a5127fff bash

to edit or update a file inside docker either install vim everytime u login in resh docker conainer like

apt-get update
apt-get install vim

or add this to dockerfile

RUN [“apt-get”, “update”]
RUN [“apt-get”, “install”, “-y”, “vim”]

see if ngrep is install , if not then install and run ngrep to get sip logs isnode that docker container

apt update
apt install ngrep
ngrep -p "14795778704" -W byline -d any port 5060

docker volume – Volumes are used for persisting data generated by and used by Docker containers.
docker volumes have advantages over blind mounts such as easier to backup or migrate , managed by docker APIs, can be safely shared among multiple containers etc

docker stack – Lets to manager a cluster of docker containers thorugh docker swarm can be defined via docker-compose.yml file

docker service

  • create Create a new service
  • inspect Display detailed information on one or more services
  • logs Fetch the logs of a service or task
  • ls List services
  • ps List the tasks of one or more services
  • rm Remove one or more services
  • rollback Revert changes to a service’s configuration
  • scale Scale one or multiple replicated services
  • update Update a service

Run docker containers

sample run command

docker run -it -d --name opensips -e ENV=dev imagename:2.2

-it flags attaches to an interactive tty in the container.
-e gives envrionment variables
-d runs it in background and prints container id

Remove docker entities

To remove all stopped containers, all dangling images, and all unused networks:

docker system prune -a

To remove all unused volumes

docker system prune --volumes

To remove all stopped containers

docker container prune
sometimes docker images keep piling with stopped congainer such as 

REPOSITORY                                                             TAG                 IMAGE ID            CREATED             SIZE                                                                              d1dcfe2438ae        15 minutes ago      753MB                                                                           2d353828889b        16 hours ago        910MB                                                          ...
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                        PORTS               NAMES

0dd6698a7517        2d353828889b        "/entrypoint.sh"         13 minutes ago      Exited (137) 13 minutes ago                       hardcore_wozniak

to remove such images and their conainer , first stop and remove confainers

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

then remove all dangling images

docker rmi  $(docker images -aq --filter dangling=true)

Infrastructure management using terraform

Terraform is used for building, changing and versioning infrastructure.
Infra as Code – can run single application to datacentres via configuration files which create execution plan.
It can manage low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.
Resource Graph – builds a graph of all your resources

tfenv can be used to manage terraform versions

> brew unlink terraform
tfenv install 0.11.14
tfenv list 

Terraform configuration language

This is used for declaring resources and descriptions of infrastructure and associated files have a .tf or .tf.json file extension
Group of resources can be gathered into a module. Terraform configuration consists of a root module, where evaluation begins, along with a tree of child modules created when one module calls another.

Example : launch a single AWS EC2 instance , fle server1.tf

provider "aws" {
  profile    = "default"
  region     = "us-east-1"
}

resource "aws_instance" "server1" {
  ami           = "ami-2757fxxx"
  instance_type = "t2.micro"
}

note : AMI IDs are region specific.
profile attribute here refers to the AWS Config File in ~/.aws/credentials

Terraform command line interface (CLI)

engine for evaluating and applying Terraform configurations.
uses plugins called providers that each define and manage a set of resource types

Command Usage: terraform [-version] [-help] [args]

  • apply Builds or changes infrastructure
  • console Interactive console for Terraform interpolations
  • destroy Destroy Terraform-managed infrastructure
  • env Workspace management
  • fmt Rewrites config files to canonical format
  • get Download and install modules for the configuration
  • graph Create a visual graph of Terraform resources
  • import Import existing infrastructure into Terraform
  • init Initialize a Terraform working directory
  • output Read an output from a state file
  • plan Generate and show an execution plan
  • providers Prints a tree of the providers used in the configuration
  • refresh Update local state file against real resources
  • show Inspect Terraform state or plan
  • taint Manually mark a resource for recreation
  • untaint Manually unmark a resource as tainted
  • validate Validates the Terraform files
  • version Prints the Terraform version
  • workspace Workspace management
  • 0.12upgrade Rewrites pre-0.12 module source code for v0.12
  • debug Debug output management (experimental)
  • force-unlock Manually unlock the terraform state
  • push Obsolete command for Terraform Enterprise legacy (v1)
  • state Advanced state management

terraform init
Initialize a working directory containing Terraform configuration files.

terraform validate
checks that verify whether a configuration is internally-consistent, regardless of any provided variables or existing state.

Kubernetes

container orchestration platform , automating deployment, scaling, and management of containerized applications. Can deploy to cluster of computers, automating the distribution and scheduling as well

Service discovery and load balancing – gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.

Automatic bin packing – Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.

Storage orchestration – Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, Ceph, Cinder, or Flocker.

Self-healing – Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.

Automated rollouts and rollbacks – progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn’t kill all your instances at the same time.

Secret and configuration management – Deploy and update secrets and application configuration without rebuilding your image and without exposing secrets in your stack configuration.

Batch execution– manage batch and CI workloads, replacing containers that fail, if desired.

Horizontal scaling – Scale application up and down with a simple command, with a UI, or automatically based on CPU usage.

create minikube cluster and deploy pods

prerequisities : docker , curl , redis , others

install minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
chmod +x minikube
install minikube /usr/local/bin

Install kubectl

snap install kubectl --classic
ln -s /snap/bin/kubectl /usr/local/bin

Setup Minikube

minikube start --vm-driver=none
minikube addons enable registry-creds
kubectl -n kube-system create secret generic registry-creds-ecr
kubectl -n kube-system create secret generic registry-creds-gcr
kubectl -n kube-system create secret generic registry-creds-dpr
minikube addons configure registry-creds
Starting Kubernetes…minikube version: v1.3.0
 commit: 43969594266d77b555a207b0f3e9b3fa1dc92b1f
 minikube v1.3.0 on Ubuntu 18.04
 Running on localhost (CPUs=2, Memory=2461MB, Disk=47990MB) …
 OS release is Ubuntu 18.04.2 LTS
 Preparing Kubernetes v1.15.0 on Docker 18.09.5 …
 kubelet.resolv-conf=/run/systemd/resolve/resolv.conf
 Pulling images …
 Launching Kubernetes …
 Done! kubectl is now configured to use "minikube"
 dashboard was successfully enabled
 Kubernetes Started 

Basic Commands

  • start Starts a local kubernetes cluster
  • status Gets the status of a local kubernetes cluster
  • stop Stops a running local kubernetes cluster
  • delete Deletes a local kubernetes cluster
  • dashboard Access the kubernetes dashboard running within the minikube cluster

Images Commands:

  • docker-env Sets up docker env variables; similar to ‘$(docker-machine env)’
  • cache Add or delete an image from the local cache.

Configuration and Management Commands:

  • addons Modify minikube’s kubernetes addons
  • config Modify minikube config
  • profile Profile gets or sets the current minikube profile
  • update-context Verify the IP address of the running cluster in kubeconfig.

Networking and Connectivity Commands:

  • service Gets the kubernetes URL(s) for the specified service in your local cluster
  • tunnel tunnel makes services of type LoadBalancer accessible on localhost

Advanced Commands:

  • mount Mounts the specified directory into minikube
  • ssh Log into or run a command on a machine with SSH; similar to ‘docker-machine ssh’
  • kubectl Run kubectl

Troubleshooting Commands:

  • ssh-key Retrieve the ssh identity key path of the specified cluster
  • ip Retrieves the IP address of the running cluster
  • logs Gets the logs of the running instance, used for debugging minikube, not user code.
  • update-check Print current and latest version number

kubectl

controls the Kubernetes cluster manager.

Basic Commands (Beginner):

  • create Create a resource from a file or from stdin.
  • expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
  • run Run a particular image on the cluster
  • set Set specific features on objects
  • explain Documentation of resources
  • get Display one or many resources
  • edit Edit a resource on the server
  • delete Delete resources by filenames, stdin, resources and names, or by resources and label selector

Deploy Commands:

  • rollout Manage the rollout of a resource
  • scale Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
  • autoscale Auto-scale a Deployment, ReplicaSet, or ReplicationController

Cluster Management Commands:

  • certificate Modify certificate resources.
  • cluster-info Display cluster info
  • top Display Resource (CPU/Memory/Storage) usage.
  • cordon Mark node as unschedulable
  • uncordon Mark node as schedulable
  • drain Drain node in preparation for maintenance
  • taint Update the taints on one or more nodes

Troubleshooting and Debugging Commands:

  • describe Show details of a specific resource or group of resources
  • logs Print the logs for a container in a pod
  • attach Attach to a running container
  • exec Execute a command in a container
  • port-forward Forward one or more local ports to a pod
  • proxy Run a proxy to the Kubernetes API server
  • cp Copy files and directories to and from containers.
  • auth Inspect authorization

Advanced Commands:

  • diff Diff live version against would-be applied version
  • apply Apply a configuration to a resource by filename or stdin
  • patch Update field(s) of a resource using strategic merge patch
  • replace Replace a resource by filename or stdin
  • wait Experimental: Wait for a specific condition on one or many resources.
  • convert Convert config files between different API versions
  • kustomize Build a kustomization target from a directory or a remote url.

Settings Commands:

  • label Update the labels on a resource
  • annotate Update the annotations on a resource
  • completion Output shell completion code for the specified shell (bash or zsh)

Other Commands:

  • api-resources Print the supported API resources on the server
  • api-versions Print the supported API versions on the server, in the form of “group/version”
  • config Modify kubeconfig files
  • plugin Provides utilities for interacting with plugins.
  • version Print the client and server version information

DevOps monitoring tools nagios

Manage Docker configs

  • create Create a config from a file or STDIN
  • inspect Display detailed information on one or more configs
  • ls List configs
  • rm Remove one or more configs

Manage containers

  • attach Attach local standard input, output, and error streams to a running container
  • commit Create a new image from a container’s changes
  • cp Copy files/folders between a container and the local filesystem
  • create Create a new container
  • diff Inspect changes to files or directories on a container’s filesystem
  • exec Run a command in a running container
  • export Export a container’s filesystem as a tar archive
  • inspect Display detailed information on one or more containers
  • kill Kill one or more running containers
  • logs Fetch the logs of a container
  • ls List containers
  • pause Pause all processes within one or more containers
  • port List port mappings or a specific mapping for the container
  • prune Remove all stopped containers
  • rename Rename a container
  • restart Restart one or more containers
  • rm Remove one or more containers
  • run Run a command in a new container
  • start Start one or more stopped containers
  • stats Display a live stream of container(s) resource usage statistics
  • stop Stop one or more running containers
  • top Display the running processes of a container
  • unpause Unpause all processes within one or more containers
  • update Update configuration of one or more containers
  • wait Block until one or more containers stop, then print their exit codes

Alternatives, Senu multi-cloud monitoring or Raygun

Monitoring, debugging, logs analysis and alarms

Aggregate logs into logstash and provide search and filtering via Elastic Search and Kibana. Can also trigger alerts or notifications on specific keyword searches in logs such as WARNING or ERRRO or call_failed.

Some common alert scenarios include :

SBC and proxy gateways failures – check states of VM instance

DNS caching alerts – Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, router advertisement and network boot alerts from service such as dnsmasq

Disk usage alert – setup alerts for 80% usage and trigger an alarm to either manually prune or create automatic timely archive backups.
check the percentage of DISK USAGE

df -h

Mostly it is either the logs file or pcap recorder which need to be archieved in external storage.

Use logrotate – it can rotates, compresses, and mails system logs

config file for logrorate – logrotate -vf /etc/logrotate.conf

/var/log/messages {
    rotate 5
    weekly
    postrotate
        /usr/bin/killall -HUP syslogd
    endscript
}

Elevated Call failure SIP 503 or Call timeout SIP 408 – high frequency of failed calls indicate an internal issue and must be followed up by smoke testing the entire system to identify any probable issue such as undetected frequent crashes of any individual component or any blacklisting by a destination endpoint etc

sudo tail -f sip.log | grep 503

or

sudo tail -f sip.log | grep WARNING

cron service or processed alerts

 ps axf
  PID TTY      STAT   TIME COMMAND
    2 ?        S      0:00 [kthreadd]
    3 ?        I<     0:00  \_ [rcu_gp]
    4 ?        I<     0:00  \_ [rcu_par_gp]
    5 ?        I      0:00  \_ [kworker/0:0-eve]
    6 ?        I<     0:00  \_ [kworker/0:0H-kb]
    7 ?        I      0:00  \_ [kworker/0:1-eve]
    8 ?        I      0:00  \_ [kworker/u4:0-nv]
    9 ?        I<     0:00  \_ [mm_percpu_wq]
   10 ?        S      0:00  \_ [ksoftirqd/0]
   11 ?        I      0:00  \_ [rcu_sched]
   12 ?        S      0:00  \_ [migration/0]
   13 ?        S      0:00  \_ [cpuhp/0]
   14 ?        S      0:00  \_ [cpuhp/1]
   15 ?        S      0:00  \_ [migration/1]
   16 ?        S      0:00  \_ [ksoftirqd/1]
   17 ?        I      0:00  \_ [kworker/1:0-eve]
   18 ?        I<     0:00  \_ [kworker/1:0H-kb]

or checks cron status

service cron status
● cron.service - Regular background program processing daemon
   Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2016-06-26 03:00:37 UTC; 1min 17s ago
     Docs: man:cron(8)
 Main PID: 845 (cron)
    Tasks: 1 (limit: 4383)
   CGroup: /system.slice/cron.service
           └─845 /usr/sbin/cron -f

Jun 26 03:00:37 ip-172-31-45-21 systemd[1]: Started Regular background program processing daemon.
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (pidfile fd = 3)
Jun 26 03:00:37 ip-172-31-45-21 cron[845]: (CRON) INFO (Running @reboot jobs)

restart or start cron service if required

DB connections / connection pool process – keep listening for any alerts on DB connections failure or even warnings as this can be due to too many read operations such as in DDOS and can escalate very quickly

netstat -nltp  | grep db 
tcp        0      0 0.0.0.0:5433            0.0.0.0:*               LISTEN      5792/db-server * 

Routine deepstatus checks is a good practice too

Port checks – Regular checks if servers are lsietning on ports such as 5060 for SIP

netstat -nltp | grep 5060
tcp        0      0 x.x.x.x:5060       0.0.0.0:*               LISTEN      8970/kamailio  

cron zombie process checks – zombie process or defunct process is a process that has completed execution (via the exit system call) but still has an entry in the process table: it is a process in the “Terminated state”. List xombie process and kill them with pid to free up .

kill -9 <PID1>

bulk calls checks – consult ongoing call cmd commands for application server such as
For Freeswitch use

fs_ctl> show channels 

For kamailio use kamcmd

kamcmd dlg.list

For asterisk watch or show cmmand

watch -n 1 "sudo asterisk -vvvvvrx 'core show channels' | grep call"

Incase of DDOS or other macious attacker IP identification block the IP

iptables -I INPUT -s y.y.y.y -j DROP   

Can also use fail2ban

>apt-get update && apt-get install fail2ban

Additionally check how many dispatchers are responding on outbound gateway

opensipsctl dispatcher dump

Process control supervisor or pm2 checks – supervisor is a Linux Process Control System that allows its users to monitor and control a number of processes

ps axf | grep supervisor

for pm2

> pm2 status
[PM2] Spawning PM2 daemon with pm2_home=/Users/altanai/.pm2
[PM2] PM2 Successfully daemonized
┌─────┬───────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │

htop to check memeory and CPU

Health and load on the reverse proxy, load balancer as Nginx – perform a direct curl request to host to check if Nginx responds with a non 4xx / 5xx response or not

curl -v <public-fqdn-of-server> 

Incase of error response , restart

/etc/init.d/nginx start

Incase of updates restart ngnix config

nginx -s reload

For HTTP/SSL proxy daemon such as tiny proxy which are used for fast resposne , set the MinSpareServers, MaxSpareServers , MaxClients , MaxRequestsPerChild etc appropriately

VPN checks – restart fireealls or IPsec incase of ssues

/etc/init.d/ipsec restart

Additionally also check ssh service

ps axf | grep sshd

restart sshd if required

SSL cert expiry checks – to keep the operations running securely and prevent and abrupt termination it is a good practise to run regular certificate expiry checks for SSL certs especially on secure HTTP endpoint like APIs , web server and also on SIP applications servers for TLS. If any expiry is due in < 10 days to trigger an alert to renew the certs

Health of Task scheduling services such as RabbitMQ, Celery Distributed Task Queue – remote debugging of these can be set up via pdb which supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame.

import pdb; pdb.set_trace()
python3 -m pdb myscript.py

It can also be set up via using the client libraries provided by these Queue services themselves

cluster status – setup an efficient health check service which monitors the cluster status for High Availability. Learn more about ensuring HA –
JSON object depicting the status of cluster shards

{
  "cluster_name" : "ABC-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 14,
  "number_of_data_nodes" : 6,
  "active_primary_shards" : 200,
  "active_shards" : 300,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0
}

Status of Crticial Server Server

fscli > show status
UP 0 years, 0 days, 0 hours, 58 minutes, 33 seconds, 15 milliseconds, 58 microseconds
FreeSWITCH (Version 1.6.20 git 987c9b9 2018-01-23 21:49:09Z 64bit) is ready
3 session(s) since startup
0 session(s) - peak 1, last 5min 1
0 session(s) per Sec out of max 30, peak 1, last 5min 1
1000 session(s) max
min idle cpu 0.00/80.83
Current Stack Size/Max 240K/8192K

Programming or Syntax error in the production environment – mostly arising due to incomplete QA/testing before pushing new changes to production. Should trigger alerts for dev teams and meet with hot patches.

Many programing application development frameworks have inbuild libs for debugging , exceotion handling and reporting such as

  • backend service in Django
  • API service in Go

Distributed memory caching – redis , memcahe

Redis info

>redis-cli info
# Server
redis_version:6.0.4
redis_git_dirty:0
redis_mode:standalone
os:Darwin 18.7.0 x86_64
arch_bits:64
multiplexing_api:kqueue
atomicvar_api:atomic-builtin
gcc_version:4.2.1
tcp_port:6379

# Clients
connected_clients:1
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:1065648
used_memory_human:1.02M
number_of_cached_scripts:0
maxmemory:0
allocator_frag_bytes:1123680
allocator_rss_ratio:1.00
rss_overhead_bytes:37888
mem_fragmentation_ratio:2.16
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:0
..

# Replication
role:master
connected_slaves:0
..

# CPU
used_cpu_sys:0.011198
used_cpu_sys_children:0.000000

# Modules

# Cluster
cluster_enabled:0

SMS service using smsc on Kannel From the kannel servers, you should see the PANIC error (most of the time Assertion error crashing kannel):

grep PANIC /var/log/kannel/bearerbox.log

IF you are going to restart , Flush redis cache

sudo redis-cli FLUSHALL
sudo redis-cli SAVE

restart kannel

sudo /etc/init.d/kannel restart

If the carriers are throttling the SMS request , verify “ERROR” responses using

sudo grep -i "throttling" bearerbox.log

Alternatives include AWS logs services ,

  • Scalyr logging
  • Sensu monitoring for multi-cloud monitoring using event pipeline

References :


Secure Communication with SRTP and key managemnt protocols like SDES , ZRTP and DTLS

With advent of Voice over IP , the real time streaming of data/audio/video also became critically important to be protected from eavesdropping or modification over the open internet.

While Secure Real-time Transport Protocol (SRTP) is a profile of the Real-time Transport Protocol (RTP), which can provide confidentiality, message authentication, and replay protection to the RTP traffic and to the control traffic for RTP, the Real-time Transport Control Protocol (RTCP).

ZRTP is a protocol that negotiates the keys and other information required to setup a SRTP audio and video session

To read about RealTime Transport protocol (RTP) , RTP control protocol (RTCP ) , before reading about adding security to RTP , RTCP and its feedback go here .

SRTP (Secure Real-time Transport Protocol)

SRTP provides a framework for encryption and message authentication of RTP and RTCP streams by negotiating keys.

It is not a transport but a profile of the Real-time Transport Protocol (RTP) for securing RTP streams in addition to providing confidentiality, integrity protection, source authentication, and replay protection.

The SRTP specification also defines how to setup and maintain a cryptographic context. This context holds all necessary data to perform the security operations, for example the SRTP encryption keys, the packet sequence counters, authentication keys, and so on. Each SRTP session, which is the same as a RTP session, has its own context. Thus a bidirectional SRTP communication requires two different SRTP cryptographic contexts.

Features of SRTP

framework for encryption and message authentication of RTP and RTCP streams
confidentiality and integrity of the entire RTP and RTCP packets, together with protection against replayed packets.
secure for unicast and multicast RTP applications
low computational cost and small footprint
high throughput and low packet expansion to support bandwidth economy.
permits upgrading with new cryptographic transforms,
protection for heterogeneous environments (mix of wired and wireless networks)

Independant from the underlying transport, network, and physical layers used by RTP, in particular high tolerance to packet loss and re-ordering.

Normal RTP Packet
SecureRTP Packet

SRTCP ( Secure RTCP)

Secure RTCP (SRTCP) is similar to the SRTP format of the SRTCP packet which has the authentication tag and MKI headers, including two additional headers:

  • SRTCP index
  • Encrypt-flag

Key management protocols for SRTP

Since SRTP does not contain an integrated key management solution, one can employ any of the following key management protocols

SDES (Session Description Protocol Security Descriptions) – SRTP Key management

It is a way to negotiate the key/cryptographic parameters for SRTP.
Keys are transported in the SDP attachment of a SIP message using TLS transport layer (SSLv3/TLSv1) or other methods like S/MIME.

media attribute defined by SDES is “crypto”
a=crypto: inline: [session-parms]

SDES packet

3 commonly used crypto suites are :

  1. AES_CM_128_HMAC_SHA1_80
  2. AES_CM_128_HMAC_SHA1_32
  3. F8_128_HMAC_SHA1_32

DTLS – SRTP Key management

DTLS keying happens on the media path, independent of any out-of-band signalling channel present.

Jitsi Client SRTP configuration

An offer can include any of –

  • plain RTP (RTP/AVP),
  • RTP with RTCP-based feedback (RTP/AVPF),
  • Secure RTP (RTP/SAVP), or
  • Secure RTP with RTCP-based feedback (RTP/SAVPF)

SDP for RTP/AVP

v=0
o=987654321-jitsi.org 0 0 IN IP4 x.x.x.x.
s=-
c=IN IP4 x.x.x.x
t=0 0
m=audio 24380 RTP/AVP 9
a=rtcp-xr:voip-metrics
a=rtpmap:9 G722/8000
a=sendrecv
m=audio 24400 RTP/AVP 9
a=rtcp-xr:voip-metrics
a=rtpmap:9 G722/8000
a=sendrecv

or

v=0.
o=987654321-jitsi.org 0 0 IN IP4 x.x.x.x.
s=-.
c=IN IP4 x.x.x.x.
t=0 0.
m=audio 5018 UDP/TLS/RTP/SAVP 9.
a=rtpmap:9 G722/8000.
a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level.
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level.
a=rtcp-xr:voip-metrics.
a=setup:actpass.
a=fingerprint:sha-1 B9:0F:89:EE:BD:1F:B1:C4:86:B6:D7:5C:25:88:53:F4:02:F4:F5:91.
m=audio 5018 RTP/SAVPF 9.
a=rtpmap:9 G722/8000.
a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level.
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level.
a=rtcp-xr:voip-metrics.
a=setup:actpass.
a=fingerprint:sha-1 B9:0F:89:EE:BD:1F:B1:C4:86:B6:D7:5C:25:88:53:F4:02:F4:F5:91.

The m line indicates which mode of RTP and RTCP is it offering .

Case where offerer/calleer wants to establish a Secure RTP audio stream on plain RTP with DTLS-SRTP as the key management protocol.

type: offer, sdp: 
v=0
o=- 2977074634695769063 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1 2
a=msid-semantic: WMS i2CKXQdort5QF76tyO5SUKyyyyPfMYR4kjZO
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:w5/T
a=ice-pwd:zuPM49QcEX3cKRQiKylJU4Y6
a=ice-options:trickle
a=fingerprint:sha-256 5A:70:05:55:C1:5A:82:51:02:D3:00:A3:BF:E7:EF:62:DF:29:EB:F2:9F:5F:51:58:12:D9:4C:AA:41:36:86:13
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=sendrecv
a=msid:i2CKXQdort5QF76tyO5SUKyyyyPfMYR4kjZO 5ffdb0f9-48b1-43bc-9f63-ea032643aeba
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:2215726670 cname:e6egqLfRbLu6vH45
a=ssrc:2215726670 msid:i2CKXQdort5QF76tyO5SUKyyyyPfMYR4kjZO 5ffdb0f9-48b1-43bc-9f63-ea032643aeba
a=ssrc:2215726670 mslabel:i2CKXQdort5QF76tyO5SUKyyyyPfMYR4kjZO
a=ssrc:2215726670 label:5ffdb0f9-48b1-43bc-9f63-ea032643aeba
m=application 9 DTLS/SCTP 5000
c=IN IP4 0.0.0.0
a=ice-ufrag:w5/T
a=ice-pwd:zuPM49QcEX3cKRQiKylJU4Y6
a=ice-options:trickle
a=fingerprint:sha-256 5A:70:05:55:C1:5A:82:51:02:D3:00:A3:BF:E7:EF:62:DF:29:EB:F2:9F:5F:51:58:12:D9:4C:AA:41:36:86:13
a=setup:actpass
a=mid:2
a=sctpmap:5000 webrtc-datachannel 1024

SRTP on kamailio

For Secure Communication kamailio supports – Digest SIP User authentication , Authorization via ACL or group membership , IP and Network authentication , TLS support for SIP signaling , transparent handling of SRTP for secure audio , TLS domain name extension support ,authentication and authorization against database (MySQL, PostgreSQL, UnixODBC, BerkeleyDB, Oracle, text files), RADIUS and DIAMETER.

Code to set flag rtp_secure_media to true if both TLS and SRTP are active

<condition field="${rtp_has_crypto}" expression="^(AES_CM_128_HMAC_SHA1_32|AES_CM_128_HMAC_SHA1_80)$" break="never">	<action application="set" data="rtp_secure_media=true"/></condition>

Invite from Jitsi client alternatively offering 3 different types of audio SDP’s – RTP/SAVPF , RTP/SAVP and RTP/AVP. Which ever will be accepted by the other endpoint will be communicated back using SDP in 200 OK.

INVITE sip:99999999999@x.x.x.x:5080 SIP/2.0
   Call-ID: 2a34d1e981602c82c345513f3f2f89ed@0:0:0:0:0:0:0:0
   CSeq: 1 INVITE
   From: "altanai" ;tag=bed49270
   To: 
   Via: SIP/2.0/UDP y.y.y.y:5060;branch=z9hG4bK-3130-9657d2ae9b662779bc08cdd32881828f
   Max-Forwards: 70
   Contact: "altanai" 
   User-Agent: Jitsi2.10.5550Mac OS X
   Content-Type: application/sdp
   Content-Length: 2336
   v=0
   o=7777777777-jitsi.org 0 0 IN IP4 y.y.y.y
   s=-
   c=IN IP4 y.y.y.y
   t=0 0
   m=audio 5016 UDP/TLS/RTP/SAVP 9
   a=rtpmap:9 G722/8000
   a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level
   a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
   a=rtcp-xr:voip-metrics
   a=setup:actpass
   a=fingerprint:sha-1 55:CF:25:5D:D5:65:71:C8:D9:FF:97:AD:CC:F2:08:DB:38:DD:81:38
m=audio 5016 RTP/SAVPF 9
   a=rtpmap:9 G722/8000
   a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level
   a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
   a=rtcp-xr:voip-metrics
   a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:Ekb2qAA8F7VCmz0FMSrad0rIt8duHQFedu/KxMbD
   a=crypto:2 AES_CM_128_HMAC_SHA1_32 inline:rEeGiaLCUbFw0sS0FxARgX9i5pwEj/frxxbgGkch
   a=crypto:3 AES_192_CM_HMAC_SHA1_80 inline:up9VO2T/rfu8V0cecA4RuG0aWgSaCC5gD/p/RdY1odg1p/0Pto0=
   a=crypto:4 AES_192_CM_HMAC_SHA1_32 inline:6yLDM31gAuwrlL0qkH72QYJLwtzX1IX+Z+7UML3VA5CpIbUWeAw=
   a=crypto:5 AES_256_CM_HMAC_SHA1_80 inline:2Q3b3UpPJMosXTrm/0Ui5q3Mw8tQ6ig5Xq0jt4Ibj0t5hVQx5KBRbC+8sMJDMg==
   a=crypto:6 AES_256_CM_HMAC_SHA1_32 inline:yVs8C3xPFY2LAUXIH+dlgBBNSz+jm1cbAQlAgv8hPKGe1zfu2wzx1d465UfFzQ==
   a=crypto:7 F8_128_HMAC_SHA1_80 inline:bhIPhj1TryAB63p/g8B3gL5NXJJ7V4kbjXqYaU54
   a=setup:actpass
   a=fingerprint:sha-1 55:CF:25:5D:D5:65:71:C8:D9:FF:97:AD:CC:F2:08:DB:38:DD:81:38
m=audio 5016 RTP/SAVP 9
   a=rtpmap:9 G722/8000
   a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level
   a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
   a=rtcp-xr:voip-metrics
   a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:Ekb2qAA8F7VCmz0FMSrad0rIt8duHQFedu/KxMbD
   a=crypto:2 AES_CM_128_HMAC_SHA1_32 inline:rEeGiaLCUbFw0sS0FxARgX9i5pwEj/frxxbgGkch
   a=crypto:3 AES_192_CM_HMAC_SHA1_80 inline:up9VO2T/rfu8V0cecA4RuG0aWgSaCC5gD/p/RdY1odg1p/0Pto0=
   a=crypto:4 AES_192_CM_HMAC_SHA1_32 inline:6yLDM31gAuwrlL0qkH72QYJLwtzX1IX+Z+7UML3VA5CpIbUWeAw=
   a=crypto:5 AES_256_CM_HMAC_SHA1_80 inline:2Q3b3UpPJMosXTrm/0Ui5q3Mw8tQ6ig5Xq0jt4Ibj0t5hVQx5KBRbC+8sMJDMg==
   a=crypto:6 AES_256_CM_HMAC_SHA1_32 inline:yVs8C3xPFY2LAUXIH+dlgBBNSz+jm1cbAQlAgv8hPKGe1zfu2wzx1d465UfFzQ==
   a=crypto:7 F8_128_HMAC_SHA1_80 inline:bhIPhj1TryAB63p/g8B3gL5NXJJ7V4kbjXqYaU54
m=audio 5016 RTP/AVP 9
a=rtpmap:9 G722/8000
a=extmap:1 urn:ietf:params:rtp-hdrext:csrc-audio-level
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=rtcp-xr:voip-metrics

Kamailio in secure mode selects the SRTP block of Audio SDP and responds in 200 OK

RTP to SRTP Bridging in Freeswitch

Enable ZRTP globally. Can override this on a per channel basis http://wiki.freeswitch.org/wiki/ZRTP (on how to enable zrtp)

When SRTP it’s critical to not offer or accept variable bit rate codecs, doing so would leak information and possibly compromising SRTP stream. (FS-6404)

Supported SRTP Crypto Suites:

AEAD_AES_256_GCM_8

This algorithm is identical to AEAD_AES_256_GCM (see Section 5.2 of [RFC5116]), except that the tag length, t, is 8, and an
authentication tag with a length of 8 octets (64 bits) is used. An AEAD_AES_256_GCM_8 ciphertext is exactly 8 octets longer than its
corresponding plaintext.

AEAD_AES_128_GCM_8

This algorithm is identical to AEAD_AES_128_GCM (see Section 5.1 of [RFC5116]), except that the tag length, t, is 8, and an
authentication tag with a length of 8 octets (64 bits) is used. An AEAD_AES_128_GCM_8 ciphertext is exactly 8 octets longer than its
corresponding plaintext.

AES_CM_256_HMAC_SHA1_80 | AES_CM_192_HMAC_SHA1_80 | AES_CM_128_HMAC_SHA1_80

AES_CM_128_HMAC_SHA1_80 is the SRTP default AES Counter Mode cipher and HMAC-SHA1 message authentication with an 80-bit authentication
tag. The master-key length is 128 bits and has a default lifetime of a maximum of 2^48 SRTP packets or 2^31 SRTCP packets, whichever comes
first.

AES_CM_256_HMAC_SHA1_32 | AES_CM_192_HMAC_SHA1_32 | AES_CM_128_HMAC_SHA1_32

This crypto-suite is identical to AES_CM_128_HMAC_SHA1_80 except that the authentication tag is 32 bits. The length of the base64-decoded key and salt value for this crypto-suite MUST be 30 octets i.e., 240 bits; otherwise, the crypto attribute is considered invalid.

AES_CM_128_NULL_AUTH

The SRTP default cipher (AES-128 Counter Mode), but to use no authentication method. This policy is NOT RECOMMENDED unless it is unavoidable; see Section 7.5 of [RFC3711].

SRTP variables that modify behaviors based on direction/leg:

rtp_secure_media

possible values:
mandatory – Accept/Offer SAVP negotiation ONLY
optional – Accept/Offer SAVP/AVP with SAVP preferred
forbidden – More useful for inbound to deny SAVP negotiation
false – implies forbidden
true – implies mandatory

default if not set is accept SAVP inbound if offered.

rtp_secure_media_inbound | rtp_secure_media_outbound

This is the same as rtp_secure_media, but would apply to either inbound
or outbound offers specifically.

How to specify crypto suites:

By default without specifying any crypto suites FreeSWITCH will offer crypto suites from strongest to weakest accepting the strongest each
endpoint has in common. If you wish to force specific crypto suites you can do so by appending the suites in a comma separated list in the order that you wish to offer them in.

Examples:
rtp_secure_media=mandatory:AES_CM_256_HMAC_SHA1_80,AES_CM_256_HMAC_SHA1_32
rtp_secure_media=true:AES_CM_256_HMAC_SHA1_80,AES_CM_256_HMAC_SHA1_32
rtp_secure_media=optional:AES_CM_256_HMAC_SHA1_80
rtp_secure_media=true:AES_CM_256_HMAC_SHA1_80

Additionally you can narrow this down on either inbound or outbound by specifying as so:

rtp_secure_media_inbound=true:AEAD_AES_256_GCM_8
rtp_secure_media_inbound=mandatory:AEAD_AES_256_GCM_8
rtp_secure_media_outbound=true:AEAD_AES_128_GCM_8
rtp_secure_media_outbound=optional:AEAD_AES_128_GCM_8

rtp_secure_media_suites

Optionaly you can use rtp_secure_media_suites to dictate the suite list and only use rtp_secure_media=[optional|mandatory|false|true] without having to dictate the suite list with the rtp_secure_media* variables.

In vars.xml

SIP and TLS settings. http://wiki.freeswitch.org/wiki/Tls valid options: sslv2,sslv3,sslv23,tlsv1,tlsv1.1,tlsv1.2 default: tlsv1,tlsv1.1,tlsv1.2

TLS cipher suite: default ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH The actual ciphers supported will change per platform. openssl ciphers -v ‘ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH’ Will show you what is available in your verion of openssl.

SRTP to RTP over multiple Crypto suits

Logs :

A client at 7777777777@ is trying to call 9999999999@ , which freeswtch has to proxy and convert from RTP to SRTP.
The following debug logs form sofia external show this process.

recv 1215 bytes from udp/[]:4642 at 07:08:27.374857:


INVITE sip:9999999999@:5080;transport=UDP SIP/2.0
   Via: SIP/2.0/UDP :47851;branch=z9hG4bK-524287-1---7cc8ad9383e9787d;rport
   Max-Forwards: 70
   Contact: :47851;transport=UDP>
   To: :5080;transport=UDP>
   From: :5080;transport=UDP>;tag=5df9f82c
   Call-ID: lFNvnuABQfOpROxfFp-MZQ..
   CSeq: 1 INVITE
   Allow: INVITE, ACK, CANCEL, BYE, NOTIFY, REFER, MESSAGE, OPTIONS, INFO, SUBSCRIBE
   Content-Type: application/sdp
   User-Agent: Z 5.2.28 rv2.8.115
   Allow-Events: presence, kpml, talk
   Content-Length: 607
v=0
   o=Z 20472192 0 IN IP4 
   s=Z
   c=IN IP4 
   t=0 0
   m=audio 8000 RTP/AVP 106 9 3 111 0 8 97 110 112 98 101 100 99 102
   a=rtpmap:106 opus/48000/2
   a=fmtp:106 minptime=20; cbr=1; maxaveragebitrate=40000; useinbandfec=1
   a=rtpmap:111 speex/16000
   a=rtpmap:97 iLBC/8000
   a=fmtp:97 mode=20
   a=rtpmap:110 speex/8000
   a=rtpmap:112 speex/32000
   a=rtpmap:98 telephone-event/48000
   a=fmtp:98 0-16
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-16
   a=rtpmap:100 telephone-event/16000
   a=fmtp:100 0-16
   a=rtpmap:99 telephone-event/32000
   a=fmtp:99 0-16
   a=rtpmap:102 G726-32/8000
   a=sendrecv

[NOTICE] switch_channel.c:1104 New Channel sofia/external/7777777777@:5080 [ed5e07ee-bd00-4a47-b4e1-6abc9dd23ed6]

[DEBUG] switch_core_state_machine.c:584 (sofia/external/7777777777@:5080) Running State Change CS_NEW (Cur 1 Tot 33)

[DEBUG] sofia.c:10078 sofia/external/7777777777@:5080 receiving invite from :4642 version: 1.9.0 -742-8f1be0 64bit

[DEBUG] sofia.c:7291 Channel sofia/external/7777777777@:5080 entering state [received][100]

[DEBUG] sofia.c:7301 Remote SDP:
v=0
o=Z 20472192 0 IN IP4 
s=Z
c=IN IP4 
t=0 0
m=audio 8000 RTP/AVP 106 9 3 111 0 8 97 110 112 98 101 100 99 102
a=rtpmap:106 opus/48000/2
a=fmtp:106 minptime=20; cbr=1; maxaveragebitrate=40000; useinbandfec=1
a=rtpmap:111 speex/16000
a=rtpmap:97 iLBC/8000
a=fmtp:97 mode=20
a=rtpmap:110 speex/8000
a=rtpmap:112 speex/32000
a=rtpmap:98 telephone-event/48000
a=fmtp:98 0-16
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=rtpmap:100 telephone-event/16000
a=fmtp:100 0-16
a=rtpmap:99 telephone-event/32000
a=fmtp:99 0-16
a=rtpmap:102 G726-32/8000
[DEBUG] sofia.c:7693 (sofia/external/7777777777@:5080) State Change CS_NEW -> CS_INIT
State NEW
Running State Change CS_INIT (Cur 1 Tot 33)
Standard INIT
State Change CS_INIT -> CS_ROUTING
State INIT going to sleep
Running State Change CS_ROUTING (Cur 1 Tot 33)
Callstate Change DOWN -> RINGING
State ROUTING

send 389 bytes to udp/[]:4642 at 07:08:27.376085:

SIP/2.0 100 Trying

Via: SIP/2.0/UDP :47851;branch=z9hG4bK-524287-1---7cc8ad9383e9787d;rport=4642;received=
   From: :5080;transport=UDP>;tag=5df9f82c
   To: :5080;transport=UDP>
   Call-ID: lFNvnuABQfOpROxfFp-MZQ..
   CSeq: 1 INVITE
   User-Agent: FreeSWITCH-mod_sofia/1.9.0-742-8f1b7e0~64bit
   Content-Length: 0

After the invote is recived and processed with 100 trying reply , the routing and rtp secure trabformation begins by adding crypto keys and forarding to

Standard EXECUTE
ed5e07ee EXECUTE sofia/external/7777777777@:5080 set(rtp_secure_media=optional)
[rtp_secure_media]=[optional]
ed5e07ee EXECUTE sofia/external/7777777777@:5080 log(INFO Forwarding calls 9999999999@ )
Forwarding calls 9999999999@
…
Set Local audio crypto Key [1 AEAD_AES_256_GCM_8 inline:aHJ1yquBtm4Lzfi2oMpe6cV7IBEy3YgKxrJ3qjvLuRXSuZfHcV4VtVNwHDw]
Set Local video crypto Key [1 AEAD_AES_256_GCM_8 inline:qeJbqlSbnKBNew575hSZ3LX78o6GBsjgOrSMxzGH/zb1E7mkls1Mda93U9w]
Set Local text crypto Key [1 AEAD_AES_256_GCM_8 inline:VghMVsjWQwnOAAjBJ1NTB3jZgfpNV/Yu4poxkAPMqkC7C+fhPKApCJrWg3U]
Set Local audio crypto Key [2 AEAD_AES_128_GCM_8 inline:7XNrjjwC/eOVnWlBSp74DfiIGAEYn/BN+latfA]
Set Local video crypto Key [2 AEAD_AES_128_GCM_8 inline:UQrFpy9Q7L5DI/ww4e5IAmwy7BxSw5yd/T0v0Q]
Set Local text crypto Key [2 AEAD_AES_128_GCM_8 inline:ZqkEPrUFHkaQ+7CROp52H/JO0MbrYWk/Eyl9lQ]
Set Local audio crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:PTGAm2KlbfuKtIUVGtXknKKzALAzfILZJuPOjfO9S07eWRE6FR0aMUvjuehJgw]
Set Local video crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:ahHIB0o/dp3SliYWK9BkxM7TfzILwG0bjDn7JuvYi+puRkTM4mYvvsSmywLaYA]
Set Local text crypto Key [3 AES_CM_256_HMAC_SHA1_80 inline:crAs8dPcWJkEEGj5nqTvFGl/TWpxxb86k+dX5gBXhh+q6DO2pEqWNkQmm55aLA]
Set Local audio crypto Key [4 AES_CM_192_HMAC_SHA1_80 inline:SLBJWjgMdfiYX7TUwWQ9CmqUsILLJrpBIVjbfuQmpBIFLvvA/XU]
Set Local video crypto Key [4 AES_CM_192_HMAC_SHA1_80 fNazWgWwNRPjUKNHVqkz44]
Set Local text crypto Key [4 AES_CM_192_HMAC_SHA1_80 inline:hbe9qqETBSK5hRQ8DI9mXL4QAjjGSR8tGDiTHCJF3yxCrRk1ajk]
Set Local audio crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:8q8mer9N2V4qVxnaazuJeT0KXgW2scONy36J3KaS]
Set Local video crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:TP5NQ1yB8ZSCCwZMgXur9VHZ5SlpNfnXePj7eZrk]
Set Local text crypto Key [5 AES_CM_128_HMAC_SHA1_80 inline:HT3F3iYG8H/majhBZbOs2Z8ye/WEVGT5Oytx2oQS]
Set Local audio crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:fEohh92lX2xLmeFYlt8YouM2jN4z5pU05d90BYfoAKU6m4CWv8g8AnifDUKk9A]
Set Local video crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:+uBNmLcvj41hXoMxNlMNBpq68gU4PmLwYcdopEB/X/jfPElkUgHfguPIgIFJUg]
Set Local text crypto Key [6 AES_CM_256_HMAC_SHA1_32 inline:cqk7D3+KMQ+31R4FFDRRzn/aluyIgjxBL59vfxcsdf5OW9izEJtU+06GewJyIA]
Set Local audio crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:Tv25TfP9fQZ+ljs/tFlHohkckiK4F6cemzEjHSvo2+q6No4ai+o]
Set Local video crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:CY/Dizd1QrlobZtgnigr0hWE+oDSx4S1F51Zpo4aZamN+8ZMdp8]
Set Local text crypto Key [7 AES_CM_192_HMAC_SHA1_32 inline:aEox/7IMps5c+uOWbosZ618+opkJV/GnrKc2EnAhVnDNeo91+No]
Set Local audio crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:0LwKGyljIed0zhukiMMyD5ive0ZsyybwBrnevcAv]
Set Local video crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:eZN8rAG8UPPntdYxsg1kkWL4qMsVgTiGGiS4UeUM]
Set Local text crypto Key [8 AES_CM_128_HMAC_SHA1_32 inline:bAYzbfr+El8usaTkPBR6iFuTda4uLNGjyx9lQWkX]
Set Local audio crypto Key [9 AES_CM_128_NULL_AUTH inline:5m3142gGG1HZ5VnoXsAOyopSwDCYbrIsGpdbEO3D]
Set Local video crypto Key [9 AES_CM_128_NULL_AUTH inline:zXk67wjwRhSilq0kiz5TWxXqrxuTaWTA3qqbVo/G]
Set Local text crypto Key [9 AES_CM_128_NULL_AUTH inline:FRP9CJbBO+PRj6I9RSBAiMxRZ/qFtyrEXPfxocG0]
sending invite version: 1.9.0 -742-8f1b7e0 64bit
Local SDP:
v=0
o=FreeSWITCH 1552960557 1552960558 IN IP4
s=FreeSWITCH
c=IN IP4
t=0 0
m=audio 18750 RTP/SAVP 102 9 0 8 103 101
a=rtpmap:102 opus/48000/2
a=fmtp:102 useinbandfec=1; maxaveragebitrate=30000; maxplaybackrate=48000; ptime=20; minptime=10; maxptime=40; stereo=1
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:103 telephone-event/48000
a=fmtp:103 0-16
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=crypto:1 AEAD_AES_256_GCM_8 inline:aHJ1yquBtm4Lzfi2oMpe6cV7IBEy3YgKxrJ3qjvLuRXSuZfHcV4VtVNwHDw
a=crypto:2 AEAD_AES_128_GCM_8 inline:7XNrjjwC/eOVnWlBSp74DfiIGAEYn/BN+latfA
a=crypto:3 AES_CM_256_HMAC_SHA1_80 inline:PTGAm2KlbfuKtIUVGtXknKKzALAzfILZJuPOjfO9S07eWRE6FR0aMUvjuehJgw
a=crypto:4 AES_CM_192_HMAC_SHA1_80 inline:SLBJWjgMdfiYX7TUwWQ9CmqUsILLJrpBIVjbfuQmpBIFLvvA/XU
a=crypto:5 AES_CM_128_HMAC_SHA1_80 inline:8q8mer9N2V4qVxnaazuJeT0KXgW2scONy36J3KaS
a=crypto:6 AES_CM_256_HMAC_SHA1_32 inline:fEohh92lX2xLmeFYlt8YouM2jN4z5pU05d90BYfoAKU6m4CWv8g8AnifDUKk9A
a=crypto:7 AES_CM_192_HMAC_SHA1_32 inline:Tv25TfP9fQZ+ljs/tFlHohkckiK4F6cemzEjHSvo2+q6No4ai+o
a=crypto:8 AES_CM_128_HMAC_SHA1_32 inline:0LwKGyljIed0zhukiMMyD5ive0ZsyybwBrnevcAv
a=crypto:9 AES_CM_128_NULL_AUTH inline:5m3142gGG1HZ5VnoXsAOyopSwDCYbrIsGpdbEO3D
a=ptime:20
a=sendrecv

Once the SDP is ready with crypto keys it is the forwarded to the next_up

send 2104 bytes to udp/[]:5060 at 07:08:27.378167:


INVITE sip:9999999999@ SIP/2.0
   Via: SIP/2.0/UDP :5080;rport;branch=z9hG4bKmF251mK2pN35B
   Max-Forwards: 69
   From: "7777777777" >;tag=vcKeKD6SN02cB
   To: >
   Call-ID: a27898fd-c4b8-1237-ddaa-02a933b32da0
   CSeq: 1935861 INVITE
   Contact: :5080>
   User-Agent: FreeSWITCH-mod_sofia/1.9.0-742-8f1b7e0~64bit
   Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY
   Supported: timer, path, replaces
   Allow-Events: talk, hold, conference, refer
   Content-Type: application/sdp
   Content-Disposition: session
   Content-Length: 1304
   X-FS-Support: update_display,send_info
   Remote-Party-ID: "7777777777" >;party=calling;screen=yes;privacy=off
v=0
   o=FreeSWITCH 1552960557 1552960558 IN IP4 
   s=FreeSWITCH
   c=IN IP4 
   t=0 0
   m=audio 18750 RTP/SAVP 102 9 0 8 103 101
   a=rtpmap:102 opus/48000/2
   a=fmtp:102 useinbandfec=1; maxaveragebitrate=30000; maxplaybackrate=48000; ptime=20; minptime=10; maxptime=40; stereo=1
   a=rtpmap:9 G722/8000
   a=rtpmap:0 PCMU/8000
   a=rtpmap:8 PCMA/8000
   a=rtpmap:103 telephone-event/48000
   a=fmtp:103 0-16
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-16
   a=crypto:1 AEAD_AES_256_GCM_8 inline:aHJ1yquBtm4Lzfi2oMpe6cV7IBEy3YgKxrJ3qjvLuRXSuZfHcV4VtVNwHDw
   a=crypto:2 AEAD_AES_128_GCM_8 inline:7XNrjjwC/eOVnWlBSp74DfiIGAEYn/BN+latfA
   a=crypto:3 AES_CM_256_HMAC_SHA1_80 inline:PTGAm2KlbfuKtIUVGtXknKKzALAzfILZJuPOjfO9S07eWRE6FR0aMUvjuehJgw
   a=crypto:4 AES_CM_192_HMAC_SHA1_80 inline:SLBJWjgMdfiYX7TUwWQ9CmqUsILLJrpBIVjbfuQmpBIFLvvA/XU
   a=crypto:5 AES_CM_128_HMAC_SHA1_80 inline:8q8mer9N2V4qVxnaazuJeT0KXgW2scONy36J3KaS
   a=crypto:6 AES_CM_256_HMAC_SHA1_32 inline:fEohh92lX2xLmeFYlt8YouM2jN4z5pU05d90BYfoAKU6m4CWv8g8AnifDUKk9A
   a=crypto:7 AES_CM_192_HMAC_SHA1_32 inline:Tv25TfP9fQZ+ljs/tFlHohkckiK4F6cemzEjHSvo2+q6No4ai+o
   a=crypto:8 AES_CM_128_HMAC_SHA1_32 inline:0LwKGyljIed0zhukiMMyD5ive0ZsyybwBrnevcAv
   a=crypto:9 AES_CM_128_NULL_AUTH inline:5m3142gGG1HZ5VnoXsAOyopSwDCYbrIsGpdbEO3D
   a=ptime:20

Multimedia Internet Keying (MIKEY) – Key management of SRTP

can establish multiple security contexts or cryptographic sessions with a single message.
Can be used in p2p or bradcast scenarios where one entity generates the key and needs to distribute the key to a number of participants

modes of operatiosn

  • Pre-Shared Key
  • Public Key Encryption
  • Diffie-Hellman
  • HMAC-Authenticated Diffie-Hellman
  • RSA-R
  • TICKET
  • IBAKE
  • SAKKE

References


RFC 3711 The Secure Real-time Transport Protocol (SRTP)
RFC 3611 RTP Control Protocol Extended Reports (RTCP XR)

SRTP in Freeswitch – https://freeswitch.org/stash/projects/FS/repos/freeswitch/browse/libs/srtp


sipP ( SIP testing tool )

SIPp is an opensource (GNU GPL license) performance testing tool for the SIP protocol and is widely used for Quality assurabce of callflows in voip applications for UAC / UASs cenarios.

It can emulate functioing of a sip phone such as REGISTER , establishes and releases multiple calls with the INVITE and BYE methods , send other SIP requests and wait for reponses based on dafult of custom xml scenario files.

Plus factor is the dynamic display of statistics about running tests (call rate, round trip delay, and message statistics), periodic CSV statistics dumps, TCP and UDP over multiple sockets or multiplexed with retransmission management, regular expressions and variables in scenario files, and dynamically adjustable call rates.

sipp -sn uac -d 10000 -s 9876543210 127.0.0.1:5060  -l 10

It is widley used as aperformnace and load testing tool since it can test SIP equipements like SIP proxies, B2BUAs, SIP media servers, SIP/x gateways, and SIP PBXes and can also emulate thousands of user agents calling your SIP system.

More on SIPp scripts and various exmaples can be read from

https://github.com/altanai/kamailioexamples/tree/master/sipp

Installation

Pre-requisites to compile SIPp are:
– C++ Compiler
– curses or ncurses library
– For TLS support: OpenSSL >= 0.9.8
– For pcap play support: libpcap and libnet
– For SCTP support: lksctp-tools
– For distributed pauses: Gnu Scientific Libraries

sudo apt-get install dh-autoreconf ncurses-dev libssl-dev libpcap-dev libncurses5-dev libsctp-dev lksctp-tools

Either get source code from git

git clone https://github.com/SIPp/sipp.git
cd sipp
cmake . -DUSE_SSL=1 -DUSE_SCTP=1 -DUSE_PCAP=1 -DUSE_GSL=1
make

or download readymade tar , then extract and build with options like

tar -xvzf sipp-xxx.tar.gz
cd sipp
./configure --with-sctp --with-pcap --with-openssl
make

Building certs for TLS based sipp UAS server

make master dir for all certs

mkdir certs 
chmod 0700 certs
cd certs

Make CA folder, create cert and check

mkdir demoCA
cd demoCA
mkdir newcerts
echo '01' > serial
touch index.txt
openssl req -new -x509 -extensions v3_ca -keyout key.pem -out cert.pem -days 3650

Validation of the contents of certs ( optional )

openssl x509 -in cert.pem -noout -text
openssl x509 -in cert.pem -noout -dates
openssl x509 -in cert.pem -noout -purpose

Make domain folder and create the certs for the sip domain name from parent and check

cd ..
mkdir 10.10.10.10
openssl req -new -nodes -keyout key.pem -out req.pem
cd ..
openssl ca -days 730 -out 10.10.10.10/cert.pem -keyfile demoCA/key.pem -cert demoCA/cert.pem -infiles 10.10.10.10/req.pem

Verify the generated certificate for for SIP domain

openssl x509 -in 10.10.10.10/cert.pem -noout -text

Run sipp

sipp -sn uas -p 5077 -t l1 -tls_key /home/ubuntu/certs/10.10.10.10/key.pem  -tls_cert /home/ubuntu/certs/10.10.10.10/cert.pem  -i 10.10.10.10

Verify installation

Run sipp with embedded server (uas) scenario:

sipp -sn uas

On the same host, run sipp with embedded client (uac) scenario:

sipp -sn uac 127.0.0.1 -trace_msg -trace_err
output for server 

 # sipp -sn uas

------------------------------ Scenario Screen -------- [1-9]: Change Screen --

  Port   Total-time  Total-calls  Transport
  5060      32.95 s           61  UDP
0 new calls during 0.874 s period      1 ms scheduler resolution
  19 calls                               Peak was 41 calls, after 28 s
  0 Running, 63 Paused, 12 Woken up
  0 dead call msg (discarded)          
  3 open sockets                        
                             Messages  Retrans   Timeout   Unexpected-Msg

----------> INVITE 61 0 0 0
<---------- 180 61 0 <---------- 200 61 0 0 ----------> ACK E-RTD1 61 0 0 0

----------> BYE 61 0 0 0
<---------- 200 61 0
[ 4000ms] Pause 61 0
------------------------------ Test Terminated --------------------------------
----------------------------- Statistics Screen ------- [1-9]: Change Screen --

  Start Time             | 2019-02-04    13:04:32.108663 1549265672.108663         
  Last Reset Time        | 2019-02-04    13:05:04.189720 1549265704.189720         
  Current Time           | 2019-02-04    13:05:05.065119 1549265705.065119         
-------------------------+---------------------------+--------------------------
  Counter Name           | Periodic value            | Cumulative value
-------------------------+---------------------------+--------------------------
  Elapsed Time           | 00:00:00:875000           | 00:00:32:956000          
  Call Rate              |    0.000 cps              |    1.851 cps             
-------------------------+---------------------------+--------------------------

  Incoming call created  |        0                  |       61                 

  OutGoi traceings 

———————————————– 2019-02-04 13:08:13.939148
UDP message sent (530 bytes):

INVITE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-25-0
From: sipp ;tag=52422SIPpTag0025
To: service
Call-ID: 25-52422@192.x.x.x
CSeq: 1 INVITE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Type: application/sdp
Content-Length: 135
v=0
o=user1 53655765 2353687637 IN IP4 192.x.x.x
s=-
c=IN IP4 192.x.x.x
t=0 0
m=audio 6004 RTP/AVP 0
a=rtpmap:0 PCMU/8000

———————————————– 2019-02-04 13:08:13.939310
UDP message received [321] bytes :

SIP/2.0 180 Ringing
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: 
Content-Length: 0

———————————————– 2019-02-04 13:08:13.939905
UDP message received [486] bytes :

SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: 
Content-Type: application/sdp
Content-Length:   135
v=0
o=user1 53655765 2353687637 IN IP4 192.x.x.x
s=-
c=IN IP4 192.x.x.x
t=0 0
m=audio 6000 RTP/AVP 0
a=rtpmap:0 PCMU/8000

———————————————– 2019-02-04 13:08:13.940159
UDP message sent (371 bytes):

ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

~ RTP

———————————————– 2019-02-04 13:08:13.941658
UDP message sent (371 bytes):

BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

———————————————– 2019-02-04 13:08:13.952888
UDP message received [313] bytes :

SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: 
Content-Length: 0

Time

---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 293 10 ms <= n < 20 ms : 9 20 ms <= n < 30 ms : 0 30 ms <= n < 40 ms : 0 40 ms <= n < 50 ms : 0 50 ms <= n < 100 ms : 0 100 ms <= n < 150 ms : 0 150 ms <= n < 200 ms : 0 n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 0 10 ms <= n < 50 ms : 0 50 ms <= n < 100 ms : 0 100 ms <= n < 500 ms : 0 500 ms <= n < 1000 ms : 0 1000 ms <= n < 5000 ms : 262 5000 ms <= n < 10000 ms : 0 n >= 10000 ms : 0
------------------------------ Sipp Server Mode -------------------------------

Output for client

uac.xml
 
SIPp UAC Remote
 |(1) INVITE |
 |------------------>|
 |(2) 100 (optional) |
 |<------------------| 
 |(3) 180 (optional) | 
  |<------------------| 
|(4) 200             | 
|<------------------| 
|(5) ACK             | 
|------------------>|
 |                     |
 |(6) PAUSE             |
 |                     |
 |(7) BYE             |
 |------------------>|
 |(8) 200             |
 |<------------------|

sipp -sn uac 127.0.0.1 -trace_msg -trace_err
Resolving remote host ‘127.0.0.1’… Done.
—————————— Scenario Screen ——– [1-9]: Change Screen —
Call-rate(length) Port Total-time Total-calls Remote-host
10.0(0 ms)/1.000s 5061 17.32 s 98 127.0.0.1:5060(UDP)

3 new calls during 0.286 s period 1 ms scheduler resolution
0 calls (limit 30) Peak was 25 calls, after 10 s
0 Running, 101 Paused, 7 Woken up
0 dead call msg (discarded) 0 out-of-call msg (discarded)
3 open sockets

                             Messages  Retrans   Timeout   Unexpected-Msg
  INVITE ---------->         98        0         0                  
     100 <----------         0         0         0         0        
     180 <----------         98        0         0         0        
     183 <----------         0         0         0         0        
     200          98        0                            
   Pause [      0ms]         98                            0        
     BYE ---------->         98        0         0                  
     200 <----------         98        0         0         0        

—————————— Test Terminated ——————————–

----------------------------- Statistics Screen ------- [1-9]: Change Screen --

  Start Time             | 2019-02-04    13:08:03.908208 1549265883.908208         
  Last Reset Time        | 2019-02-04    13:08:20.954289 1549265900.954289         
  Current Time           | 2019-02-04    13:08:21.241152 1549265901.241152         

-------------------------+---------------------------+--------------------------
  Counter Name           | Periodic value            | Cumulative value

-------------------------+---------------------------+--------------------------
  Elapsed Time           | 00:00:00:286000           | 00:00:17:332000          

  Call Rate  

Tracings

———————————————– 2019-02-04 13:08:13.934840
UDP message received [527] bytes :

INVITE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service 
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Type: application/sdp
Content-Length:   135
v=0
o=user1 53655765 2353687637 IN IP4 192.x.x.x
s=-
c=IN IP4 192.x.x.x
t=0 0
m=audio 6004 RTP/AVP 0
a=rtpmap:0 PCMU/8000

———————————————– 2019-02-04 13:08:13.936616
UDP message sent (321 bytes):

SIP/2.0 180 Ringing
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: 
Content-Length: 0

———————————————– 2019-02-04 13:08:13.937003
UDP message sent (486 bytes):

SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-0
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 INVITE
Contact: 
Content-Type: application/sdp
Content-Length:   135
v=0
o=user1 53655765 2353687637 IN IP4 192.x.x.x
s=-
c=IN IP4 192.x.x.x
t=0 0
m=audio 6000 RTP/AVP 0
a=rtpmap:0 PCMU/8000

———————————————– 2019-02-04 13:08:13.948679
UDP message received [371] bytes :

ACK sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-5
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 1 ACK
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

~ RTP

———————————————– 2019-02-04 13:08:13.949168
UDP message received [371] bytes :

BYE sip:service@127.0.0.1:5060 SIP/2.0
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: sip:sipp@192.x.x.x:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

———————————————– 2019-02-04 13:08:13.949245
UDP message sent (313 bytes):

SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.x.x.x:5061;branch=z9hG4bK-52422-1-7
From: sipp ;tag=52422SIPpTag001
To: service ;tag=52416SIPpTag011
Call-ID: 1-52422@192.x.x.x
CSeq: 2 BYE
Contact: 
Content-Length: 0

time

---------------------------- Repartition Screen ------- [1-9]: Change Screen --
Average Response Time Repartition 1
0 ms <= n < 10 ms : 657 10 ms <= n < 20 ms : 20 20 ms <= n < 30 ms : 0 30 ms <= n < 40 ms : 0 40 ms <= n < 50 ms : 0 50 ms <= n < 100 ms : 0 100 ms <= n < 150 ms : 0 150 ms <= n < 200 ms : 0 n >= 200 ms : 0
Average Call Length Repartition
0 ms <= n < 10 ms : 649 10 ms <= n < 50 ms : 28 50 ms <= n < 100 ms : 0 100 ms <= n < 500 ms : 0 500 ms <= n < 1000 ms : 0 1000 ms <= n < 5000 ms : 0 5000 ms <= n < 10000 ms : 0 n >= 10000 ms : 0
------ [+|-|*|/]: Adjust rate ---- [q]: Soft exit ---- [p]: Pause traffic -----

Last Error: Overload warning: the major watchdog timer 3000ms has been t…

UAC with Media

SIPp UAC            Remote
    |(1) INVITE         |
    |------------------>|
    |(2) 100 (optional) |
    |<------------------|
    |(3) 180 (optional) |
    |<------------------|
    |(4) 200            |
    |<------------------|
    |(5) ACK            |
    |------------------>|
    |                   |
    |(6) RTP send (8s)  |
    |==================>|
    |                   |
    |(7) RFC2833 DIGIT 1|
    |==================>|
    |                   |
    |(8) BYE            |
    |------------------>|
    |(9) 200            |
    |<------------------|

sipp Usage:

sipp remote_host[:remote_port] [options]

Run SIPp with embedded server (uas) scenario: ./sipp -sn uas On the same host, run SIPp with embedded client (uac) scenario: ./sipp -sn uac 127.0.0.1

Scenario file options:

  • -sd : Dumps a default scenario (embedded in the SIPp executable)
  • -sf : Loads an alternate XML scenario file. To learn more about XML scenario syntax, use the -sd option to dump embedded scenarios. They contain all the necessary help.
  • -oocsf : Load out-of-call scenario.
  • -oocsn : Load out-of-call scenario.
  • -sn : Use a default scenario (embedded in the SIPp executable). If this option is omitted, the Standard SipStone UAC scenario is loaded. Available values in this version: 
    • ‘uac’ : Standard SipStone UAC (default).
    • ‘uas’ : Simple UAS responder.
    • ‘regexp’ : Standard SipStone UAC – with regexp and variables.
    • ‘branchc’ : Branching and conditional branching in scenarios – client.
    • ‘branchs’ : Branching and conditional branching in scenarios – server.
    Default 3pcc scenarios (see -3pcc option):
    • ‘3pcc-C-A’ : Controller A side (must be started after all other 3pcc scenarios)
    • ‘3pcc-C-B’ : Controller B side.
    • ‘3pcc-A’ : A side.
    • ‘3pcc-B’ : B side.

IP, port and protocol options

  • -t : Set the transport mode:
    • u1: UDP with one socket (default),
    • un: UDP with one socket per call,
    • ui: UDP with one socket per IP address. The IP addresses must be defined in the injection file.
    • t1: TCP with one socket,
    • tn: TCP with one socket per call,
    • l1: TLS with one socket,
    • ln: TLS with one socket per call,
    • c1: u1 + compression (only if compression plugin loaded),
    • cn: un + compression (only if compression plugin loaded). This plugin is not provided with SIPp.
  • -i : Set the local IP address for ‘Contact:’,’Via:’, and ‘From:’ headers. Default is primary host IP address.
  • -p : Set the local port number. Default is a random free port chosen by the system 
  • -bind_local : Bind socket to local IP address, i.e. the local IP address is used as the source IP address. If SIPp runs in server mode it will only listen on the local IP address instead of all IP addresses.
  • -ci : Set the local control IP address
  • -cp : Set the local control port number. Default is 8888.
  • -max_socket : Set the max number of sockets to open simultaneously. This option is significant if you use one socket per call. Once this limit is reached, traffic is distributed over the sockets already opened. Default value is 50000
  • -max_reconnect : Set the the maximum number of reconnection.
  • -reconnect_close : Should calls be closed on reconnect?
  • -reconnect_sleep : How long (in milliseconds) to sleep between the close and reconnect?
  • -rsa : Set the remote sending address to host:port for sending the messages.
  • -tls_cert : Set the name for TLS Certificate file. Default is ‘cacert.pem
  • -tls_key : Set the name for TLS Private Key file. Default is ‘cakey.pem’
  • -tls_ca : Set the name for TLS CA file. If not specified, X509 verification is not activated.
  • -tls_crl : Set the name for Certificate Revocation List file. If not specified, X509 CRL is not activated.
  • -tls_version : Set the TLS protocol version to use (1.0, 1.1, 1.2) — default is autonegotiate

SIPp overall behavior options:

  • -v : Display version and copyright information.
  • -bg : Launch SIPp in background mode.
  • -nostdin : Disable stdin.
  • -plugin : Load a plugin.
  • -sleep : How long to sleep for at startup. Default unit is seconds.
  • -skip_rlimit : Do not perform rlimit tuning of file descriptor limits. Default: false.
  • -buff_size : Set the send and receive buffer size.
  • -sendbuffer_warn : Produce warnings instead of errors on SendBuffer failures.
  • -lost : Set the number of packets to lose by default (scenario specifications override this value).
  • -key : keyword value Set the generic parameter named “keyword” to “value”.
  • -set : variable value Set the global variable parameter named “variable” to “value”.
  • -tdmmap : Generate and handle a table of TDM circuits. A circuit must be available for the call to be placed. Format: -tdmmap {0-3}{99}{5-8}{1-31}
  • -dynamicStart : variable value Set the start offset of dynamic_id variable
  • -dynamicMax : variable value Set the maximum of dynamic_id variable 
  • -dynamicStep : variable value Set the increment of dynamic_id variable

Call behavior options:

  • -aa : Enable automatic 200 OK answer for INFO, NOTIFY, OPTIONS and UPDATE.
  • -base_cseq : Start value of [cseq] for each call.
  • -cid_str : Call ID string (default %u-%p@%s). %u=call_number, %s=ip_address, %p=process_number, %%=% (in any order).
  • -d : Controls the length of calls. More precisely, this controls the duration of ‘pause’ instructions in the scenario, if they do not have a ‘milliseconds’ section. Default value is 0 and default unit is milliseconds.
  • -deadcall_wait : How long the Call-ID and final status of calls should be kept to improve message and error logs (default unit is ms).
  • -auth_uri : Force the value of the URI for authentication. By default, the URI is composed of remote_ip:remote_port.
  • -au : Set authorization username for authentication challenges. Default is taken from -s argument
  • -ap : Set the password for authentication challenges. Default is ‘password’
  • -s : Set the username part of the request URI. Default is ‘service’.
  • -default_behaviors: Set the default behaviors that SIPp will use. Possible values are:
    • all Use all default behaviors
    • none Use no default behaviors
    • bye Send byes for aborted calls
    • abortunexp Abort calls on unexpected messages
    • pingreply Reply to ping requests If a behavior is prefaced with a -, then it is turned off. Example: all,-bye
  • -nd : No Default. Disable all default behavior of SIPp which are the following:
  • On UDP retransmission timeout, abort the call by sending a BYE or a CANCEL
  • On receive timeout with no ontimeout attribute, abort the call by sending a BYE or a CANCEL
  • On unexpected BYE send a 200 OK and close the call
  • On unexpected CANCEL send a 200 OK and close the call
  • On unexpected PING send a 200 OK and continue the call
  • On any other unexpected message, abort the call by sending a BYE or a CANCEL
  • -pause_msg_ign : Ignore the messages received during a pause defined in the scenario 
  • -callid_slash_ign: Don’t treat a triple-slash in Call-IDs as indicating an extra SIPp prefix.

Injection file options:

  • -inf : Inject values from an external CSV file during calls into the scenarios. First line of this file say whether the data is to be read in sequence (SEQUENTIAL), random (RANDOM), or user (USER) order. Each line corresponds to one call and has one or more ‘;’ delimited data fields. Those fields can be referred as [field0], [field1], … in the xml scenario file. Several CSV files can be used simultaneously (syntax: -inf f1.csv -inf f2.csv …)
  • -infindex : file field Create an index of file using field. For example -inf ../path/to/users.csv -infindex users.csv 0 creates an index on the first key.
  • -ip_field : Set which field from the injection file contains the IP address from which the client will send its messages. If this option is omitted and the ‘-t ui’ option is present, then field 0 is assumed. Use this option together with ‘-t ui’

RTP behaviour options:

  • -mi : Set the local media IP address (default: local primary host IP address)
  • -rtp_echo : Enable RTP echo. RTP/UDP packets received on port defined by -mp are echoed to their sender. RTP/UDP packets coming on this port + 2 are also echoed to their sender (used for sound and video echo).
  • -mb : Set the RTP echo buffer size (default: 2048).
  • -mp : Set the local RTP echo port number. Default is 6000.
  • -rtp_payload : RTP default payload type.
  • -rtp_threadtasks : RTP number of playback tasks per thread.
  • -rtp_buffsize : Set the rtp socket send/receive buffer size.

Call rate options:

  • -r : Set the call rate (in calls per seconds). This value can bechanged during test by pressing ‘+’, ‘_’, ‘*’ or ‘/’. Default is 10.
    • pressing ‘+’ key to increase call rate by 1 * rate_scale,
    • pressing ‘-‘ key to decrease call rate by 1 * rate_scale,
    • pressing ‘*’ key to increase call rate by 10 * rate_scale,
    • pressing ‘/’ key to decrease call rate by 10 * rate_scale.
  • -rp : Specify the rate period for the call rate. Default is 1 second and default unit is milliseconds. This allows you to have n calls every m milliseconds(by using -r n -rp m). Example: -r 7 -rp 2000 ==> 7 calls every 2 seconds. -r 10 -rp 5s => 10 calls every 5 seconds.
  • -rate_scale : Control the units for the ‘+’, ‘-‘, ‘*’, and ‘/’ keys.
  • -rate_increase : Specify the rate increase every -rate_interval units (default is seconds). This allows you to increase the load for each independent logging period. Example: -rate_increase 10 -rate_interval 10s ==> increase calls by 10 every 10 seconds.
  • -rate_max : 

If -rate_increase is set, then quit after the rate reaches this value. Example: -rate_increase 10 -rate_max 100 ==> increase calls by 10 until 100 cps is hit.

  • -rate_interval : Set the interval by which the call rate is increased. Defaults to the value of -fd.
  • -no_rate_quit : If -rate_increase is set, do not quit after the rate reaches -rate_max.
  • -l :  Set the maximum number of simultaneous calls. Once this limit is reached, traffic is decreased until the number of open calls goes down. Default: (3 * call_duration (s) * rate).
  • -m : Stop the test and exit when ‘calls’ calls are processed
  • -users : Instead of starting calls at a fixed rate, begin ‘users’ calls at startup, and keep the number of calls constant.

Retransmission and timeout options:

  • -recv_timeout : Global receive timeout. Default unit is milliseconds. If the expected message is not received, the call times out and is aborted.
  • -send_timeout : Global send timeout. Default unit is milliseconds. If a message is not sent (due to congestion), the call times out and is aborted.
  • -timeout : Global timeout. Default unit is seconds. If this option is set, SIPp quits after nb units (-timeout 20s quits after 20 seconds).
  • -timeout_error : SIPp fails if the global timeout is reached is set (-timeout option required).
  • -max_retrans : Maximum number of UDP retransmissions before call ends on timeout. Default is 5 for INVITE transactions and 7 for others.
  • -max_invite_retrans: Maximum number of UDP retransmissions for invite transactions before call ends on timeout.
  • -max_non_invite_retrans: Maximum number of UDP retransmissions for non-invite transactions before call ends on timeout.
  • -nr : Disable retransmission in UDP mode.
  • -rtcheck : Select the retransmission detection method: full (default) or loose.
  • -T2 : Global T2-timer in milli seconds

Third-party call control options:

  • -3pcc : Launch the tool in 3pcc mode (“Third Party call control”). The passed IP address depends on the 3PCC role.
    • When the first twin command is ‘sendCmd’ then this is the address of the remote twin socket. SIPp will try to connect to this address:port to send the twin command (This instance must be started after all other 3PCC scenarios). Example: 3PCC-C-A scenario.
    • When the first twin command is ‘recvCmd’ then this is the address of the local twin socket. SIPp will open this address:port to listen for twin command. Example: 3PCC-C-B scenario.
  • -master : 3pcc extended mode: indicates the master number
  • -slave : 3pcc extended mode: indicates the slave number
  • -slave_cfg : 3pcc extended mode: indicates the file where the master and slave addresses are stored

Performance and watchdog options:

  • -timer_resol
    Set the timer resolution. Default unit is milliseconds. This option has an impact on timers precision.Small values allow more precise scheduling but impacts CPU usage.If the compression is on, the value is set to 50ms. The default value is 10ms.
  • -max_recv_loops Set the maximum number of messages received read per cycle. Increase this value for high traffic level. The default value is 1000.
  • -max_sched_loops Set the maximum number of calls run per event loop. Increase this value for high traffic level. The default value is 1000.
  • -watchdog_interval : Set gap between watchdog timer firings. Default is 400.
  • -watchdog_reset : If the watchdog timer has not fired in more than this time period, then reset the max triggers counters. Default is 10 minutes.
  • -watchdog_minor_threshold: If it has been longer than this period between watchdog executions count a minor trip. Default is 500.
  • -watchdog_major_threshold: If it has been longer than this period between watchdog executions count a major trip. Default is 3000.
  • -watchdog_major_maxtriggers : How many times the major watchdog timer can be tripped before the test is terminated. Default is 10.
  • -watchdog_minor_maxtriggers: How many times the minor watchdog timer can be tripped before the test is terminated. Default is 120.

Tracing, logging and statistics options:

  • -f : Set the statistics report frequency on screen. Default is 1 and default unit is seconds.
  • -trace_stat : Dumps all statistics in <scenario_name>_.csv file. Use the ‘-h stat’ option for a detailed description of the statistics file content.
  • -stat_delimiter : Set the delimiter for the statistics file
  • -stf : Set the file name to use to dump statistics
  • -fd : Set the statistics dump log report frequency. Default is 60 and default unit is seconds.
  • -periodic_rtd : Reset response time partition counters each logging interval.
  • -trace_msg : Displays sent and received SIP messages in __messages.log
  • -message_file : Set the name of the message log file.
  • -message_overwrite: Overwrite the message log file (default true).
  • -trace_shortmsg : Displays sent and received SIP messages as CSV in <scenario file name>__shortmessages.log
  • -shortmessage_file: Set the name of the short message log file.
  • -shortmessage_overwrite: Overwrite the short message log file (default true).
  • -trace_counts : Dumps individual message counts in a CSV file.
  • -trace_err : Trace all unexpected messages in __errors.log.
  • -error_file : Set the name of the error log file.
  • -error_overwrite : Overwrite the error log file (default true).
  • -trace_error_codes: Dumps the SIP response codes of unexpected messages to <scenario file name>__error_codes.log.
  • -trace_calldebug : Dumps debugging information about aborted calls to <scenario_name>__calldebug.log file.
  • -calldebug_file : Set the name of the call debug file.
  • -calldebug_overwrite: Overwrite the call debug file (default true).
  • -trace_screen : Dump statistic screens in the <scenario_name>__screens.log file when quitting SIPp. Useful to get a final status report in background mode (-bg option).
  • -screen_file : Set the name of the screen file.
  • -screen_overwrite: Overwrite the screen file (default true).
  • -trace_rtt : Allow tracing of all response times in __rtt.csv.
  • -rtt_freq : freq is mandatory. Dump response times every freq calls in the log file defined by -trace_rtt. Default value is 200.
  • -trace_logs : Allow tracing of actions in __logs.log.
  • -log_file : Set the name of the log actions log file.
  • -log_overwrite : Overwrite the log actions log file (default true).
  • -ringbuffer_files: How many error, message, shortmessage and calldebug files should be kept after rotation?
  • -ringbuffer_size : How large should error, message, shortmessage and calldebug files be before they get rotated?
  • -max_log_size : What is the limit for error, message, shortmessage and calldebug file sizes.

Signal handling:

SIPp can be controlled using POSIX signals. The following signals are handled: USR1: Similar to pressing the ‘q’ key. It triggers a soft exit of SIPp. No more new calls are placed and all ongoing calls are finished before SIPp exits. Example: kill -SIGUSR1 732 USR2: Triggers a dump of all statistics screens in <scenario_name>__screens.log file. Especially useful in background mode to know what the current status is. Example: kill -SIGUSR2 732

Exit codes:

Upon exit (on fatal error or when the number of asked calls (-m option) is reached, SIPp exits with one of the following exit code: 0: All calls were successful 1: At least one call failed 97: Exit on internal command. Calls may have been processed 99: Normal exit without calls processed -1: Fatal error -2: Fatal error binding a socket

Debugging

Issue1  The commonName field needed to be supplied and was missing 

Solution Given the common name while generating the certs

Issue2 If cmake error appears such as “command not found: cmake” then 

solutionsudo apt-get install build-essential cmake

References :

Gstreamer

GStreamer ( LGPL )ia a media handling library written in C for applicatioan such as streaming , recording, playback , mixing and editing attributes etc. Even enhnaced applicaiosn such as tsrancoding , media ormat conversion , streaming servers for embeeded devices ( read more about Gstreamer in RPi in my srticle here).
It encompases various codecs, filters and is modular with plugins developement to enhance its capabilities. Media Streaming application developers use it as part of their framework at either the broadcaster’s end or as media player.

gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

More detailed reading :

GStreamer-1.8.1 rtsp server and client on ubuntu – Install and configuration for a RTSP Streaming server and Client https://telecom.altanai.com/2016/05/20/gstreamer-1-8-1-rtsp-server-and-client-on-ubuntu/

crtmpserver + ffmpeg –

https://telecom.altanai.com/2016/06/19/crtmpserver-ffmpeg

Streaming / broadcasting Live Video call to non webrtc supported browsers and media players

 attempts of streaming / broadcasting Live Video WebRTC call to non WebRTC supported browsers and media players such as VLC , ffplay , default video player in Linux etc .

https://telecom.altanai.com/2015/02/17/streaming-broadcasting-live-video-call-to-non-webrtc-supported-browsers-and-media-players/

continue : Streaming / broadcasting Live Video call to non webrtc supported browsers and media players

httontinuation to the attempts / outcomes and problems in building a WebRTC to RTP media framework that successfully stream / broadcast WebRTC content to non webrtc supported browsers ( safari / IE ) / media players ( VLC )

https://telecom.altanai.com/2015/02/26/continue-streaming-broadcasting-live-video-call-to-non-webrtc-supported-browsers-and-media-players/

TO continue with basics of gstreamer keep reading

To list all packages of Gstreamer

pkg-config --list-all | grep gstreamer
  • gstreamer-gl-1.0 GStreamer OpenGL Plugins Libraries – Streaming media framework, OpenGL plugins libraries
  • gstreamer-bad-video-1.0GStreamer bad video library – Bad video library for GStreamer elements
  • gstreamer-tag-1.0 GStreamer Tag Library – Tag base classes and helper functions
  • gstreamer-bad-base-1.0 GStreamer bad base classes – Bad base classes for GStreamer elements
  • gstreamer-net-1.0GStreamer networking library – Network-enabled GStreamer plug-ins and clocking
  • gstreamer-sdp-1.0 GStreamer SDP Library – SDP helper functions
  • gstreamer-1.0 GStreamer – Streaming media framework
  • gstreamer-bad-audio-1.0 GStreamer bad audio library, uninstalled – Bad audio library for GStreamer elements, Not Installedgstreamer-allocators-1.0 GStreamer Allocators Library – Allocators implementation
  • gstreamer-player-1.0 GStreamer Player – GStreamer Player convenience library
  • gstreamer-insertbin-1.0 GStreamer Insert Bin – Bin to automatically and insertally link elements
  • gstreamer-plugins-base-1.0 GStreamer Base Plugins Libraries – Streaming media framework, base plugins libraries
  • gstreamer-vaapi-glx-1.0 GStreamer VA-API (GLX) Plugins Libraries – Streaming media framework, VA-API (GLX) plugins librariesgstreamer-codecparsers-1.0 GStreamer codec parsers – Bitstream parsers for GStreamer elementsgstreamer-base-1.0 GStreamer base classes – Base classes for GStreamer elements
  • gstreamer-app-1.0 GStreamer Application Library – Helper functions and base classes for application integration
  • gstreamer-vaapi-drm-1.0 GStreamer VA-API (DRM) Plugins Libraries – Streaming media framework, VA-API (DRM) plugins librariesgstreamer-check-1.0 GStreamer check unit testing – Unit testing helper library for GStreamer modules
  • gstreamer-vaapi-1.0 GStreamer VA-API Plugins Libraries – Streaming media framework, VA-API plugins libraries
  • gstreamer-controller-1.0 GStreamer controller – Dynamic parameter control for GStreamer elements
  • gstreamer-video-1.0 GStreamer Video Library – Video base classes and helper functions
  • gstreamer-vaapi-wayland-1.0 GStreamer VA-API (Wayland) Plugins Libraries – Streaming media framework, VA-API (Wayland) plugins libraries
  • gstreamer-fft-1.0 GStreamer FFT Library – FFT implementation
  • gstreamer-mpegts-1.0 GStreamer MPEG-TS – GStreamer MPEG-TS support
  • gstreamer-pbutils-1.0 GStreamer Base Utils Library – General utility functions
  • gstreamer-vaapi-x11-1.0 GStreamer VA-API (X11) Plugins Libraries – Streaming media framework, VA-API (X11) plugins libraries
  • gstreamer-rtp-1.0 GStreamer RTP Library – RTP base classes and helper functions
  • gstreamer-rtsp-1.0 GStreamer RTSP Library – RTSP base classes and helper functions
  • gstreamer-riff-1.0 GStreamer RIFF Library – RIFF helper functions
  • gstreamer-audio-1.0 GStreamer Audio library – Audio helper functions and base classes
  • gstreamer-plugins-bad-1.0 GStreamer Bad Plugin libraries – Streaming media framework, bad plugins libraries
  • gstreamer-rtsp-server-1.0 gst-rtsp-server – GStreamer based RTSP server

At the time of writing this article Gstreamer an much early version in 1.X , which was newer than its then stable version 0.x. Since then the library has updated many fold. summarising release highlights for major versions as the blog was updated over time .

Project : Making and IP survillance system using gstreamer and Janus

To build a turn-key easily deployable surveillance solution 

Features :

  1. Paring of Android Mobile with box
  2. Live streaming from Box to Android
  3. Video Recording inside the  box
  4. Auto parsing of recorded video around motion detection 
  5. Event listeners 
  6. 2 way audio
  7. Inbuild Media Control Unit
  8. Efficient use of bandwidth 
  9. Secure session while live-streaming

Modules

  1. Authentication ( OTP / username- password)
  2. Livestreaming on Opus / vp8 
  3. Session Security and keepalives for live-streaming sessions
  4. Sync local videos to cloud storage 
  5. Record and playback with timeline and events 
  6. Parsing and restructuring video ( transcoding may also be required ) 
  7. Coturn server for NAT and ICE
  8. Web platform on box ( user interface )+ NoSQL
  9. Web platform on Cloud server ( Admin interface )+ NoSQL
  10.  REST APIs for third party add-ons ( Node based )
  11. Android demo app for receiving the live stream and feeds

Varrying experiments and working gstreamer commands

Local Network Stream 

To create /dev/video0

modprobe bcm2835-v4l2

To stream on rtspserver using rpicamsrc using h264 parse

./gst-rtsp-server-1.4.4/examples/test-launch --gst-debug=2 '(rpicamsrc num-buffers=5000 ! 'video/x-h264,width=1080,height=720,framerate=30/1' ! h264parse ! rtph264pay name=pay0 pt=96 )'

./test-launch “( tcpclientsrc host=127.0.0.1 port=5000 ! gdpdepay ! rtph264pay name=pay0 pt=96 )”

pipe raspivid to tcpserversink

raspivid -t 0 -w 800 -h 600 -fps 25 -g 5 -b 4000000 -vf -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! gdppay ! tcpserversink host=127.0.0.1 port=5000;

Stream Video over local Network with 15 fps

raspivid -n -ih -t 0 -rot 0 -w 1280 -h 720 -fps 15 -b 1000000 -o - | nc -l -p 5001

streaming video over local network with 30FPS and higher bitrate

raspivid -n -t 0 -rot 0 -w 1920 -h 1080 -fps 30 -b 5000000 -o - | nc -l -p 5001

Recording

Audio record to file
Using arecord :

arecord -D plughw:1 -c1 -r 48000 -f S16_LE -t wav -v file.wav;

Using pulse :
pulseAudio -D

gst-launch-1.0 -v pulsesrc device=hw:1 volume=8.0 ! audio/x-raw,format=S16LE ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! flvmux ! filesink location = "testaudio.flv";

Video record to file ( mpg)

gst-launch-1.0 -e rpicamsrc bitrate=500000 ! 'video/x-h264,width=640,height=480’ ! mux. avimux name=mux ! filesink location=testvideo2.mpg;

Video record to file ( flv )

gst-launch-1.0 -e rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! flvmux ! filesink location="testvieo.flv";

Video record to file ( h264)
gst-launch-1.0 -e rpicamsrc bitrate=500000 ! filesink location=”raw3.h264″;

Video record to file ( mp4)

gst-launch-1.0 -e rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! mp4mux ! filesink location=video.mp4;

Audio + Video record to file ( flv)

gst-launch-1.0 -e /
rpicamsrc bitrate=500000 ! /
video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! muxout. /
pulsesrc volume=8.0 ! /
queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. /
flvmux name=muxout streamable=true ! filesink location ='test44.flv';

Audio + Video record to file ( flv) using pulsesrc

gst-launch-1.0 -v --gst-debug-level=3 pulsesrc device="alsa_input.platform-asoc-simple-card.0.analog-stereo" volume=5.0 mute=FALSE ! audio/x-raw,format=S16LE,rate=48000,channels=1 ! audioresample ! audioconvert ! voaacenc ! aacparse ! flvmux ! filesink location="voicetest.flv";

Audio + Video record to file (mp4)

gst-launch-1.0 -e /
rpicamsrc bitrate=500000 ! /
video/x-h264,width=320,height=240,framerate=10/1 !s h264parse ! muxout. /
pulsesrc volume=4.0 ! /
queue ! audioconvert ! voaacenc ! muxout. /
flvmux name=muxout streamable=true ! filesink location = 'test224.mp4';

Streaming

stream raw Audio over RTMP to srtmpsink

gst-launch-1.0 pulsesrc device=hw:1 volume=8.0 ! /
audio/x-raw,format=S24LE ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! flvmux ! rtmpsink location = “rtmp://192.168.0.3:1935/live/test”;

stream AACpparse Audio over RTMP to srtmpsink

gst-launch-1.0 -v --gst-debug-level=3 pulsesrc device="alsa_input.platform-asoc-simple-card.0.analog-stereo" volume=5.0 mute=FALSE ! audio/x-raw,format=S16LE,rate=48000,channels=1 ! audioresample ! audioconvert ! voaacenc ! aacparse ! flvmux ! rtmpsink location="rtmp://www.altani.com:1935/voice/1/test";

stream Video over RTMP

gst-launch-1.0 -e rpicamsrc bitrate=500000 ! /
video/x-h264,width=320,height=240,framerate=6/1 ! h264parse ! /
flvmux ! rtmpsink location = ‘rtmp://52.66.125.31:1935/live/test live=1’;

stream Audio + video over RTMP from rpicamsrc , framerate 10

gst-launch-1.0 rpicamsrc bitrate=500000 ! video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! muxout. pulsesrc volume=8.0 ! queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. flvmux name=muxout streamable=true ! rtmpsink location ='rtmp://www.altanai.com/live/test44';

stream Audio + video over RTMP from rpicamsrc , framerate 30

gst-launch-1.0 rpicamsrc bitrate=500000 ! video/x-h264,width=1280,height=720,framerate=30/1 ! h264parse ! muxout. pulsesrc ! queue ! audioconvert ! voaacenc bitrate=65536 ! aacparse ! muxout. flvmux name=muxout ! queue ! rtmpsink location ='rtmp://www.altanai.com/live/test44';

VOD ( video On Demand )

Stream h264 file over RTMP

gst-launch-1.0 -e filesrc location="raw3.h264" ! video/x-h264 ! h264p
arse ! flvmux ! rtmpsink location = 'rtmp://www.altanai.com/live/test';

Stream flv file over RTMP

gst-launch-1.0 -e filesrc location=”testvieo.flv” ! /
video/x-h264,width=320,height=240,framerate=10/1 ! h264parse ! /
flvmux ! rtmpsink location = 'rtmp://192.168.0.3:1935/live/test';

Github Repo for Livestreaming

https://github.com/altanai/Livestreaming

Contains code for Android and ios Publishers , players on various platforms including HLS and Flash , streamings servers , Wowza playing mosules , webrtc broadcast

Gstreamer 1.8.0 – 24 March 2016

Features Hardware-accelerated zero-copy video decoding on Android

New video capture source for Android using the android.hardware.Camera API

Windows Media reverse playback support (ASF/WMV/WMA)

tracing system provides support for more sophisticated debugging tools

high-level GstPlayer playback convenience API

Initial support for the new Vulkan API

Improved Opus audio codec support: Support for more than two channels; MPEG-TS demuxer/muxer can handle Opus; sample-accurate encoding/decoding/transmuxing with Ogg, Matroska, ISOBMFF (Quicktime/MP4), and MPEG-TS as container; new codec utility functions for Opus header and caps handling in pbutils library. The Opus encoder/decoder elements were also moved to gst-plugins-base (from -bad), and the opus RTP depayloader/payloader to -good.

Asset proxy support in the GStreamer Editing Services

GStreamer 1.16.0 – 19 April 2019.

GStreamer WebRTC stack gained support for data channels for peer-to-peer communication based on SCTP, BUNDLE support, as well as support for multiple TURN servers.

AV1 video codec support for Matroska and QuickTime/MP4 containers and more configuration options and supported input formats for the AOMedia AV1 encoder

Closed Captions and other Ancillary Data in video

planar (non-interleaved) raw audio

GstVideoAggregator, compositor and OpenGL mixer elements are now in -base

New alternate fields interlace mode where each buffer carries a single field

WebM and Matroska ContentEncryption support in the Matroska demuxer

new WebKit WPE-based web browser source element

Video4Linux: HEVC encoding and decoding, JPEG encoding, and improved dmabuf import/export

Hardware-accelerated Nvidia video decoder gained support for VP8/VP9 decoding, whilst the encoder gained support for H.265/HEVC encoding.

Improvements to the Intel Media SDK based hardware-accelerated video decoder and encoder plugin (msdk): dmabuf import/export for zero-copy integration with other components; VP9 decoding; 10-bit HEVC encoding; video post-processing (vpp) support including deinterlacing; and the video decoder now handles dynamic resolution changes.

ASS/SSA subtitle overlay renderer can now handle multiple subtitles that overlap in time and will show them on screen simultaneously

Meson build feature-complete (with the exception of plugin docs) and it is now the recommended build system on all platforms. The Autotools build is scheduled to be removed in the next cycle.

GStreamer Rust bindings and Rust plugins module

GStreamer Editing Services allows directly playing back serialized edit list with playbin or (uri)decodebin

References :

https://gstreamer.freedesktop.org

OTT ( Over the Top ) Communication applications

Market trends are really not in favor of Telecom Service /providers with increasing use of OTT ( Over The Top ) application like watsapp , Facebook messenger , Google hangouts , skype  , viber , etc .

OTT
OTT ( Over The Top ) Applications

What is an OTT ?

An Over The Top ( OTT ) application is one which provides communication services over Internet . Therefore these bypass the communication billing system setup by a Telecom Operator , resulting in no gain or loss of revenue to Telecom Operator who is providing the Internet service to user in first place .

Hence we see that OTT are major threat and concern for Telecom Operators whose traditional and obviously expensive ( when compared to OTTs free service ) billing models are facing disruption .


Telecom Regulatory bodies around the world

The telecom regulatory authorities in some of the countries are for example listed as :

  • Afghanistan Telecom Regulatory Authority (ATRA) – Afganistan
  • Australian Communications and Media Authority (ACMA) – Australia
  • Bangladesh Telecommunication Regulatory Commission (BTRC) – Bnagaladesh
  • Canadian Radio-television and Telecommunications Commission (CRTC) – Canada
  • Ministry of Information Industry (MII) – China
  • Autorité de Régulation des Communications Électroniques et des Postes (ARCEP) – France
  • Bundesnetzagentur (BNA) – Germany
  • Telecom Regulatory Authority of India (TRAI) – India
  • Ministry for Communications and Informatization of the Russian Federation (Minsvyaz) – Russia
  • Infocomm Development Authority of Singapore (IDA) – Singapore
  • Independent Communications Authority of South Africa (ICASA) – south Africa
  • Federal Communications Commission (FCC) , National Association of Regulatory Utility Commissioners (regulators of individual states) (NARUC) , CTIA – The Wireless Association (CTIA) – USA

Such telecom regulatory bodies get to decide whether to enforce differential price to end consumers for using OTT so that telecom service providers can benefit or keep the Internet fair and open by passing Net Neutrality Laws and Bills and amendments .

what is Net Neaurality ?

The fundamental principle of Net Neurality is that Telecom Operators should not block , slow down or charge consumers extra for using other services as their means of communication. This states that it is wrong to charge users above the regular data rates for using VOIP apps and other internet based communication services.

The following counteries have adopted principles of Net Neutrality by passing bills or making law .

  • Chile – Chile’s General Law of Telecommunications, “No [ISP] can block, interfere with, discriminate, hinder, nor restrict the right of any Internet user of using, send, receive, or offer any content, application, or legitimate service through the Internet, as well as any activity or legitimate use conducted through the Internet.”
  • Brazil – ” Internet Bill of Rights ” makes equal access to internet mandatory in Brazil .
  • Netherlands – Even European Union has adopted Netherlands’ Net Neutrality amendment which reads “traffic should be treated equally, without discrimination, restriction or interference, independent of the sender, receiver, type, content, device, service or application.”
  • USA – Citizens make ‘We the People’ platform to ‘Restore Net Neutrality By Directing the Federal Communications Commission (FCC) to Classify Internet Providers as ‘Common Carriers‘. Therefore not allowing them to either throttle speed by paid prioritization , discriminate in pricing or block any broadband access to legal content .  Above facts are from this tech.firstpost.com article.

Inspite of the fact that I Support Net Neutrality with all my heart , as a telecom engineer I understand the cost investment made by Telecom operators in providing am efficient communication network to its subscribers ( Access , Network and Application layers ). Therefor I do have my sympathies with the Telcos and to level out the wide ranging conflict between Telcos and  ISP ( Internet Service Providers ) , I pen down the following points which reflect the Telecom Operators Problems and also highlight the solutions that can be adopted to counteract the OTT threat .

Depleting revenue for Telco

  1. Messaging – OTT messaging cost operators $13.9 billion, or 9% of message revenue in 2013
  2. Voice – Voice services under threat from VOIP services like Skype, Viber
  3. OTT apps – Voice & Message apps have been the operator’s biggest headache. Its time Operator should launch its own OTT Services
  4. Data Traffic – The utilization is yet to reach its peak. Will face challenges from  WiFi access
  5. Critical Pain areas – Erosion of Operator’s revenue from voice and (especially) messaging

Telco’s OTT aPPLICATION

At this stage it is crucial for a telecom Service provider / Operator to enter the Apps market and bring forth a Messenger which is more powerful , interactive and awesome than a OTT application.  Fortunately the Operator can always couple this application with his background telecom infrastructure to provide the edge in performance and functionalists .

Road block while developing a OTT application for a Telecom Service Provider :

  • Investment in Data Network is not being utilized due to lack of service
  • Reuse of Existing business Logic and extending the service reach across devices and networks is tough
  • Operator already has full fledged network Infrastructure in Place
  • Desire for minimum CAPEX while investing in new technologies
  • compete with OTT players and open new revenue streams is a challenge

Next we find the way of solving the problems and integrating them together to form a Solution .

OTT Application for Telecom Service provider

  • Introduce new services to benefit from investment on Data Plans and Bandwidth
  • Expose REST API to enable 3trd party Integration with existing network Infrastructure
  • Partner with individual OTT players to make new services  that do not compete on core competencies like billing etc
  • Use protocols like SIP that reduce CAPEX and have goto market more quickly
  • Go for enriched service that lead to better user experience

This writeup outlines the process of creating a OTT application for a Telecom Service Provider . Components for the application include cloud Address Book , Video Chatting , Location share , Contact synchronization ,REST based thin  client , OS and device agnostic etc shown in the figure below

telco's OTT app
telco’s OTT app

The Application  is designed to close knit with Operator’s own infrastructure hence the crucial entities like Network Address Book , Location Service are synced and fetched from Backend Network .

OTT application Feature Overview

Smart Address Book

  • Automatic: Get contacts from Gmail, Facebook
  • Fast search by first, last name, frequently
  •   dialed number
  • Roadmap: View calendar events
  • Personal: Get image from Gmail and display in   contacts list

Geo Location

  • Share own location during chatting
  • Get map for calculating the distance between two chat users
  • Roadmap : Trigger device (say Switch on/off AC before reaching home) from a threshold distance away from home   location

Messaging

  • Ad-hoc Chat
  • Session Based Chat
  • Voice Input for texting
  • Presence information of contacts
  • RoadMap: Legacy message integration

Telephony

  • Voice call to mobile
  • Voice call to PSTN
  • Video call to other @imAll user
  • Share images during voice call to other

Device agnostic

  • Compatible with IOS, windows
  • Can run as native app on ipad
  • Can run as browser client on windows
  • RoadMap: native app for android, windows phone,blackberry10

Roadmap

  • To upgrade the application and provide enganced and enrich service support the I propose the following roadmap.
  • From plain vanilla voice and video calling ( supported by every other OTT application ) our application should progress towards  legacy telecom support whihc included PSTN , GSM , ISDN etc . This requires backbone of telecom network and a good setup for media codec conversion to suit various legacy media codecs .

Road Map  from Traditional to New age services 

  1. Voice and video calling
  2. Legacy services support like MMS and SMS
  3. Integration with 3rd party Vendors
  4. Give new enriched services like Multilingual support , file transfer , screen-sharing etc
  5. give facility to integrated web plugins for web calling

To keep the interest of customers it is essential that the application be supported on other popular OTT services like skype  , Gtalk . for exmaple a caller should be able to make call from Skype  / Gtalk to our application .Multilingual capabilities, support for larger protocol spectrum will just act like icing on the cake .

How does it benefit the Operator??

  1.  Saves on development cost and time
  2.  Device Agnostic OTT Applications
  3. Simplified Service deployment
  4. Saves licensing cost per client
  5. Reuses existing Messaging and   Address Book service logic.
  6. Open New Revenue Streams for operator
  7. No separate SIP stack required for the client
  8.  Faster Time to Market

Update : At the time of writing this post I did not anticipate the wave of change that bring focus on subjects like “net neutrality” , ” Save the internet” and “free internet” . However I see now that I had described this phenomenon way in advance for my time .


Legacy Telecom Networks

I use the term legacy telecom system many a times , but have not really described what a legacy system actually is . In my conferences too I am asked to just exactly define a legacy system . Often my clients are surprised to hear what they have in current operation is actually fitted in our own version of definition of ” Legacy system “. This write up is an attempt to describe the legacy landscape . It also describes its characteristics, elements and transformation .

Characteristics of Legacy Systems

1. Analog Signals

1G , introduced in 1980s , used analog signals as compared to digital in 2G onward. In 1G voice was modulated to higher frequency and then converted to digital while communication with radio towers .

2.Legacy system have ATM / Frame Relay transmission .

This is basically Hardware Specific and results in High Expenses.

3. Legacy systems have POTS / PSTN / ISDN as their access layer technology .

Access layer is the first layer of telecom architecture which is responsible for interacting directly with the end use / subscriber . Legacy system technologies are again Hardware  Specific , bear High Expenses and offer Low stability.

Physical transmission media include :

  • Twisted wire (modems)
  • Coaxial cable
  • Fiber optics and optical networks – Dense wavelength division multiplexing (DWDM)

4. Legacy system use Traditional Switches / ISDN in their Core Layer

Core layer is the main control hub of the entire telecom architecture . Using old fashioned switches render high CAPEX ( capital Expenditure ) and OPEX ( Operational Expenses ) .

5. In the service delivery front legacy system employ Traditional IN switches

These are very Hardware Centric.

Services part of Legacy Telecom Networks

a) Virtual Private Network (VPN)

An Intelligent Network (IN) service, which offers the functions of a private telephone network. The basic idea behind this service is that business customers are offered the benefits of a (physical) private network, but spared from owning and maintaining it

b) Access Screening(ASC)

 An IN service, which gives the operators the possibility to screen (allow/barring) the incoming traffic and decide the call routing, especially when the subscribers choose an alternate route/carrier/access network (also called Equal Access) for long distance calls on a call by call basis or pre-selected.

c) Number Portability(NP)

An IN service allows subscribers to retain their subscriber number while changing their service provider, location, equipment or type of subscribed telephony service. Both geographic numbers and non-geographic numbers are supported by the NP service.

Also Read


Transformation towards IMS (Total IP)

The telecommunications industry has been going through a significant transformation over the past few years. At the outset incumbent operators used to focus on mainly basic voice services and still remained profitable due to the limited number of players in the space and requirement of huge amounts as initial investment.

However, with the advent of competitive vendors, rise in consumer base, and introduction of cost effective IP based technologies a major revolution has come about. This has enabled operators to come out of their traditional business models to maintain and enhance subscriber base by providing better and cheaper voice, multimedia and data services in order to grab the biggest possible share in this multi- billion dollar industry.

The evolution in Telecom industry has been accelerating all the time. The Next-Generation Operators wants to keep pace with the rapidly changing technology by, adapting to market needs and looking at the system and business process from multiple perspectives concurrently. Communication Service Providers (CSPs) need to consider several factors in mind before proposing any solution. They need to deploy solutions which are highly automated, highly flexible, caters to customer needs coupled with ultra low operating costs.

By hosting new services on the new platform and combining new and old services CSP‟s aim to provide service bundles that would generate new revenue streams. This process is largely dependant on IMS ( IP Multimedia Subsystem ) architecture .

Transformation towards IMS (Total IP)
Transformation towards IMS (Total IP)

Optimization in operator landscape evolve as result of synergistic technologies that come together to address the innovation and cost optimization needs of operator for better user experience. In following sections different technological evolutions that are affecting overall operator ecosystems have been discussed with focus towards Service Layer.

Legacy to IP transformation

This section broadly covered the aspects of migration from legacy IN solution to new age JAINSLEE framework based one. Applies to Legacy IN hosting voice based services mostly  such as VPN, Access Screening ,Number Portability, SIP-Trunking ,Call Gapping.

Most operator environments have seen a rise in the number of service delivery platforms. Also complexity of telecom networks have increased manifold hence CSPs are facing multiple challenges. Increased efforts and costs are required for maintaining all the SDP platforms. These platforms are generally of different vendors and cater to different technologies thereby greatly increase chances of limiting the scalability and flexibility of the operator landscape. More effort required for sustaining the life cycle of the platform and challenges in integrating non compatible SDPs due to proprietary design have been stumbling blocks in the progress of CSPs across the world.

To overcome these challenges there is trend in the market to move towards SDP consolidation wherein instead of maintaining several SDPs with their proprietary design CSPs prefer maintaining a single or less number of SDPs having standardized interfaces.

SDP consolidation SDP consolidation (1) SDP consolidation (2)

As illustrated in the above figure there is a transition that is taking place in the industry towards consolidation of service delivery session control. This would provide a cost effective sustenance of existing applications and the rapid creation and deployment of new services leading to increased revenue recognition by CSPs.

  • Agile Development
  • Innovative services
  • open SOA based architectures
  • IN/NGN Platform and Services
  • Reuse of existing investments in legacy service platforms
  • low cost of new service development
  • faster time to market
  • Monetize investment in Network Infrastructure uplift – SIP trunking, VoLTE etc.

Services that should be covered  in the Scope of Migration from fixed line to IP telephony are:

  • Virtual Private Network (VPN) : An Intelligent Network (IN) service, which offers the functions of a private telephone network. The basic idea behind this service is that business customers are offered the benefits of a (physical) private network, but spared from owning and maintaining it.
  • Access Screening(ASC): An IN service, which gives the operators the possibility to screen (allow/barring) the incoming traffic and decide the call routing, especially when the subscribers choose an alternate route/carrier/access network (also called Equal Access) for long distance calls on a call by call basis or pre-selected.
  • Number Portability(NP) : An IN service allows subscribers to retain their subscriber number while changing their service provider, location, equipment or type of subscribed telephony service. Both geographic numbers and non-geographic numbers are supported by the NP service.

WebRTC based Unified Communication platform

Using WebRTC Solution for Delivering In Context Voice which provides new monetizing benefits to the Enterprise customers of Service Providers. This includes following components:

  • WebRTC Gateway for implementation for inter-connect with SIP Legacy
  • Enhancement of WebRTC Client with new features like Cloud Address Book, Conferencing & Social Networking hooks.
  • Cloud based solutions

INtoJAISNLEE

Challenges in Migration to IMS  (Total IP )

Since long I have been advocating the benefits of migration to IMS  from a current fixed line / legacy/ proprietary VOIP / SS7 based system . However I decided to write this post on the challenges in migration to IMS system from a telecom provider’s view.  Though I could think of many , I have jot down the major 4 . they are as follows :

Data Migration challenges

  • Establishing a common data model definition
  • Data migration seamlessly
  • Configuration management
  • Extracting data from multiple sources and vendors , that includes legacy systems
  • Extracting data due to its large scale and volume

Training

  • Creating an effective knowledge share and transfer for live operations
  • Training in fallback plans, standards and policies .

Customer impact

  • Minimized customer outage
  • Enhance customer experience by delivering quality services on schedule
  • Ensuring security of customer’s confidential data
  • Transfer of customer services without any impact.

Testing in replicated environment

  • Physical pre-transfer test
  • Reducing cycle time
  • Verification and validation at every change in data environment
  • Detect production issues early in the test -lifecycle

Fallback plans

  • Pilot program and real network simulation for ensuring preparedness
  • Tracking changes in new network


Service Broker Architecture for IN and IMS

We know that Service broker is a service abstraction layer between the network and application layer in  telecom environment.SB( Service Broker ) enables us to make use of existing applications and services from Intelligent Network’s SCP ( Service control Point ) , IMS’s Application Server as well as other sources  in a harmonized manner .

service broker

The service provider can  combine the services from various sources written in various languages in numerous permutations and combinations .  This saves the time , energy and rework required to launch a new services.

I have written couple of posts before on Service Broker .Post on What is Service Broker . It definitions and application can be found here  : https://altanaitelecom.wordpress.com/2013/03/19/service-broker/. This also defines service orchestration and harmonization .

Another post on Service Borker’s role and function can be found here : https://altanaitelecom.wordpress.com/2013/08/07/service-broker-2/. This mentions the service brokering role in network environment. But ofcourse it was a mere introduction  . The following post clarifies the concept in greater light . 

I believe and it truly is a wonderful thing to make use of Service Broker while network migration from IN to IMS .The following architecture model depict the placement of Service Broker component in IN and IMS integrated environment .

sb1

The figure above portrays how a  service provider acts as a central Node for Services invocation and services composition. SB is responsible for Services Orchestration / Interaction , service development, third party integration and acts like a protocol gateway .

Let us discuss service broker in a full fleshed network’s structure . It includes the access network components and detailed core network components with the name of interfaces between all nodes.

sb2

The Applications as described by the above figure could be majorly of 4 types :

1. applications developed on a SIP application Server and invoked via SIP/ ISC

2. Applications developed over SIP servlets or JAINSLEE platform such as mobicents , Opencloud Rhino etc

3. Application developed on a SCP ( Service Control Point ) of a IN ( Intelligent Network ) . This is invoked via INAP CS1/CS1+ or CAP

4. Application developed on a J2EE server Invocated via http REST API like GSMA OneAPI such as

  • Call Control API for voice.
    Messaging API for SMS, MMS.
    localisation API.

Provisioning via fixed/mobile brands & « service profile» in SB

Provisioning via fixed/mobile brands & « service profile» in Service Broker
Provisioning via fixed/mobile brands & « service profile» in Service Broker

BDD « Services » in SB

BDD « Services » in ServiceBroker
BDD « Services » in Service Broker

Architecture of SDP / Service Broker

Architecture of SDP / Service Broker
Architecture of SDP / Service Broker