master/coturn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,467 Commits 1 Branch 0 Tags 4.8 MiB
C 90.4%
CMake 4.4%
Shell 1.7%
C++ 1.7%
Makefile 1.1%
Other 0.6%
13ecb7d012
Go to file
Thibaut ACKERMANN 13ecb7d012
Redis memleaks and socketleaks (#1015) 
Hello,

while using the `redis-statsdb` option, I found that coturn is leaking
sockets (and memory) in case of redis reconnection.

This occurs a lot to me, because in my setup I have a `coturn -> haproxy
-> redis` and if all my redis servers are down, HaProxy abruptly close
the connection to coturn and coturn reconnects periodically. After some
time I can see thousands of pending sockets (`CLOSE_WAIT`) :

```
user@server[11:32:48 UTC]:~$ sudo lsof -i | grep turn
turnserve 461797       root   15u  IPv4 12856075      0t0  TCP server:3478 (LISTEN)
turnserve 461797       root   22u  IPv4 12856081      0t0  TCP server:3478 (LISTEN)
turnserve 461797       root   23u  IPv4 12857384      0t0  UDP server:3478 
turnserve 461797       root   24u  IPv4 12857385      0t0  UDP server:3478 
turnserve 461797       root   36u  IPv4 12857390      0t0  TCP server:5766 (LISTEN)
turnserve 461797       root   43u  IPv4 12856096      0t0  TCP server:10059->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   46u  IPv4 12857403      0t0  TCP server:10087->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   48u  IPv4 12856124      0t0  TCP server:53867->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   50u  IPv4 12857633      0t0  TCP server:53875->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   51u  IPv4 12856138      0t0  TCP server:53877->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   54u  IPv4 12857738      0t0  TCP server:10001->haproxy-server:redis (CLOSE_WAIT)
turnserve 461797       root   55u  IPv4 12856152      0t0  TCP server:10003->haproxy-server:redis (CLOSE_WAIT)
.... (many many lines)
```


After searching and using valgrind I found 2 interesting leaks:

```
...
==460721== 32 bytes in 1 blocks are definitely lost in loss record 586 of 1,053
==460721==    at 0x483877F: malloc (vg_replace_malloc.c:307)
==460721==    by 0x1414FF: RyconninfoParse (dbd_redis.c:69)
==460721==    by 0x141B04: get_redis_async_connection (dbd_redis.c:169)
==460721==    by 0x110D7B: create_ioa_engine (ns_ioalib_engine_impl.c:407)
==460721==    by 0x12ECB0: setup_admin_thread (turn_admin_server.c:1309)
==460721==    by 0x127724: run_admin_server_thread (netengine.c:1815)
==460721==    by 0x4DA9EA6: start_thread (pthread_create.c:477)
==460721==    by 0x4EC0AEE: clone (clone.S:95)
...
==460979== 2,170 (688 direct, 1,482 indirect) bytes in 2 blocks are definitely lost in loss record 1,029 of 1,049
==460979==    at 0x483AD7B: realloc (vg_replace_malloc.c:834)
==460979==    by 0x49A1BD0: ??? (in /usr/lib/x86_64-linux-gnu/libhiredis.so.0.14)
==460979==    by 0x49A2829: redisAsyncConnect (in /usr/lib/x86_64-linux-gnu/libhiredis.so.0.14)
==460979==    by 0x13DB37: redis_reconnect (hiredis_libevent2.c:331)
==460979==    by 0x13D1A7: redisLibeventReadEvent (hiredis_libevent2.c:101)
==460979==    by 0x4D5135E: ??? (in /usr/lib/x86_64-linux-gnu/libevent_core-2.1.so.7.0.1)
==460979==    by 0x4D51A9E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent_core-2.1.so.7.0.1)
==460979==    by 0x126D5A: run_events (netengine.c:1579)
==460979==    by 0x127272: run_general_relay_thread (netengine.c:1707)
==460979==    by 0x4DA9EA6: start_thread (pthread_create.c:477)
==460979==    by 0x4EC0AEE: clone (clone.S:95)
==460979== 
...
```

I made 1 commit for each fix.

Obviously with these fixes, I don't have anymore the leaks of thousands
of sockets (even after some time)

Thanks & hope it helps.

Thibaut
2022-10-11 21:57:23 +02:00
.github Build CI with prometheus support (#999) 2022-09-29 14:03:03 -07:00
cmake Add cmake 2021-02-26 09:02:50 +08:00
docker Upgrade Docker image to 4.6.0 Coturn version 2022-09-13 12:38:37 +03:00
examples Use single SSL_CTX for DTLS support (#996) 2022-10-05 10:26:46 +02:00
man/man1 Merge branch 'sysvinit/toggle-username-labels-upstream' of github.com:wireapp/coturn into wireapp-sysvinit/toggle-username-labels-upstream 2022-07-29 09:47:17 +02:00
rpm Fixes #692 2021-01-18 10:31:25 +01:00
src Redis memleaks and socketleaks (#1015) 2022-10-11 21:57:23 +02:00
turndb Add hash algorithm for key value to redis userdb schema 2021-01-14 09:57:10 -06:00
.dockerignore Avoid duplication via common rootfs/ dir 2021-04-20 10:36:52 +03:00
.gitignore Add 'debian' image and run as non-root 2021-04-01 17:41:55 +03:00
.travis.yml Remove trusty add focal 2021-05-08 06:39:35 +02:00
AUTHORS replace email address 2018-09-27 22:09:10 +02:00
ChangeLog Replace references to non-existent pdf file with links (#1002) 2022-09-29 11:09:10 -07:00
CMakeLists.txt Fix cmake build on macOS (#949) 2022-08-22 16:27:21 -07:00
configure Preserve file timestamps when using install(1) (#983) 2022-09-16 09:44:00 +02:00
INSTALL Modify INSTALL 2021-04-05 11:26:49 +08:00
LICENSE initial code import 2014-04-20 21:10:18 +00:00
LICENSE.OpenSSL initial code import 2014-04-20 21:10:18 +00:00
make-man.sh man pages util fixed 2017-02-20 01:10:38 -08:00
Makefile.in Move acme to new file 2020-12-15 13:04:11 +00:00
netarch.txt fix typo in netarch.txt 2018-09-04 08:02:42 +02:00
NOTE working on 4.4.5.4 2015-07-18 21:54:16 -07:00
postinstall.txt fix(doc): Updated copy on postinstall.txt 2018-05-14 16:32:32 -07:00
README.md Fix "enabeled" typo in README (#831) 2021-10-05 14:12:10 +03:00
README.turnadmin Update README to fix #658 2020-12-08 13:40:49 +01:00
README.turnserver Use a single SSL context object (#989) 2022-09-28 09:50:25 +02:00
README.turnutils Update README to fix #658 2020-12-08 13:40:49 +01:00
STATUS working on multi-tenant server based upon oauth 2015-09-14 00:16:13 -07:00
TODO TODO updated 2015-04-11 05:45:18 +00:00

README.md

This project evolved from rfc5766-turn-server project (https://code.google.com/p/rfc5766-turn-server/). There are many new advanced TURN specs which are going far beyond the original RFC 5766 document. This project takes the code of rfc5766-turn-server as the starter, and adds new advanced features to it.

Downloads page

Docker image

Wiki pages

Free open source implementation of TURN and STUN Server

The TURN Server is a VoIP media traffic NAT traversal server and gateway. It can be used as a general-purpose network traffic TURN server and gateway, too.

On-line management interface (over telnet or over HTTPS) for the TURN server is available.

The implementation also includes some extra experimental features.

Supported RFCs:

TURN specs:

STUN specs:

  • RFC 3489 - "classic" STUN
  • RFC 5389 - base "new" STUN specs
  • RFC 5769 - test vectors for STUN protocol testing
  • RFC 5780 - NAT behavior discovery support
  • RFC 7443 - ALPN support for STUN & TURN
  • RFC 7635 - oAuth third-party TURN/STUN authorization

Supported ICE and related specs:

  • RFC 5245 - ICE
  • RFC 5768 – ICE–SIP
  • RFC 6336 – ICE–IANA Registry
  • RFC 6544 – ICE–TCP
  • RFC 5928 - TURN Resolution Mechanism

The implementation fully supports the following client-to-TURN-server protocols:

Supported relay protocols:

  • UDP (per RFC 5766)
  • TCP (per RFC 6062)

Supported user databases (for user repository, with passwords or keys, if authentication is required):

  • SQLite
  • MySQL
  • PostgreSQL
  • Redis
  • MongoDB

Redis can also be used for status and statistics storage and notification.

By default a prometheus exporter endpoint is disabled, if it is enabled it will listen on port 9641 under path /metrics

Supported message integrity digest algorithms:

  • HMAC-SHA1, with MD5-hashed keys (as required by STUN and TURN standards)

Supported TURN authentication mechanisms:

  • 'classic' long-term credentials mechanism;
  • TURN REST API (a modification of the long-term mechanism, for time-limited secret-based authentication, for WebRTC applications: http://tools.ietf.org/html/draft-uberti-behave-turn-rest-00);
  • experimental third-party oAuth-based client authorization option;

When used as a part of an ICE solution, for VoIP connectivity, this TURN server can handle thousands simultaneous calls per CPU (when TURN protocol is used) or tens of thousands calls when only STUN protocol is used. For virtually unlimited scalability a load balancing scheme can be used. The load balancing can be implemented with the following tools (either one or a combination of them):

  • DNS SRV based load balancing;
  • built-in 300 ALTERNATE-SERVER mechanism (requires 300 response support by the TURN client);
  • network load-balancer server.

Traffic bandwidth limitation and congestion avoidance algorithms implemented.

The supported project target platforms are:

  • Linux (Debian, Ubuntu, Mint, CentOS, Fedora, Redhat, Amazon Linux, Arch Linux, OpenSUSE)
  • BSD (FreeBSD, NetBSD, OpenBSD, DragonFlyBSD)
  • Solaris 11
  • Mac OS X
  • Cygwin (for non-production R&D purposes)

Other server platforms can be supported by request.

Any client platform is supported, including Android, iOS, Linux, OS X, Windows, and Windows Phone.

This project can be successfully used on other *NIX platforms, too, but that is not officially supported.

The implementation is supposed to be simple, easy to install and configure. The project focuses on performance, scalability and simplicity. The aim is to provide an enterprise-grade TURN solution.

To achieve high performance and scalability, the TURN server is implemented with the following features:

  • High-performance industrial-strength Network IO engine libevent2 is used
  • Configurable multi-threading model implemented to allow full usage of available CPU resources (if OS allows multi-threading)
  • Multiple listening and relay addresses can be configured
  • Efficient memory model used
  • The TURN project code can be used in a custom proprietary networking environment. In the TURN server code, an abstract networking API is used. Only couple files in the project have to be re-written to plug-in the TURN server into a proprietary environment. With this project, only implementation for standard UNIX Networking/IO API is provided, but the user can implement any other environment. The TURN server code was originally developed for a high-performance proprietary corporate environment, then adopted for UNIX Networking API
  • The TURN server works as a user space process, without imposing any special requirements on the system

To download the TURN Server software, the client messaging library and the test programs, click the tab "Downloads".

Contact information:

https://groups.google.com/forum/#!forum/turn-server-project-rfc5766-turn-server

email:misi@majd.eu mom040267@gmail.com

Feedback is very welcome (bugs, issues, suggestions, stories, questions).

Volunteers are welcome, too.