Trafficmeter

Welcome to the Trafficmeter.org. This is the site for the Trafficmeter source.

The latest stable version of the Trafficmeter is: 3.0.2 ( 3.0.2.tar.bz2 ) 2015-05-08

What is the Trafficmeter?

The Trafficmeter is a total network traffic registration system.

The software saves packets including payload and indexes it. You can get statistics or flow information of the selected IP or MAC address. Or you can retrieve traffic of the IP address in .pcap format. The word 'total' means that the filters by IP or MAC addresses is not necessary for storing.

In this case the software has 2 roles of using:

The Trafficmeter has GPL license.

System Requirements

System requirements depends on source traffic intensity. The disk system capasity depends on necessary payload storing interval. In example, for source traffic up to 1 Gbps and payload storing interval up to 48 hours, the below system had been tested:

The Trafficmeter can support Emulex DAG cards (former Endace DAG cards) with Ethernet interface.

Software Requirements

The Trafficmeter compiled and tested under Debian 7.8, Ubuntu 14.04, FreeBSD 10.1, Mac OS X 10.10.3.

Installation

Below is the sample of installation commands.

	$ ./configure
	
	$ make
	
	# make install
	
	# mkfs.ext4 -O ^has_journal /dev/sdb

	# echo deadline > /sys/block/sdb/queue/scheduler

	# vi /etc/rc.local
	    Add the previous command

	# mkdir /var/lib/trafficmeter

	# mount /dev/sdb /var/lib/trafficmeter

	# blkid /dev/sdb

	# vi /etc/fstab
	    Add the /var/lib/trafficmeter mounting point with this blkid

	# vi /etc/security/limits.conf
	    Add rows
		root	hard	nofile	16384
		root	soft	nofile	16384
		*	hard	nofile	16384
		*	soft	nofile	16384
	
	# cp src/tm.conf /etc/

	# vi /etc/tm.conf
	    Edit configuration
	
	# /usr/local/libexec/tmd [-p]
	

Using

The Trafficmeter has command line utilites to work with stored traffic information. All commands have option --help that describes syntax. This is the list of commands.

Tips

	$ tmflow 217.15.189.4 

	Interval	[ 2015-05-01 22:05:00+0600; 2015-05-01 22:10:00+0600 )
	Traffic of	217.15.189.4 

	            time proto   host                     peer                                             i.traffic      o.traffic
	2015-05-01T22:05     6   217.15.189.4 :1433       1.93.37.147                                             40              0
	2015-05-01T22:05     6   217.15.189.4             e673.e9.akamaiedge.net :443                          44292           8741
	2015-05-01T22:05     6   217.15.189.4             us-courier.push-apple.com.akadns.net :443               52            142
	2015-05-01T22:05     6   217.15.189.4             p7-buy.itunes-apple.com.akadns.net :443              15722           8717
	2015-05-01T22:05     6   217.15.189.4             sp.itunes-apple.com.akadns.net :443                  16287           3125
	2015-05-01T22:05     6   217.15.189.4             mzuserxp.itunes-apple.com.akadns.net :443             5621           1899
	2015-05-01T22:05     6   217.15.189.4             service.gc.apple.com.akadns.net :443                 26678          16045
	2015-05-01T22:05    17   217.15.189.4             17.173.254.222                                         220            220
	2015-05-01T22:05    17   217.15.189.4             17.173.254.223                                         396            132
	2015-05-01T22:05     6   217.15.189.4             adc-adserver-autoscaling-1425990358.us-east-1.elb.amazonaws.com :443            112            220
	2015-05-01T22:05     6   217.15.189.4             adc-adserver-autoscaling-1425990358.us-east-1.elb.amazonaws.com :443           8263           1863
	2015-05-01T22:05     6   217.15.189.4             sdk-session-event-api-v3-887129086.us-east-1.elb.amazonaws.com :443           5718            684
	2015-05-01T22:05     6   217.15.189.4             e6845.ce.akamaiedge.net :80                           2100            551
	2015-05-01T22:05     6   217.15.189.4             e8218.ce.akamaiedge.net :80                           2328            633
	2015-05-01T22:05     6   217.15.189.4             e5871.e9.akamaiedge.net :443                         12266           2411
	2015-05-01T22:05     6   217.15.189.4             e16.whatsapp.net :443                                  547            931
	2015-05-01T22:05     6   217.15.189.4             lb.us-east-1.applifier.info :443                       120            168
	2015-05-01T22:05     6   217.15.189.4             data.flurry.com :443                                     0            128
	2015-05-01T22:05     6   217.15.189.4             data.flurry.com :443                                    96            474
	2015-05-01T22:05     6   217.15.189.4             cloud.rovio.com :443                                198580          63634
	2015-05-01T22:05     6   217.15.189.4             dub408-m.hotmail.com :443                             5137           2891
	2015-05-01T22:05     6   217.15.189.4             a1961.g1.akamai.net :80                                120            104
	2015-05-01T22:05     6   217.15.189.4             a1856.g2.akamai.net :80                               8120            769
	2015-05-01T22:05     6   217.15.189.4             a1859.g2.akamai.net :80                             462873           9412
	2015-05-01T22:05     6   217.15.189.4             s.mopub.com :80                                         60            104
	2015-05-01T22:05     6   217.15.189.4             s.mopub.com :80                                        608            728
	2015-05-01T22:05     6   217.15.189.4             sdds4.intermaps.com :80                                  0             52
	2015-05-01T22:05     6   217.15.189.4             data.flurry.com :443                                     0            128
	Total:                                                                                                816356         124906
	28 rows (0.001 sec)
	
	$ tmpacket -s 2015-04-30T12:50 -e 2015-04-30T12:55 217.15.189.5 | tcpdump -r -

	reading from file -, link-type EN10MB (Ethernet)
	12:52:35.166836 IP 27.3.9.214.2701 > 217.15.189.5.microsoft-ds: Flags [S], seq 3779347191, win 65535, options [mss 1460,nop,nop,sackOK], length 0
	12:52:38.080549 IP 27.3.9.214.2701 > 217.15.189.5.microsoft-ds: Flags [S], seq 3779347191, win 65535, options [mss 1460,nop,nop,sackOK], length 0
	12:53:32.287364 IP n219077012099.netvigator.com.3532 > 217.15.189.5.microsoft-ds: Flags [S], seq 946892382, win 65535, options [mss 1440,nop,nop,sackOK], length 0
	12:53:35.084078 IP n219077012099.netvigator.com.3532 > 217.15.189.5.microsoft-ds: Flags [S], seq 946892382, win 65535, options [mss 1440,nop,nop,sackOK], length 0
	
	$ tmstat -s 2015-04-01 -e 2015-05-01 -D -m -p 217.15.189.0/29

	Interval	[ 2015-04-01 00:00:00+0600; 2015-05-01 00:00:00+0600 )
	Traffic of	217.15.189.0/29 

	            time        i.flows   i.packets      i.traffic     o.flows   o.packets      o.traffic
	2015-04-01T00:00        1508699   117320429      131082.61     1507244    88952170       17549.16
	2015-04-02T00:00        1480525   103244014      114788.58     1502253    79701812       17534.80
	2015-04-03T00:00        1280333    88980848      102069.08     1233865    67420197       13303.10
	2015-04-04T00:00        1417777    62079396       68500.30     1397334    49535327       12552.89
	2015-04-05T00:00        2372223    65634154       68065.40     2384921    53364888       14577.27
	2015-04-06T00:00         954192    96772001      107832.94      949103    71866711       12442.53
	2015-04-07T00:00        2263176   124738757      130755.26     1886263    92946408       17884.57
	2015-04-08T00:00        1504850   132381576      152743.61     1401771    93238370       13455.05
	2015-04-09T00:00        1847276   237424479      275606.15     1697640   155927976       23261.90
	2015-04-10T00:00        1615202   201829381      240181.15     1637498   139737981       20616.55
	2015-04-11T00:00        1123254   201277923      244141.19     1133761   133158992       19482.67
	2015-04-12T00:00        1555219   116174292      131433.73     1541886    88182449       19316.70
	2015-04-13T00:00        2228400   103368356      114389.22     1835684    77091014       15378.01
	2015-04-14T00:00        1778029   101806619      115545.80     1745776    75480915       13490.81
	2015-04-15T00:00        2210591   140855719      161535.29     2225661   103642169       20155.43
	2015-04-16T00:00        2150195   243577243      276541.63     2239556   165725745       33097.96
	2015-04-17T00:00        1967713   199479403      219951.17     1927912   148107100       41033.63
	2015-04-18T00:00        1557582   132561084      145437.82     1463553   105476009       30665.12
	2015-04-19T00:00        1371129   119370196      133901.11     1281414    89967812       22937.86
	2015-04-20T00:00        1485195   114471975      124418.79     1490044    89181104       19992.31
	2015-04-21T00:00        2189268   143058409      151452.68     2142545   111222340       31404.74
	2015-04-22T00:00        2137401   127653288      139492.89     2074789    99804733       22634.92
	2015-04-23T00:00        2118557   123067309      122915.95     2054725   103113286       33119.27
	2015-04-24T00:00        1637263   136752195      157296.93     1584019   101591908       20462.74
	2015-04-25T00:00        1240945   107414843      126110.90     1125387    73470818       13987.20
	2015-04-26T00:00        1071206    73768453       83857.77      950509    55383450       12298.61
	2015-04-27T00:00        1080577    63649286       71244.59      988629    48204984       10308.50
	2015-04-28T00:00        1059557    79311998       89258.57     1017311    57625239       11751.60
	2015-04-29T00:00        1915953   114429521      123131.30     1877061    84084296       17249.79
	2015-04-30T00:00        1136083   110447608      129883.54     1092056    80412362       12745.00
	Total:                 49258370  3782900755     4253565.95    47390170  2783618565      584690.70
	30 rows (0.051 sec)
	
	$ tmflow unknown

	Interval	[ 2015-05-01 23:20:00+0600; 2015-05-01 23:25:00+0600 )
	Traffic NOT of	31.148.248/22 31.223.192/20 81.17.160/20 91.135.192/20 91.216.220/24 93.170.26/24 109.238.160/20 192.168.20/22 192.168.57/24 194.0.108/22 195.226.208/24 217.11.64/20 217.15.176/20 

	            time proto   host                     peer                                             i.traffic      o.traffic
	2015-05-01T23:20    17   169.254.248.255 :137     169.254.255.255 :137                                     0           2574
	2015-05-01T23:20    17   169.254.255.255 :137     169.254.248.255 :137                                  2574              0
	2015-05-01T23:20     6   192.168.99.156           ksn-file-geo.kaspersky-labs.com :443                     0            152
	2015-05-01T23:20     6   192.168.99.156           ksn-url-geo.kaspersky-labs.com :443                      0            152
	2015-05-01T23:20     6   195.122.177.135 :443     192.168.99.156                                         152              0
	2015-05-01T23:20     6   195.122.177.165 :443     192.168.99.156                                         152              0
	Total:                                                                                                  2878           2878
	6 rows (0.100 sec)
	
	$ tmflow -S b5 known

	Interval	[ 2015-05-02 10:35:00+0600; 2015-05-02 10:40:00+0600 )
	Traffic of	31.148.248/22 31.223.192/20 81.17.160/20 91.135.192/20 91.216.220/24 93.170.26/24 109.238.160/20 192.168.20/22 194.0.108/22 195.226.208/24 217.11.64/20 217.15.176/20 

	            time proto   host                     peer                                            i.flows   i.packets      i.traffic     o.flows   o.packets      o.traffic
	2015-05-02T10:35     6   81.17.164.146            cs1-41v4.vk-cdn.net :80                              74      152114      220145680          73       18719        1003644
	2015-05-02T10:35     6   81.17.170.138            a128.li5g5.akafms.net                                24      203477      188896832          24       89092        3666084
	2015-05-02T10:35     6   109.238.161.244          counterstrike.org.ua :80                              1      115497      166314777           1        4606         241240
	2015-05-02T10:35     6   109.238.162.166          r2.sn-pivhx-n8ve.googlevideo.com :443                 8      105705      158009198           8       57009        3780862
	2015-05-02T10:35     6   81.17.174.138            a1507.d.akamai.net :80                                1       91238      132292281           1       14078         732673
	Other:                                                                                             234125     5687096     3599538030      210176     4032646     1227003318
	Total:                                                                                             234233     6355127     4465196798      210283     4216150     1236427821
	5 rows (0.241 sec)
	
	$ tmflow -S p5 -p known

	Interval	[ 2015-05-02 10:30:00+0600; 2015-05-02 10:35:00+0600 )
	Traffic of	31.148.248/22 31.223.192/20 81.17.160/20 91.135.192/20 91.216.220/24 93.170.26/24 109.238.160/20 192.168.20/22 194.0.108/22 195.226.208/24 217.11.64/20 217.15.176/20 

	            time proto   host                     peer                                            i.flows   i.packets      i.traffic     o.flows   o.packets      o.traffic
	2015-05-02T10:30    17   109.238.160.164          sip.dtx.kz                                         5674     1530896       92833026         489      863766       51825976
	2015-05-02T10:30     6   109.238.161.246          m2.bigcinema.tv :80                                   1      215383      323053681           1       10281         540613
	2015-05-02T10:30     6   81.17.164.146            cs1-41v4.vk-cdn.net :80                              82      175974      254691554          82       22169        1186684
	2015-05-02T10:30     6   91.216.220.4 :80         185.57.72.175                                         0           0              0           3      151619        7897760
	2015-05-02T10:30     6   109.238.161.244          download-cs.net :80                                   4      149909      224789532           4       72065        5348008
	Other:                                                                                             246595     4241678     3430208649      234434     3416088     1460307333
	Total:                                                                                             252356     6313840     4325576442      235013     4535988     1527106374
	5 rows (0.237 sec)
	
	$ tmflow -S f5 -p known

	Interval	[ 2015-05-02 10:30:00+0600; 2015-05-02 10:35:00+0600 )
	Traffic of	31.148.248/22 31.223.192/20 81.17.160/20 91.135.192/20 91.216.220/24 93.170.26/24 109.238.160/20 192.168.20/22 194.0.108/22 195.226.208/24 217.11.64/20 217.15.176/20 

	            time proto   host                     peer                                            i.flows   i.packets      i.traffic     o.flows   o.packets      o.traffic
	2015-05-02T10:30    17   109.238.160.164          sip.dtx.kz                                         5674     1530896       92833026         489      863766       51825976
	2015-05-02T10:30    17   109.238.161.242          6771.com :53                                       1432        2127         222561        1432        2126         138516
	2015-05-02T10:30     6   81.17.163.42             imgcdn.ptvcdn.net :80                                 0           0              0         868        3420         205200
	2015-05-02T10:30    17   81.17.170.138            195.189.31.14 :53                                   821         837          76895         854         870          79703
	2015-05-02T10:30     6   217.15.181.46 :80        89.218.64.66                                        725       10075         921992         725       12520       14335740
	Other:                                                                                             243704     4769905     4231521968      230645     3653286     1460521239
	Total:                                                                                             252356     6313840     4325576442      235013     4535988     1527106374
	5 rows (0.234 sec)
	

Comments

For comments about the web pages & file arhive, please send mail to e-mail.