0. 前言
这段时间在搭建一个IPCamera项目服务器。视频点对点通话,客户端会查看设备端的音视频实时流。为了省流量,是通过P2P进行穿透。但是由于NAT设备的原因和IPV4的枯竭。有些设备是无法进行点对点传输实时流。所以需要进行服务器转发。这里为了快速实现原型,同时参考现在主流的流媒体协议。发现很多使用的是RTMP协议。
下图是总体设计图,为了整合多平台,会自建RTMP流媒体服务器和使用云厂商SaaS的RTMP流媒体服务。但是由于有时候会传输一些非流媒体数据,需要传输一些二进制文件,所以会需要自定义媒体转发服务。
以下是我实际项目中,用到的架构实现流程图。
1
2
3
4
5
6
7 |
1. 客户端A无法进行P2P穿透,请求业务服务器要进行转发。
2. 业务服务器根据客户端A,请求类型,返回对应的转发服务器地址和对应的房间号RoomID/Token等信息
3. 上述请求类型,可以是请求自建RTMP流媒体服务,购买于云厂商RTMP流媒体服务或者自定义协议媒体转发服务
4. 客户端A得到业务服务器返回的媒体服务器地址和RoomID/Token
5. 通过信令服务器或者MQTT服务器,把对应的媒体服务器地址和RoomID/Token告诉另一端客户端B
6. 客户端A和客户端B同时进入相同房间Room,客户端A进行推流,客户端B进行拉流
7. 其他媒体信息,如编解码格式,清晰度,播放,暂停,拍照等命令,通过上述信令或MQTT服务器进行命令控制 |
1. 编译Nginx
RTMP流媒体服务器,现成的开源方案有很多,有SRS,Red5,wowoza,FMS等,我这里使用的是Nginx的rtmp插件实现实时流转发。
下载 nginx-rtmp-module https://github.com/arut/nginx-rtmp-module
重新编译nginx
1 |
--prefix= /opt/nginx --with-stream --with-http_ssl_module --with-stream_ssl_module --with-debug --add-module=.. /nginx-rtmp-module |
2. 配置Nginx.conf
基本的nginx配置,这里就不进行介绍了,需要了解的可以参考我其他博客,里面有介绍。这里只介绍rtmp段的定义。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22 |
rtmp{
server{
listen 8081;
access_log logs /rtmp_access .log;
on_connect http: //127 .0.0.1:8080 /v1/rtmp/on_connect ;
application rtmp {
live on;
notify_method get;
on_play http: //127 .0.0.1:8080 /v1/rtmp/on_play ;
on_publish http: //127 .0.0.1:8080 /v1/rtmp/on_publish ;
on_done http: //127 .0.0.1:8080 /v1/rtmp/on_done ;
on_play_done http: //127 .0.0.1:8080 /v1/rtmp/on_play_done ;
on_publish_done http: //127 .0.0.1:8080 /v1/rtmp/on_publish_done ;
on_record_done http: //127 .0.0.1:8080 /v1/rtmp/on_record_done ;
on_update http: //127 .0.0.1:8080 /v1/rtmp/on_update ;
notify_update_timeout 10s;
}
application vod {
play /opt/openresty/video ;
}
}
} |
3. HTTP异步通知回调
Nginx-rtmp-module插件实现了针对RTMP协议的一些命令做了事件通知。这里我通过一个简单的SpringBoot项目,快速搭建一个HTTP服务来接收RTMP的回调。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54 |
package com.wunaozai.rtmp.notify.controller;
import javax.servlet.http.HttpServletRequest;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping(value= "/v1/rtmp/" )
public class RTMPNotifyController {
@GetMapping(value= "/on_connect" )
public String onConnect(HttpServletRequest request){
debug(request, "on_connect" );
return "on_connect" ;
}
@GetMapping(value= "/on_play" )
public String onPlay(HttpServletRequest request){
debug(request, "on_play" );
return "on_play" ;
}
@GetMapping(value= "/on_publish" )
public String onPublish(HttpServletRequest request){
debug(request, "on_publish" );
return "on_publish" ;
}
@GetMapping(value= "/on_done" )
public String onDone(HttpServletRequest request){
debug(request, "on_done" );
return "on_done" ;
}
@GetMapping(value= "/on_play_done" )
public String onPlayDone(HttpServletRequest request){
debug(request, "on_play_done" );
return "on_play_done" ;
}
@GetMapping(value= "/on_publish_done" )
public String onPublishDone(HttpServletRequest request){
debug(request, "on_publish_done" );
return "on_publish_done" ;
}
@GetMapping(value= "/on_record_done" )
public String onRecordDone(HttpServletRequest request){
debug(request, "on_record_done" );
return "on_record_done" ;
}
@GetMapping(value= "/on_update" )
public String onUpdate(HttpServletRequest request){
debug(request, "on_update" );
return "on_update" ;
}
private String debug(HttpServletRequest request, String action){
String str = action + ": " + request.getRequestURI() + " " + request.getQueryString();
System.out.println(str);
return str;
}
} |
4. 运行效果
(1) 启动nginx和SpringBoot
(2) 以下是SpringBoot打印信息(各位可以简单分析一下这些日志的)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37 |
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178269841&call=connect
on_publish: /v1/rtmp/on_publish app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=541&call=publish&name=room& type =live
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=541&call=update_publish& time =10×tamp=3999&name=room
on_done: /v1/rtmp/on_done app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=541&call= done &name=room
on_publish_done: /v1/rtmp/on_publish_done app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=541&call=publish_done&name=room
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178305623&call=connect
on_publish: /v1/rtmp/on_publish app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=publish&name=room& type =live
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =10×tamp=7296&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =20×tamp=17248&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =30×tamp=27328&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =40×tamp=37280&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =50×tamp=47296&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =60×tamp=57312&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =70×tamp=67264&name=room
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178380351&call=connect
on_play: /v1/rtmp/on_play app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=557&call=play&name=room&start=4294966296&duration=0&reset=0&pass=12345
on_play_done: /v1/rtmp/on_play_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=557&call=play_done&name=room&pass=12345
on_done: /v1/rtmp/on_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=557&call= done &name=room&pass=12345
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =80×tamp=77344&name=room
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178388202&call=connect
on_play: /v1/rtmp/on_play app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=563&call=play&name=room&start=4294966296&duration=0&reset=0&pass=12345
on_done: /v1/rtmp/on_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=563&call= done &name=room&pass=12345
on_play_done: /v1/rtmp/on_play_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=563&call=play_done&name=room&pass=12345
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =90×tamp=87360&name=room
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178396146&call=connect
on_play: /v1/rtmp/on_play app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=569&call=play&name=room&start=4294966296&duration=0&reset=0&pass=12345
on_done: /v1/rtmp/on_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=569&call= done &name=room&pass=12345
on_play_done: /v1/rtmp/on_play_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=569&call=play_done&name=room&pass=12345
on_connect: /v1/rtmp/on_connect app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&epoch=178403666&call=connect
on_play: /v1/rtmp/on_play app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=574&call=play&name=room&start=4294966296&duration=0&reset=0&pass=12345
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =100×tamp=97311&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=574&call=update_play& time =10×tamp=105504&name=room&pass=12345
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =110×tamp=107199&name=room
on_done: /v1/rtmp/on_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=574&call= done &name=room&pass=12345
on_play_done: /v1/rtmp/on_play_done app=rtmp&flashver=&swfurl=&tcurl=rtmp: //rtmp .wunaozai.com:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=574&call=play_done&name=room&pass=12345
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =120×tamp=117344&name=room
on_update: /v1/rtmp/on_update app=rtmp&flashver=FMLE /3 .0%20(compatible%3B%20FMSc /1 .0)&swfurl=&tcurl=rtmp: //120 .24.210.62:8081 /rtmp &pageurl=&addr=113.74.127.195&clientid=547&call=update_publish& time =130×tamp=122815&name=room |
(3) 客户端进行推流,这里的推流软件,我是使用这个 http://www.iavcast.com/html/ruanjian/iavcast.html
(4) 移动端,我使用微信小程序里的 腾讯视频云 这个小程序里面有RTMP测试
1
2
3
4
5
6
7
8
9
10
11
12
13
14 |
1 113.74.127.195 [05 /Aug/2018 :16:18:08 +0800] PUBLISH "rtmp" "room" "" - 2646572 687 "" "FMLE/3.0 (compatible; FMSc/1.0)" (1m 46s)
2 113.74.127.195 [05 /Aug/2018 :16:19:49 +0800] PLAY "rtmp" "room" "pass=12345" - 413 542 "" "" (4s)
3 113.74.127.195 [05 /Aug/2018 :16:19:57 +0800] PLAY "rtmp" "room" "pass=12345" - 413 542 "" "" (4s)
4 113.74.127.195 [05 /Aug/2018 :16:20:05 +0800] PLAY "rtmp" "room" "pass=12345" - 413 542 "" "" (4s)
5 113.74.127.195 [05 /Aug/2018 :16:20:13 +0800] PLAY "rtmp" "room" "pass=12345" - 413 542 "" "" (4s)
6 113.74.127.195 [05 /Aug/2018 :16:30:39 +0800] PLAY "rtmp" "room" "pass=12345" - 413 871 "" "" (4s)
7 113.74.127.195 [05 /Aug/2018 :16:30:54 +0800] PLAY "rtmp" "room" "pass=12345" - 413 647163 "" "" (12s)
8 113.74.127.195 [05 /Aug/2018 :16:31:08 +0800] PUBLISH "rtmp" "room" "" - 4961955 409 "" "FMLE/3.0 (compatible; FMSc/1.0)" (1m 30s)
9 113.74.127.195 [05 /Aug/2018 :23:06:47 +0800] PUBLISH "rtmp" "room" "" - 425763 529 "" "FMLE/3.0 (compatible; FMSc/1.0)" (13s)
10 113.74.127.195 [05 /Aug/2018 :23:08:29 +0800] PLAY "rtmp" "room" "pass=12345" - 413 871 "" "" (4s)
11 113.74.127.195 [05 /Aug/2018 :23:08:37 +0800] PLAY "rtmp" "room" "pass=12345" - 413 871 "" "" (4s)
12 113.74.127.195 [05 /Aug/2018 :23:08:45 +0800] PLAY "rtmp" "room" "pass=12345" - 413 871 "" "" (4s)
13 113.74.127.195 [05 /Aug/2018 :23:09:05 +0800] PLAY "rtmp" "room" "pass=12345" - 413 926026 "" "" (17s)
14 113.74.127.195 [05 /Aug/2018 :23:09:30 +0800] PUBLISH "rtmp" "room" "" - 7061016 409 "" "FMLE/3.0 (compatible; FMSc/1.0)" (2m 20s) |
5. RTMP鉴权方式
一般商用的话,为了防止被其他人使用和安全性考虑,所以需要对RTMP进行鉴权处理。鉴权如果有特殊性的,可以通过修改nginx-rtmp-module的源代码,然后进行修改,其实就是增加个auth函数,这个函数可以查询数据库之类的,然后决定返回0成功还是-1表示失败。
除了上面说到的方式,还可以通过简单的方式,就是上面提到的HTTP回调。如果HTTP回调返回的HTTP状态码是2xx的,表示成功。如果是返回5xx的状态码,那么表示失败。那样的话,服务器就是断开RTMP连接。
就是在 rtmp://rtmp.wunaozai.com/rtmp_live/room?username=username&password=password
至于实现,这里暂时还没有,其实就是在SpringBoot项目中对每个请求,判断一下参数即可。如果后面有机会就详细写一下,关联Redis数据库,实现房间号功能。但是可能不会写了,因为实际上不难。就是整个流程跑通还是比较多代码要写的,在博客里贴太多代码有点不好。博客最主要的还是提供思路。实际实现就应该在项目中实现了。
6. 其他
这里是一些配置说明和示例
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
|