Skip to content

Home

Jump to bottom
George Stoyanov edited this page Jul 11, 2017 · 10 revisions

How to build a simple Origin Server wiki!

This wiki describes the steps you need to take if you want to build an origin server based on Ubuntu and NGINX. The current version which I am using of Ubuntu is 16.04, but in theory it should work on all other modern Linux distributions. My test setup is two servers, one acting as a source pushing the HLS chunks and the manifest file over HTTP PUT and a receiving server, which is acting as an origin server, storing the chunks, manifest files and also providing them to the end-users, which can watch the content either using VLC or some other player able to play the manifest.m3u8 file.

NGINX Installation

First thing is to update your system and install NGINX. This could be done using the following command:

sudo apt update
sudo apt upgrade -y && sudo apt dist-upgrade -y && sudo apt autoremove -y
sudo apt install nginx -y

NGINX configuration

Confirm that the HTTP PUT method can be enabled on NGINX.

You should see in the version that NGINX is compiled with HTTP DAV support. You can use the following code to confirm that:

2>&1 nginx -V | grep with-http_dav_module

In case that this support is enabled it should show you all the options enabled in NGINX and color in red the http_dav_module. If the command doesn't return anything it means that the DAV support is not enabled. In this case you need to compile NGINX with DAV support, but this is not covered in this tutorial.

Create the target directory for the HLS chunks and manifest file upload and change its ownership:

sudo mkdir /var/www/html/vod
sudo mkdir /var/www/html/live
sudo chown -R www-data:www-data /var/www/html/vod /var/www/html/live

Set upload file limit of the files in the NGINX configuration larger than 1Mb

Open the nginx.conf file with your preferred text editor:

sudo vi /etc/nginx/nginx.conf

And add the following text: client_max_body_size 50m; within the http {} section, this will increase the limit of the upload files to 50Mb.

Enable the HTTP PUT and DELETEmethod in NGINX configuration and enable an access list of the device/s with access to it.

By default NGINX the PUT and DELETE methods are disabled, so you need to explicitly enable then. You can do it by adding the following section in the /etc/nginx/sites-available/default

sudo vi /etc/nginx/sites-available/default

Add the following section there:

location /vod {

                dav_methods  PUT;
                create_full_put_path on;
                limit_except GET HEAD {
                        allow 192.168.178.0/24;
                        deny  all;
                }
        }

So the above example is enabling the PUT method in the /var/www/html/vod folder only. The crete_full_put_path on; option is allowing creation of directories in case they do not exist. The limit section is limiting the HTTP PUT access only to IPs from network 192.168.178.0/24 and deny HTTP PUT access from all other IPs, the latter will have access only to GET and HEAD methods. If you want to limit to only one IP you can use /32. For example: 192.168.178.101/32. More information on the subject can be found here.

For the live section, I have added the DELETE method to the supported methods, this will allow the deletions of the chunks if this is requested by the remote server. This is very important in case of live streaming and prevent your disk to be overloaded with ts chunk files. The difference with the vod directory is that there the DELETE method is not allowed. The syntax is the following :

location /live {

                dav_methods  PUT DELETE;
                create_full_put_path on;
                limit_except GET HEAD {
                        allow 192.168.178.0/24;
                        deny  all;
                }
        }

Now you need to confirm that the NGINX configuration is consistent and restart the nginx service in order to apply the changes:

sudo nginx -t
sudo service nginx restart

I am also adding a script which will delete automatically all files with .ts extension if they are older than 1 minute in the /var/www/html/live/ directory. Open the crontab with your favorite text editor:

$ crontab -e

And add the following line:


* * * * * sudo find /var/www/html/live/ -name "*.ts" -mmin +1 -exec rm -f {} \;

FFMPEG installation and configuration

HTTP PUT support is enabled in FFMPEG version > 3.0. By default Ubuntu 16.04 comes with version 2.8.2 so you need to remove the current installation of FFMPEG if it is lower than 3.0. You can change the version number using the following command:

ffmpeg -version

If the version is lower then 3.0 you need to re-install ffmpeg. You can do it using the following commands:

sudo apt purge fffmpeg -y && sudo apt autoremove -y Add the PPA repository containing the latest version of FFMPEG. At the time of writing of this wili it is 3.3.2.1.

sudo add-apt-repository ppa:jonathonf/ffmpeg-3
sudo apt update && sudo apt install ffmpeg libav-tools x264 x265

HLS transcoding with FFMPEG

You can see a full list of all available commands related with the HLS transcoding with the following command:
ffmpeg -h muxer=hls

So my code is the following:

ffmpeg -i input.mp4 -c:v libx264 -x264opts "keyint=48:min-keyint=48:no-scenecut" -profile:v high -level 5.2 -s 1920x1080 -c:a aac -strict -2 -start_number 0 -hls_time 6 -hls_list_size 0 -hls_playlist_type vod -hls_base_url http://192.168.178.100/vod/ -f hls -method PUT http://192.168.178.100/vod/index.m3u8

Description of the command:

  • -i input.mp4 - defines the input file
  • -c:v libx264 - sets the video encoding to H.264
  • -x264opts "keyint=48:min-keyint=48:no-scenecut" - defines the GoP size to 48 frames, which in my case is 2 seconds (24fps input file). The GoP size should be evenly dividable by the hls_time, otherwise your chunks will be with different duration.
  • -profile:v high - defines the H.264 profile
  • -level 5.2 - defines the H.264 level
  • -s 1920x1080 - sets the resolution
  • -c:a aac - sets the Audio encoding
  • -strict -2 - is needed because of the experimental support of AAC
  • -start_number 0 - defines the chunks starting number to be 0
  • -hls_time 6 - sets the HLS chunk duration
  • -hls_list_size 0 - by default this is set to 5, which means that only the last 5 chunks will be defined in the manifest file. Since this is not live streaming, I want all chunks to be defined there, that is why it has to be set to 0
  • -hls_playlist_type vod - adds #EXT-X-PLAYLIST-TYPE:VOD line in the manifest file
  • -f hls - sets the format of the output file
  • -hls_base_url http://192.168.178.100/vod/ - by default ffmpeg is defining relative path to the chunks, which means that if you download the manifest file to your PC it would not work. This parameter is appending the defined path here at the beginning and transforming the relative to absolute URL path
  • -method PUT - defines the method. Please note that PUT is the default method. Here you can change the method to POST, but in this case you need to handle the HTTP POST requests and write them to the server using some custom script.
  • http://192.168.178.100/vod/index.m3u8 - sets the target directory. This will create index.m3u8 manifest file and index0.ts, index1.ts and so on TS chunk files.

P.S. Please note that the drawback of this method based on ffmpeg is that there is no way to create a true CBR (constant bitrate) output. In ffmpeg the output is always floating in rather wide margins, thus making it not very suitable for HLS transcoding in case of mobile devices connected to WiFi networks. The reason is that the mobile device will connect to the appropriate profile based on the available bitrate and the bitrate defined in the master manifest file. In this case the actual bitrate will vary and this could potentially cause additional buffering and deteriorate the overall user experience.

Clone this wiki locally