FAQ :: Common questions/answers related to troubleshooting hardware/software issues

(I will modify and add to this list as needed and as my time permits!)
Q. I opened index.php and see a blank page. What should I do?
A. First, turn on error reporting in "inc/index_header.php" by editing the following line (at the very top of the file) like so:

PHP Code:
ini_set('display_errors',1); 
This can also be accomplished with:

PHP Code:
error_reporting(E_ALL); 
Both commands will enable the output of PHP errors to the browser.

Also, check your Apache error log for PHP errors and warnings.

If you see a PHP error or warning that you don't understand, please copy and paste the error into Google and behold the plethora of other people who have experienced -- and overcome -- similar errors. Leverage Google to your advantage in this case, and benefit from the experiences of others who have had the same problem.

Q. When error reporting is enabled or when I check the Apache error log, I see the following PHP warning: "PHP Warning: Division by zero...". What does this mean?
A. This warning is of no consequence. It is a side-effect of the download progress bar. It is harmless. If you disable error reporting in your script, which you should always do on a production server to protect you from hackers (and it is disabled by default), then you won't have to endure the awful pain of seeing this warning.

Q. When error reporting is enabled or when I check the Apache error log, I see the following PHP notice "Notice: ob_flush() : failed to flush buffer. No buffer to flush...". What does this mean?
A. Again, this notice is of no consequence. Please see the previous FAQ question/answer.

Q. I opened index.php and see something like: "Parse error: syntax error, unexpected T_FUNCTION in /home/user/public_html/index.php on line...". What should I do?
A. This is generally an indication that you do not have at least PHP 5.3 installed. Anonymous functions usually cause this error, and they were not introduced until PHP 5.3. You need to install PHP 5.3+ to eliminate this error.

Q. I opened index.php and see the following message: "This domain is not authorized to access this software!". What should I do?
A. Open Config.class.php and edit the $_authorizedDomains array to include all domain names (and subdomains!) that are allowed to access/display the software. ("www" IS a subdomain! If you want "www.yoursite.com" as well as "yoursite.com" to work, then you must include both!) For example:

PHP Code:
public static $_authorizedDomains = array(            
   
// List of domain (and subdomain!) names authorized to run this software. Do NOT prepend domains with 'http://' or 'https://'!!            
   // "www" IS a subdomain! If you want "www.yoursite.com" as well as "yoursite.com" to work, then you must include both below!            
   // Leave this array empty to allow all domain names to access the software. (NOT RECOMMENDED!!)            
   
'localhost',
   
'yourconversionsite.com',
   
'www.yourconversionsite.com',
   
'anothersite.com',
   
'www.anothersite.com',
   
'208.67.222.222'
); 
Leaving this array empty will allow ALL domains to access your software. (NOT RECOMMENDED!!)

Q. When I enter a URL and click the "Convert" button, the page reloads and I see a bunch of strange characters and symbols filling up the screen. Why is this?
A. Please ensure that the 'store' directory is chmod to 0777 permissions. The script is trying to send the video download to the 'store/videos' directory, but adequate permissions are not set. So the raw, downloaded data has nowhere to go and is ultimately output to the screen.

Q. When I enter a URL and click the "Convert" button, the page reloads, shows me a "Download" progress bar, and nothing happens and/or I see the message "Error downloading remote file!" (or "Error downloading video!"). Why is this?
A. First, ensure that you have installed recent versions of both cURL and the PHP cURL extension.

Verify that the _TEMPVIDDIR constant in Config.class.php is the correct path to your 'store/videos' directory.

If you have moved the 'store/videos' directory to outside of the web root, then you will have to use an absolute path instead of a relative path.

Verify that software.xml exists in the 'store' directory. Also, ensure that 'store' directory permissions are set to chmod 0777. If software.xml exists, try deleting the file and allowing it to automatically regenerate.

In the commercial software, you could also try setting the 'extractors' directory permissions to chmod 0777. In some cases, this must be set in order for the YouTube decryption code auto-update mechanism to work -- and, ultimately, for the download to succeed.

Check the contents of your 'store/videos' directory. Are there video files in the directory? If so, are the video files playable? Are the videos fully downloaded? Are the videos 0 kb in length?

Try executing a simple PHP cURL request to the video hosting site's video page, and return the contents of the page to the screen. For example (for YouTube videos), place the following code in a new file, replacing the "vidID" URL query string parameter value with a valid YouTube video ID:

PHP Code:
<?php

// create a new cURL resource
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL"http://www.youtube.com/watch?v=vidID");
curl_setopt($chCURLOPT_HEADER0);
curl_setopt($chCURLOPT_FOLLOWLOCATION1);

// grab URL and pass it to the browser
curl_exec($ch);

// print out any cURL errors as well as HTTP response information
echo 'Curl error: (' curl_errno($ch) . ') ' curl_error($ch) . '<br>';
print_r(curl_getinfo($ch));

// close cURL resource, and free up system resources
curl_close($ch);

?>
Save the file, upload it to your server, and open it in a browser. Do you see the HTML page located at that URL? If the video is blocked in your server's country, then this will be indicated in the screen's output. A blank page or CAPTCHA screen could indicate that your IP has either been temporarily banned or flagged (via the CAPTCHA) for exceeding video pull quota limits. It could also mean that your server is having networking issues.

Ensure that the PHP "safe_mode" directive is set to "Off" and the PHP "open_basedir" directive is set to no value or "". Both directives can be modified in your server's php.ini file, PHP's main configuration file on your server.

If you are installing the free software using XAMPP/WAMP, you can try adding the following code to the cURL request in the SaveVideo() method of YouTubeToMp3Converter.class.php:

PHP Code:
curl_setopt($chCURLOPT_SSL_VERIFYPEERfalse); 
Finally, check your Apache error log for PHP errors and warnings.

Q. Videos are downloaded and converted, but I never see the download progress bar (only the conversion progress bar appears). How come?
A. This could be an indication of a networking problem on your server (relating to the configuration of network card software on your server). Contact your hosting provider to see if there is anything that they can do on their end.

This might be a cURL, PHP cURL extension, or PHP output buffering issue. Verify that cURL, PHP, and the PHP cURL extension are installed and configured correctly. Check the output buffering directives in your server's php.ini.

Your server might have a very fast download speed. If the speed is fast enough, consistently, it is possible that the download progress bar is absent because it never has a chance to clear the output buffer before the download completes.

If you are running cPanel, Plesk, Virtualmin, or a similar hosting control panel software, check to see if any of the settings in your control panel are conflicting with the download progress bar. In particular, via your control panel or otherwise, try configuring PHP to run as an Apache module instead of via a CGI wrapper (or vice versa?). Often, adjusting this setting will cause the download progress bar to start working.

If you are using Plesk, try disabling Nginx as an Apache/HTTP accelerator and/or reverse proxy server. Plesk appears to install Nginx as a dependency package and has configured it (with Apache and PHP) in such a way that it may interfere with the progress bar.

If you're using Cloudflare and have enabled its CDN/HTTP proxy, then this can disrupt the download progress bar. In this case, you must create a new Page Rule for, e.g., "www.yourdomain.com/*" to "Disable Security" for this domain. While this might seem counterintuitive, this rule actually does not reduce the default security afforded by Cloudflare. Instead, it disables little-used features that only marginally enhance security in some situations and/or are unavailable in the free plan.

Sometimes gzip HTTP compression can conflict with progress bar performance on Apache. If you suspect this is true in your case (particularly if you are running PHP via PHP-FPM), then you can first try disabling gzip for index.php only by pasting the following code in your .htaccess file:

Code:
<IfModule mod_setenvif.c>
	SetEnvIfNoCase Request_URI (index\.php)$ no-gzip dont-vary
</IfModule>
If that doesn't work (for Apache), then try adding the following HTTP response header to the very top of "inc/index_header.php":

PHP Code:
header('Content-Encoding: none'); 
Nginx, when used as a replacement for Apache, performs its own layer of buffering that will prevent proper progress bar display. To disable this behavior, first add the following directive to the corresponding vhost file (in "/etc/nginx/sites-available"):

Code:
fastcgi_keep_conn on;
Then, for Nginx, you MUST put 2 additional HTTP response headers at the very top of "inc/index_header.php", to disable web server buffering and gzip compression, respectively, at runtime:

PHP Code:
header('X-Accel-Buffering: no');
header('Content-Encoding: identity'); 
If you are using PHP 5.5+, AND you have an older version of my software, then you MUST add an additional "cURL resource" argument to the beginning of the parameter list of the UpdateVideoDownloadProgress() function in VideoConverter.class.php (YouTubeToMp3Converter.class.php in the free version), like so:

Code:
function UpdateVideoDownloadProgress($curlResource, $downloadSize, $downloaded, $uploadSize, $uploaded)
Also, since progress bar performance is dependent on PHP version, ensure that the _PHP_VERSION constant in Config.class.php is set to the correct PHP version number.

Q. The video download completes, the "Conversion" progress bar is shown, and then nothing happens and/or I see the message "Error generating converted file!". What can I do?
A. First, check that the _FFMPEG constant value in Config.class.php points to the correct location of the FFmpeg binary on your server. On Linux, you can generally locate FFmpeg via the "which ffmpeg" command in the console. (However, IF you installed FFmpeg using the commercial software's auto-install feature, then FFmpeg will be located at, e.g., "/home/user/public_html/store/bin/ffmpeg".)

In the free version, ensure that both the 'mp3' and 'logs' directories are chmod to 0777 permissions.

Check to see if videos are actually being downloaded to 'store/videos' (or 'videos' in free version) prior to conversion.

Check your 'store/logs' directory (or 'logs' directory in free version) for FFmpeg log files. Are there any log files in the directory? If so, look for errors in a log file that corresponds to a failed conversion.

Does FFmpeg work from the command console? Try running a FFmpeg command directly in the command line interface. Preferably, run a command that is executed by my software during conversions.

Are you running the most recent version of FFmpeg? (Check your version of FFmpeg by typing "ffmpeg" in the command console. If the dates shown in the resulting output are more than 1-2 years old, then you may have an outdated version of FFmpeg.) Also, the commercial version of my software may no longer work as expected with FFmpeg builds downloaded from the apt-get (or yum, etc.) repositories. Instead, try downloading a recent, static build from here: FFmpeg Static Builds. This tutorial can assist you further with this process.

You may need to set permissions of the FFmpeg binary to 0755, so that my software has permission to access and run FFmpeg. This especially holds true for people using a static build of FFmpeg.

Have you installed at least the libvorbis, libvpx, libmp3lame, libx264, and aac codecs on your server? You can check quickly by executing commands like the following in the console: "ffmpeg -codecs | grep x264" or "ffmpeg -codecs | grep mp3" or "ffmpeg -codecs | grep aac". The word or phrase after "grep" tells FFmpeg how to filter the installed codecs (you can change this search word or phrase as needed). When you type any of these commands, check for a capital "E" in front of the codec name ("E" signifies that you can encode with this codec). If the "E" is missing, then there is something wrong with the codec installation.

If you have moved the 'store/output' or 'store/logs' directories (or 'mp3' and 'logs' directories in free version) to outside of the web root, then you will have to use an absolute path instead of a relative path for the corresponding constant values in Config.class.php.

In the commercial version, try using an absolute path for the 'store/logs' directory in the FFmpeg commands located in the $_convertedFileTypes array in Config.class.php, i.e.:

PHP Code:
'ffmpeg' => '%ffmpeg% -i %tempFile% -vol %volume% -y -acodec libmp3lame -ab %quality%k %newFile% 2> /home/user/public_html/store/logs/%id%.txt'
In the free version, the FFmpeg command is located in GenerateMP3() in YouTubeToMp3Converter.class.php, and the same principle applies (with regard to using an absolute path for the 'logs' directory).

An improperly configured DNS can cause HTTP requests (made from your server to exec_ffmpeg.php) to fail. To test this possibility, save the following code to a file, and then load the file in a web browser:

PHP Code:
<?php

// create a new cURL resource
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL"http://www.your-domain-here.com");
curl_setopt($chCURLOPT_HEADER0);
curl_setopt($chCURLOPT_FOLLOWLOCATION1);

// grab URL and pass it to the browser
curl_exec($ch);

// print out any cURL errors as well as HTTP response information
echo 'Curl error: (' curl_errno($ch) . ') ' curl_error($ch) . '<br>';
print_r(curl_getinfo($ch));

// close cURL resource, and free up system resources
curl_close($ch);

?>
Now repeat the process, substituting "http://www.your-domain-here.com" with "http://www.google.com" (or any other domain name other than your site's domain name!) this time.

If the above code consistently fails for any reason, in any way, when CURLOPT_URL is set to "http://www.your-domain-here.com", but consistently works when CURLOPT_URL is set to "http://www.google.com" (or any other domain name other than your site's domain name!), then you likely have an issue with your DNS. Evidently, HTTP requests made from your server are able to correctly resolve the DNS of all sites except your own -- indicating an issue with your own DNS configuration. Possible fixes include using another DNS provider, changing nameservers, and/or having a professional configure your DNS for you.

Q. On the "Edit file further" screen (following successful download/conversion), the loading box never disappears. It just loads forever and ever. How can I fix this?
A. You probably have gzip compression enabled for Apache. Media file formats like mp3, mp4, etc., are already compressed. So if you try to compress these files further with gzip, the audio/video player on the "Edit file further" screen will not be able to read the media correctly. To address this problem (again, assuming you have gzip enabled on your server), you will need to add the following line to the software's .htaccess file:

Code:
SetEnvIfNoCase Request_URI (\.mp3|\.mp4|\.webm|\.f4v|\.3gp)$ no-gzip

Q. I am unable to download my edited files (following converted file cropping or MP3 meta data editing), or MP3 meta data editing is not working. Why?
A. First, verify that an edited file was actually created by checking the contents of the "store/output_edited" directory. If no corresponding file exists in the "store/output_edited" folder, then there is either 1) a problem with permissions/ownership of the 'store' folder (permissions should be set to chmod 0777), 2) there is a problem/error with the FFmpeg command in ffmpeg_edit.php, and/or 3) there is a problem with the compilation/configuration of FFmpeg itself.

After ruling out permissions/ownership of the 'store' folder as an issue, verify that you have a recent build of FFmpeg installed. (Check your version of FFmpeg by typing "ffmpeg" in the command console. If the dates shown in the resulting output are more than 1-2 years old, then you may have an outdated version of FFmpeg.)

The next step is to troubleshoot the FFmpeg command in ffmpeg_edit.php. Add the highlighted line of code to ffmpeg_edit.php, given the following context code:

PHP Code:
$ffmpegCmd .= Config::_EDITED_CONVERTED_FILEDIR .  $song.".".$extension;
echo 
'<script type="text/javascript">console.log("FFmpeg command:  '.$ffmpegCmd.'");</script>';   // Add new line of code here !!!
exec($ffmpegCmd); 
Then run the software, open up the JavaScript console in your browser, and check the console output. You should see a FFmpeg command printed there. Copy and paste this command into a command prompt on the server. Before you run the command in the terminal, you will need to verify that the file in the command exists in the 'store/output' folder, and you may need to use absolute paths for ALL file paths in the command (i.e., '/home/user/public_html/store/output/someconvertedfile_uuid-sadfasdfasdf.mp3').

After executing the command in the server command prompt, check for the new file in the 'store/output_edited' folder. Also take note of any FFmpeg runtime errors you see in the terminal. If you see errors, then troubleshoot your FFmpeg command and/or installation accordingly.

Q. How long are converted files saved on the server?
A. In the commercial version, the length of time files remain on the server can be set in Config.class.php (via the _MAX_AGE_FILES constant value). And a separate PHP file (schedule.php), intended to be run as a cron job (scheduled task) on your server, deletes old converted files per the corresponding setting in the Config class. I recommend setting up the cron job to run at least once a day (and more often if caching is enabled), i.e.:

Code:
0 0 * * * /usr/bin/php -q /absolute/path/to/schedule.php
Also, ensure that schedule.php has at least 0755 permissions, otherwise the cron daemon will not be able to execute it.

Note: Please open up schedule.php in a script editor prior to creating the cron job. Per the comments at the top of this file, you MUST uncomment the PHP unlink() and rmdir() functions for converted files to actually be deleted.

When caching is enabled, file deletion code corresponding to the "store/output" folder ignores _MAX_AGE_FILES and uses the _MAX_CACHE_SIZE constant value instead. Regardless of caching configuration, files in the "store/output_edited", "store/videos", and "store/logs" directories are always deleted per _MAX_AGE_FILES.

The free version does not allow for the periodic removal of converted files.

Q. How can I integrate the software into my existing website? Can I run the software inside an HTML <iframe>?
A. The easiest way to integrate the software into an existing site is to load the software's index.php inside an iframe (on your site). The iframe size will need to be large enough to accommodate the software content. You can make the iframe background transparent via HTML/CSS so that it blends in with the rest of your site. Any additional styling of the iframe or iframe contents can be achieved by directly editing the HTML/CSS of the software files. For the commercial version of the software, you may also want to remove the navigation and multi-language components from the top of the default design/layout. Finally, be sure to include the following line at the very top of the software's "inc/index_header.php":

PHP Code:
header('X-Frame-Options: SAMEORIGIN'); 
This header ensures that your iframe cannot be embedded on any other domain but your own.

Also, when the software is run inside an iframe, you will need to insert the following code at the very top of the parent window's/file's source code to enable custom and dynamic page meta data:

PHP Code:
<?php include 'VideoConverter/inc/page_meta_data.php'?>
...where 'VideoConverter' is the directory in which the software is installed (relative to the web root directory). Alternatively, if the software is installed directly in the web root, then the code should read:

PHP Code:
<?php include 'inc/page_meta_data.php'?>
You will then need to edit the $appRootDirectory value at the very top of 'inc/page_meta_data.php', so that its value similarly reflects the application root directory (i.e., 'VideoConverter/' or '' -- an empty value -- to indicate the web root directory).

Q. I think YouTube is blocking or banning my IP address, and/or I am experiencing a CAPTCHA page when trying to connect to a YouTube video page. What can I do?
A. First of all, bans are usually temporary. In my experience, YouTube only blocks/bans your IP for a finite period of time. The amount of time will vary -- generally, anywhere from a few minutes to a few days.

If you are frequently getting blocked by YouTube, OR you are experiencing a CAPTCHA page, then you can rotate between multiple outgoing IPs for HTTP requests made to YouTube's servers. My commercial software facilitates this process for you.

First, you need to purchase some additional IP (IPv4) addresses from your hosting provider. (Often your hosting provider will include additional IPs for free with your hosting plan!) Extra IP addresses generally only cost a couple of dollars apiece, and it's reasonable to "start" with 4-5 IPs for the purpose of IP rotation. You can always buy more IPs later and add them to the rotation as needed. That said, the more IPs you have, the better off you will be. In general, the number of additional IPs required for your site is directly proportional to the amount of traffic to your site.

(Note: Do NOT use "proxy" IPs purchased from a dedicated, 3rd-party proxy provider! The intent of IP rotation is not to hide or obscure your server's identity, but merely to change the outgoing IP address seen by YouTube. Also, dedicated proxy IPs are much slower than IPs used as outgoing network interfaces, so you will see a marked decrease in speed and performance if you use proxies.)

Next, you need to configure these additional IP addresses so that they can be used as "outgoing network interfaces". Simply put, your server needs to be configured so that it can "bind" to these extra IPs for HTTP requests. The easiest way to accomplish this is to have your hosting provider do this for you. If you must do this yourself, then you can refer to OVH's guide for setting up "Network IP Aliasing" for a number of different Linux distributions (and Windows): http://docs.ovh.ca/en/guides-network-ipaliasing.html.

Once your additional IPs are set up, then you can configure the corresponding constant values in Config.class.php in preparation for IP rotation. This file is well-documented, and all constants related to IP rotation and the database are throroughly explained in the associated comment sections.

Finally, you must use the SQL provided in 'docs/ips.sql' to create the required database table that will hold your IPs. Ensure that you replace the IPs in the SQL INSERT command with your own IP addresses "before" executing the SQL! Additional IP addresses can be added to the INSERT command by copying the format and adding lines to the command. In the future, whenever you want to add another IP address to the rotation, then you simply add the new IP or IPs to this database table.

If you find that, with IP rotation enabled, you are still experiencing IP bans and/or CAPTCHAs, then that is generally a sign that you need to add more IP addresses to the rotation (to the corresponding database table). You must keep adding IPs until YouTube downloads/conversions start working normally.

If you suspect that there is a problem with the IPs themselves, or the IP configuration on your server, then you should first test each additional IP using the following, simple PHP cURL code:

PHP Code:
<?php
// create a new cURL resource
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL"http://www.youtube.com/watch?v=vidID");
curl_setopt($chCURLOPT_HEADER0);
curl_setopt($chCURLOPT_FOLLOWLOCATION1);
curl_setopt($chCURLOPT_REFERER'');
curl_setopt($chCURLOPT_INTERFACE'23.232.234.2');  // Your extra IP goes here!
curl_setopt($chCURLOPT_SSL_VERIFYHOSTfalse);
curl_setopt($chCURLOPT_IPRESOLVECURL_IPRESOLVE_V4);
curl_setopt($chCURLOPT_SSL_VERIFYPEERfalse);

// grab URL and pass it to the browser
curl_exec($ch);

// print out any cURL errors as well as HTTP response information
echo 'Curl error: (' curl_errno($ch) . ') ' curl_error($ch) . '<br>';
print_r(curl_getinfo($ch));

// close cURL resource, and free up system resources
curl_close($ch);
?>
Ensure that you change the CURLOPT_INTERFACE value to your IP address. Then save the code to a .php file and run it on your server. If you see a video but it won't play, you don't see a video, or you get a cURL error and/or curious HTTP response, then there is something wrong with the IP or its configuration as an outgoing network interface on your server.

To address this issue, on some servers, you can try commenting out this line of code, like so:

PHP Code:
//curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4); 
You can also try completely disabling IPv6 on your server. Please consult with your hosting company or server admin to do this.

Q. I think a video or group of videos is blocked in my server's country. What can I do?
A. All countries have some blocked videos. All of them. The only way that you could "unblock" all of these videos is to leverage a global proxy network like Tor: https://www.torproject.org/docs/documentation.html.en. In fact, Tor is the only global proxy network that I am aware of at this time, and it is notoriously known for its slow connection speeds.

A more practical approach is to unblock as many videos as you can without spending a lot of extra money, making a lot of extra work for youself, and generally inducing massive, debilitating headaches.

The easiest way to avoid blocked videos is to choose a web server in a country that, generally, has few blocked videos. In my experience, the USA, Canada, and France have the least number of blocked videos. You would do yourself a favor to choose a datacenter located in one of these countries.

Another way to thwart blocked videos is to use dedicated proxy IPs in your HTTP and download requests. In this case, you can buy one or more proxy IPs from a dedicated, 3rd-party proxy service like http://instantproxies.com/billing/aff.php?aff=061. Again, perferably, the proxies should be located in the USA, Canada, or France. Then you integrate the proxy IPs into your PHP cURL requests like so, i.e.:

PHP Code:
curl_setopt($chCURLOPT_PROXY'208.67.222.222');
curl_setopt($chCURLOPT_USERAGENT'');
curl_setopt($chCURLOPT_REFERER'');
curl_setopt($chCURLOPT_SSL_VERIFYPEERFALSE);
curl_setopt($chCURLOPT_SSL_VERIFYHOSTFALSE); 
When using proxies, you should test for a blocked video first by scraping the content of the video page and checking for text that indicates the video is blocked. If the test indicates that a given video is in fact blocked, then you should use a proxy IP. If not, then don't use a proxy IP. The goal is to only use proxy IPs when you need to because, like Tor, using proxies will always be slower than not using proxies.

When choosing proxies, ideally you'll want to do the following:

1) Choose proxies that are geographically very near to your web server. Ideally, the proxy server would be sitting right next to the web server in the same data center.
2) You are the only person using the proxy server, and you are not using it for anything more than proxy requests.
3) Your proxy server is not a virtual server running on a physical server that is hosting other virtual proxy instances (that you don't operate).
4) Spread out your requests to multiple, numerous proxies instead of only having a couple of proxies, for example.
5) The proxy server(s) should be running with modest CPU load averages and comparable network bandwidth and connection speeds (to your web server).

Do NOT use free proxy IPs. You are sharing those proxies with the rest of the world, and they get banned quickly by popular sites (like YouTube).

I have not yet implemented proxy integration with my software. That said, it is on my "To-Do" list, among other things!

Q. How do I enable caching of downloaded/converted files?
A. By default, the software is configured with caching disabled and "direct download" enabled. (When enabled, the direct download feature prevents the software from using FFmpeg if it is not required. For example, if a site user requests an mp4 version of a video, and the corresponding video hosting site already has an mp4 version available, then the video is downloaded and no further FFmpeg conversion occurs.)

A quick disclaimer: Caching isn't for everyone. Some may choose to leave caching disabled, especially those concerned with storing potentially copyrighted material on their server for prolonged periods of time.

For those comfortable with the concept, caching can be a boon with respect to the software's effecient operation on your server! In a nutshell, the feature stores and reuses previously downloaded/converted files, so that the same files aren't downloaded and converted over and over again. This ultimately results in a significant savings of resources (foremost, CPU and bandwidth) on your server!

It is important to note that, in my software, caching relies on the prudent use of FFmpeg. One of the primary challenges of caching, in the context of my software, is to prevent partial files in the cache. That is, files that are not entirely downloaded and/or converted prior to the completion of either process. In general, partial files are the result of user or conversion abandonment (i.e., if the user navigates to another page or closes the browser or browser tab during a given download/conversion). So, to eliminate the possiblity of partial files, I use FFmpeg (in one way or another) for every file that is ultimately stored in the cache. The thing about FFmpeg is that 99.99% of the time, a FFmpeg process will finish. Because it is run as an independent process on the server (separate from the process that serves a given webpage in a user's browser), there is virtually no way for FFmpeg to be interrupted and subsequently fail to complete. Thus, this makes FFmpeg use ideal for determining what is and isn't a partial file, and what should and should not be stored in the cache!

(Please read the code comments associated with the _CACHING_ENABLED constant in Config.class.php to further supplement this explanation of how caching is implemented in my software!)

There are 2 forms of caching available. As described above, each method of caching judiciously (and necessarily) employs FFmpeg, and each method has its own merits. Which version you choose will depend on your specific use case:
  1. The first form of caching is the most complete caching solution. It requires the following constants to be set as such in Config.class.php:

    PHP Code:
    const _CACHING_ENABLED true;
    const 
    _ENABLE_DIRECT_DOWNLOAD false
    By disabling "direct download", the software is forced to use FFmpeg for "every" download/conversion. While at first, this may seem unadvisable because of the resources required to use FFmpeg for every conversion, all is not as it seems!

    In this case, video/audio is only completely re-encoded if the selected file type is not available from the corresponding video/audio hosting site. Otherwise, if the selected file type IS available, then the video/audio is downloaded and only a FFmpeg bitstream copy is performed!

    Compared to fully re-encoding a file, FFmpeg bitstream copy requires only a tiny fraction of the CPU required. That's because bitstream copy is only copying the downloaded file's raw data and codecs into another file "container". It is not performing any additional alterations (i.e., changing codecs, bitrate, volume, etc.) that would require the downloaded file to be fully re-encoded.

    So, via minimal extra use of FFmpeg -- for initital conversions only, we are able to cache all file types, from all video/audio hosting sites! Subsequent downloads/conversions of the same files are delivered from the cache (assuming they are still in the cache), and no further FFmpeg is required.
  2. The second form of caching combines caching with "direct download" to create a kind of hybrid solution. It requires the following constants to be set as such in Config.class.php:

    PHP Code:
    const _CACHING_ENABLED true;
    const 
    _ENABLE_DIRECT_DOWNLOAD true
    With "direct download" enabled, only files that require FFmpeg conversion are cached. Specifically, only mp3 and acc files (excluding SoundCloud) are cached, because only these file types (extracted from sites other than SoundCloud) always require FFmpeg conversion to deliver the end user his/her selected file type.

    Most other files types are available directly from the video/audio hosting sites, and thus these files are downloaded directly without requiring further FFmpeg conversion. (A notable exception is m4a, which can be downloaded directly from YouTube, but must be converted to m4a via FFmpeg for other supported sites.) While these files are not cached (and they must be downloaded every time they are requested), they never require the use of FFmpeg.

    This method of caching has the advantage of using less FFmpeg initially, at the cost of using more bandwidth later to re-download files that are not cached.

Q. How do I configure and maintain the size of the converted files cache?
A. There are 2 relevant constants in Config.class.php that need to be set according to your server resources and preference:

PHP Code:
const _MAX_CACHE_SIZE 100000000;
const 
_CACHE_SIZE_BUFFER 1000000
You also need to set up schedule.php to run regularly as a cron job or scheduled task on your server. (This is the same file used to delete files prior to the development of caching and when caching is disabled!) When caching is enabled, it is recommended that schedule.php run hourly. Again, you must remember to uncomment all unlink() and rmdir() commands in the file for cached files to actually be deleted!

Essentially, _MAX_CACHE_SIZE is the approximate, maximum size that the cache ("store/output" folder) can grow on your server, measured in kilobytes, before the oldest cached files are deleted by schedule.php. Note: Because file deletions are performed as a regularly scheduled task on the server, the actual size of the "store/output" folder may exceed this value between scheduled tasks. So, it is IMPORTANT that this value is set to significantly lower than the actual space available for the "store/output" folder on the hard drive! You can further mitigate this issue by increasing the _CACHE_SIZE_BUFFER constant value, running the scheduled task more often, and/or ensuring that you always have plenty of unused, available hard disk space.

_CACHE_SIZE_BUFFER (also measured in kilobytes) is in fact a buffer of sorts, and it is used to help keep the actual size of the cache as close to _MAX_CACHE_SIZE as possible (in between regular executions of schedule.php). So, if caching is enabled and "store/output" folder size is greater than _MAX_CACHE_SIZE, then the oldest files in this directory are deleted (by schedule.php) one at a time until the cache size is less than _MAX_CACHE_SIZE minus _CACHE_SIZE_BUFFER.

Now, let's examine a real-life example: By default, _MAX_CACHE_SIZE is set to 100 GB and _CACHE_SIZE_BUFFER is 1 GB. Let's assume that these values are sufficient for your purposes. And let's assume that you schedule the cron job (schedule.php) to run at the first (minute) of every hour. Given this scenario, the cache might grow to 110 GB after 45 minutes since the last run of schedule.php, and after a full hour, the cache has grown to 115 GB. At this point, the cron job (schedule.php) runs again, and schedule.php deletes the oldest files in the cache ("store/output" folder) one by one until the cache returns to _MAX_CACHE_SIZE, or 100 GB in this case. Next, if _CACHE_SIZE_BUFFER is set to a value greater than zero (and it is in this example), then schedule.php continues to delete an additional _CACHE_SIZE_BUFFER, or 1 GB, worth of files. So, when all is said and done, the cache is roughly 99 GB when schedule.php execution finally completes. And then the cycle repeats, over and over again, forever.

After each iteration of schedule.php, the idea is to make the cache a little less than _MAX_CACHE_SIZE to accomodate for the duration between scheduled cron jobs (and the expansion of the cache during this time). So that the cache size generally and consistently stays under or as close as possible to the _MAX_CACHE_SIZE.

It's not a perfect science -- to keep the cache size from never going over X size units. Not when cache files are only purged every X units of time. So you will have to experiment with different values for _MAX_CACHE_SIZE, _CACHE_SIZE_BUFFER, and the frequency with which schedule.php runs, all while maintaining an awareness of available hard drive space, to make caching work well enough given the amount of traffic to your site. And, should you experience a sudden increase or decrease in site traffic, then you will want to consider adjusting these settings again, if necessary.

Q. Can I keep popular (or frequently downloaded) converted files in the cache longer than other, less popular files?
A. Yes! This feature is enabled and configurable via the following constants in Config.class.php:

PHP Code:
const _ATTR '/usr/bin/attr'
const 
_DELAY_POPULAR_FILE_DELETION false;  
const 
_MAX_DELETION_DELAY 432000;  
const 
_MAX_POPULAR_FILE_INCREMENT 86400
The _ATTR constant value indicates the absolute path of the "attr" package installation on your server. If not installed by default, this package is easily added via the "apt-get install attr" (Ubuntu/Debian) or "yum install attr" (CentOS/CloudLinux) commands. The "attr" package enables extended attributes for converted files in the cache, so that the file creation date and time can be included in a given file's metadata. (Linux filesystems record last modified times by default, but do not reliably record and retain the true creation times.) Thus, by storing the creation date/time for each cached file in an extended attribute, we can easily determine the true age of any given file.

The _DELAY_POPULAR_FILE_DELETION constant is what toggles this feature on and off. By default, the constant is set to "false", or off. When the value is changed to "true", AND when caching is enabled (see the _CACHING_ENABLED constant), then this feature is switched on.

The final 2 constants define "how" and "to what extent" this feature performs. The exact purpose of these 2 constants, and the overall gist of how this feature works, is as follows:
  1. "Popular" files are files that are downloaded/converted every _MAX_POPULAR_FILE_INCREMENT seconds or less. I.e., a cached file must be downloaded every _MAX_POPULAR_FILE_INCREMENT seconds or less to remain popular.
  2. Every time a downloaded/converted file (brand-new or already in the cache) is downloaded by the end user, the following occurs:
    • The code checks for the existence of a "created" extended file attribute (using the "attr" package), which stores the file creation time in the form of a Unix timestamp.
    • If the "created" file attribute does not yet exist (i.e., this is a brand-new downloaded/converted file), then it is generated and assigned the current time.
    • The code then checks for 2 conditions:
      1. The current time minus the "created" time is less than _MAX_DELETION_DELAY seconds. I.e., _MAX_DELETION_DELAY is the maximum length of time (in seconds) that a popular file is allowed to have its deletion delayed.
      2. The current time minus the last modified time of the file is less than _MAX_POPULAR_FILE_INCREMENT seconds. Again, _MAX_POPULAR_FILE_INCREMENT is the maximum length of time (in seconds) between a given file's downloads for the file to remain "popular".
    • If these 2 conditions both return true, then the file's last modified time is updated to the current time.
  3. The net result is that popular files in the cache, for as long as they remain popular and do not exceed _MAX_DELETION_DELAY seconds, will always be the newest files in the cache (in terms of last modified times of files). So this delays their eventual and organic deletion from the cache, since schedule.php deletes old files in the cache based on the oldest last modified times.
This feature is currently only available in the Linux version of the commercial software.

Q. How can I enable the "Save to Dropbox" and "Save to OneDrive" buttons, so converted files can be saved to the cloud?
A. There are a total of 4 constants in Config.class.php that must be configured to allow this:

PHP Code:
const _ENABLE_DROPBOX_SAVING true;
const 
_DROPBOX_APP_KEY 'INSERT_APP_KEY_HERE';
const 
_ENABLE_ONEDRIVE_SAVING true;
const 
_ONEDRIVE_APP_KEY 'INSERT_APP_KEY_HERE'
For Dropbox integration, please follow these steps:
  1. Set _ENABLE_DROPBOX_SAVING to 'true'.
  2. Proceed to the following page and either log in with an existing Dropbox account or create a new one.
  3. By default, "Drop-ins App (Chooser or Saver)" is selected. Keep this selected, give your new app a name, and click the "Create app" button.
  4. Add your website's host name (i.e., "yourconvertersite.com") to the "Chooser/Saver domains" field!
  5. Use the provided "App key" value as the value for _DROPBOX_APP_KEY in my software.
  6. You're done!
For OneDrive integration, please follow these steps:
  1. Ensure that you have a valid SSL certificate installed and https:// forwarding is enabled. (Microsoft has made this a requirement!)
  2. Set _ENABLE_ONEDRIVE_SAVING to 'true'.
  3. Proceed to the following page and either log in with an existing Microsoft account or create a new one.
  4. Click on the "Add an app" button.
  5. Give your new application a name, and click the "Create application" button.
  6. Click the "Add Platform" button and select "Web".
  7. "Redirect URLs" should be populated with the following URL formats relative to your site:
    • https://yourconvertersite.com{_APPROOT value from Config.class.php}
    • https://www.yourconvertersite.com{_APPROOT value from Config.class.php}
    • https://yourconvertersite.com{_APPROOT value from Config.class.php}index.php
    • https://www.yourconvertersite.com{_APPROOT value from Config.class.php}index.php
    • https://yourconvertersite.com{_APPROOT value from Config.class.php}edit.php
    • https://www.yourconvertersite.com{_APPROOT value from Config.class.php}edit.php
  8. Click the "Save" button at the bottom of the page to save your app settings.
  9. Use the provided "Application Id" value as the value for _ONEDRIVE_APP_KEY in my software.
  10. You're done!

Q. How do I add a new language to the languages drop-down menu?
A. To add a new language, you need to do the following:
  1. Copy "langs/us.xml" into a new file (in the "langs" folder) and replace all content inside <value> tags with the new language's translations.
  2. Name the new language file using its corresponding 2-letter ISO country code. E.g., the French translation file is named "fr.xml". (Language file names must be all lowercase!)
  3. Save the new file with UTF-8 encoding (without BOM).
  4. Edit "langs/index.xml" and add a new XML <country> node corresponding to your new language file, e.g.:

    Code:
        <country id="FR">
            <name>France</name>
            <language>Français</language>
            <langcode>fr</langcode>
            <direction>ltr</direction>
            <file>fr.xml</file>
        </country>
    Please note:
    • The <country> "id" attribute is used to generate a country flag for the corresponding language in the languages drop-down menu. It is also used when setting the default language in Config.class.php. (This value again corresponds to the 2-letter ISO country code.)
    • <langcode> indicates the 2-letter ISO language code.
    • <direction> indicates whether the language is read "left-to-right" (ltr) or "right-to-left" (rtl). The tag only accepts one of these 2 values.
    • <file> must match the name of your new language file.
  5. Again, make sure to save index.xml with UTF-8 encoding (without BOM).
  6. Clear your browser's cache and cookies to see the new language in the languages menu.
  7. That's it!
NOTE: If you add a new language translation to the software, please consider donating your translation so that others who have purchased (or will purchase) my software can benefit from your hard work. You can send your new language file and updated index.xml to admin {at} rajwebconsulting {dot} com. All donated language files are added to future releases of the software (which are free for existing customers).
Q. How do I force all software requests to use "https://" instead of "http://"?
A. If you have a valid SSL/TLS certificate, then you can permanently redirect all HTTP requests to HTTPS via the included .htaccess file.

First of all, in my software, I make a distinction between 2 different kinds of SSL:
  • SSL via CloudFlare, and
  • Regular SSL (or any implementation of SSL that does not leverage CloudFlare).
With this in mind, open the .htaccess file in the software's root directory and locate the following lines:

Code:
    #Uncomment the following lines as instructed to redirect all http:// requests to https://
    #If using Cloudflare SSL, uncomment this line
    	#RewriteCond %{HTTP:CF-Visitor} {"scheme":"http"}
    #If NOT using Cloudflare SSL, uncomment this line
    	#RewriteCond %{HTTPS} off
    #Always uncomment these lines
    	#RewriteCond %{REQUEST_URI} !(.mp3|.aac|.m4a|.mp4|.webm|.f4v|.3gp)$
    	#RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
Per the instructions in this file, uncomment (i.e., remove the # symbol that prepends) the "RewriteCond" that corresponds to your kind of SSL.

Finally, uncomment the last "RewriteCond" and "RewriteRule" in this section and upload the modified .htaccess file to your server. You're done! The software will now redirect all HTTP requests to secure HTTPS connections.
Q. Can I easily integrate interstitial ads (like adf.ly) with the software to generate additional income?
A. Yes! The software facilitates the integration of interstitial ads via the following array in Config.class.php:

PHP Code:
public static $_interstitialAdUrls = array(
   
// Comma-separated list of URL "prefixes" (corresponding to "URL Shortener" services) used for the display of "interstitial" ads.
   // A valid "URL Shortener" is any service that creates an interstitial ad and then forwards the end-user to the provided destination URL, via the general format: "http://ShortenerUrlPrefix/http://ConvertedFileDownloadURL". (I.e., "http://ShortenerUrlPrefix/" is the URL prefix, and "http://ConvertedFileDownloadURL" is the software-generated destination URL where converted file download occurs.)
   // Ads are served after a user clicks on the download button and before he/she downloads the corresponding converted file (following initial download/conversion AND post-conversion editing).
   // Each URL Shortener is used equally by each end-user.
   // Valid URL Shorteners include (but are not limited to) http://adf.ly/?id=12156987, http://adhy.pe/r/0BW, https://shorte.st/ref/df7b233333, and http://ouo.io/ref/x2T0ivf7 .
   // Leave this array empty to disable this feature.    
        
   // The following are Examples! - Replace these URL prefixes with your own prior to use!!
   //'http://adf.ly/6606618/',
   //'http://sh.st/st/c0a11cb9f00cc3b5df2179929f4d63ed/',
   //'http://ah.pe/a/0BW/',
   //'http://ouo.io/s/W51X7Xno?s='
); 
Enabling this feature (i.e., adding one or more valid, ad network URL "prefixes" to the array) serves an interstitial ad to end-users after they click on the download button and before they download the corresponding converted file. Ads are served following initial download/conversion AND following post-conversion editing. If there is more than one URL prefix in the array, then each end-user rotates between ad networks (i.e., URL prefixes) for each subsequent download/conversion -- to maximize the total revenue generated by any given user. This feature is disabled (i.e., the array is empty) by default.

A valid ad network is defined as any service that creates an interstitial ad, that subsequently forwards the end-user to a converted file download prompt following ad impression, via the general URL format: "http://AdNetworkUrlPrefix/http://ConvertedFileDownloadURL". (In this example, "http://AdNetworkUrlPrefix/" is the URL prefix, and "http://ConvertedFileDownloadURL" is the software-generated destination URL where converted file download occurs.)

Valid ad networks include (but are not limited to):

Ad Network Example URL Prefix
adf.ly http://adf.ly/6606618/
adhy.pe http://ah.pe/a/0mU/
shorte.st http://sh.st/st/c0a11cb9f00cc3b5df2179929f4d63ed/
ouo.io http://ouo.io/s/W51X7Xno?s=