Download url and store to local file

Troubleshooting.

Storing a page from a website is possible with the build-in copy function of PHP. Unfortunately this does not always work. This page shows how you can investigate where the problem is.

Troubeshooting

It is necessary that allow_url_fopen is enabled. You can check this with the following script. Just save it as a file on your webserver and run it.

<?php
if (ini_get('allow_url_fopen')) {
	echo 'I can handle urls';
} else {
	echo 'I cannot handle urls. Trying to enable url handling...';
	ini_set('allow_url_fopen', true);
	if (ini_get('allow_url_fopen')) {
		echo ' --- succeeded.
'; } else { echo ' --- failed.
'; } }

You should see something like

  • I can handle urls – this is fine, everything should be working
  • I cannot handle urls. Trying to enable url handling… – url handling is not enabled by your webhoster / your php configuration in php.ini.
    — succeeded. – url handling had been enabled, lucky guy!
    — failed. – url handling could not be enabled, will be the default case. Bad luck, look for another webhoster that allows this (or change your php.ini settings).

Work-a-round

Also known as a hack, quick and dirty solution, workaround, we simulate the copy function. Just use the following function instead of the build-in php copy function.

function storeUrlToFilesystem($url, $localFile) {
	try {
		$data = file_get_contents($source);
		$handle = fopen($destination, "w");
		fwrite($handle, $data);
		fclose($handle);
		return true;	
	} catch (Exception $e) {
    		echo 'Caught exception: ',  $e->getMessage(), "\n";
	}
	return false;
}

You can use it like this:

<?php
$source = 'http://www.google.com';
$destination = 'internet.txt';

if (storeUrlToFilesystem($source, $destination)) {
	echo 'Download of the whole web to '.$destination.' succeeded.';
}

Hope this helps.

Licensing: just use this as you wish. Links are always great. 🙂