Unpack Uploaded files and restrict by MIME types with proftpd

Posted on 2012-04-02 14:50:27
by Geert Vandeweyer

Loading Content

Unpack Uploaded files and restrict by MIME types with proftpd

Posted on 2012-04-02 14:50:27
by Geert Vandeweyer

1. Configuring the proftpd server

I've tested this approach on ubuntu, by compiling proftpd v1.3.4a. 

# download and extract proftpd 
gvandeweyer@ubuntu: wget proftpd-version.tar.gz
gvandeweyer@ubuntu: tar xzvf proftpd-version.tar.gz
gvandeweyer@ubuntu: cd proftpd-version

# download and put vroot module in proftpd/contrib
gvandeweyer@ubuntu: wget mod-vroot.tar.gz
gvandeweyer@ubuntu: tar xzvf mod_vroot.tar.gz
gvandeweyer@ubuntu: cp mod_vroot/mod_vroot.c proftpd-version/contrib/

# compile
gvandeweyer@ubuntu: ./configure --with-modules=mod_sql:mod_sql_mysql:mod_sql_passwd:mod_exec:mod_vroot --with-includes=/usr/include/mysql

# make && make install
gvandeweyer@ubuntu: make && sudo make install

Put the following in the proftpd.conf file

# Use this to jail all users in their homes
<IfModule mod_vroot.c>
    VRootEngine on
    DefaultRoot ~
    VRootLog /var/log/proftpd/vroot.log
</IfModule> 


# RUN THE FOLLOWING COMMAND ON STORE COMMAND
ExecEngine on
ExecOnCommand STOR /full_path_to_script/ftp_check.pl %f
ExecOptions logStdout logStderr sendStdout
ExecLog /var/log/proftpd/execlog
ExecTimeout 600 

2. Create the ftp_check.pl script

Put this script somewhere on the system and make it executable to all users that will log into the ftp server. The script will be run by the user currently logged in, with corresponding permissions. 

#!/usr/bin/perl
$|++;
## use cwd
use Cwd;
use Cwd 'abs_path';
use File::Basename;
## looking at this file : 
my $file = $ARGV[0];
## located in this directory
my $cwd = getcwd.dirname(abs_path($file));
my $file = basename(abs_path($file));

## what is allowed?
my $allowed;
my %unpack;

# allowed type: txt, gzip
$allowed{'text/plain'} = 1;
 
## unpack : tar, tar.gz, bzip, rar, zip
$unpack{'application/x-tar'} = 'tar xf';
$unpack{'application/x-tar-gz'} = 'tar xzf';
$unpack{'application/x-gzip'} = 'gunzip';
$unpack{'application/zip'} = 'unzip';
$unpack{'application/x-rar'} = 'rar x';

print "Checking if uploaded file is of allowed MIME\n";
## wait for the main upload to finish.
my $filesize =  -s "$cwd/$file";
sleep 2;
while ($filesize != -s "$cwd/$file") {
	$filesize = -s "$cwd/$file";
	print "Upload not finished ($filesize bytes so far)\n";
	sleep 3
}

## start processing
&ProcessFile($file,$cwd);


## subroutines
sub ProcessFile {
	my $item = $_[0];
	my $basedir = $_[1];
	print "Working Dir: $basedir\n";
	print "Working Item = $item\n";
	my $mime = `cd $basedir && file --mime-type -b "$item"`;
	chomp($mime);
	print "Item Mime = $mime\n";
	if ($allowed{$mime} == 1) {
		print "  FileCheck '$item' : $mime => ok\n";
		if ("$basedir" ne "$cwd") {
			print "\t=>Moving to Base directory as ";
			my $target = $item;
			while (-e "$cwd/$target") {
				# append digit to prevent overwriting
				$target =~ m/(.*)\.(\w+)$/;
				my $name = $1;
				my $suffix = $2;
				if ($name =~ m/(.*\.v)(\d+)/) {
					$name = "$1".($2+1);
				}
				else {
					$name = "$name.v2";
				}
				$target = "$name.$suffix";
			}
			print "$target\n";
			`mv "$basedir/$item" "$cwd/$target"`;
			
		}
	}
	else {
		if (exists($unpack{$mime})) {
			## make tmp dir
			my $rand = "unpack_".int(rand(500));
			`mkdir $basedir/$rand`;
			## move file (symlinking gave some issues...
			`mv "$basedir/$item" "$basedir/$rand/$item"`;
			my $dump = `cd $basedir/$rand && $unpack{$mime} $item`;
			my @files = `cd $basedir/$rand/ && ls `;
			foreach (@files) {
				chomp($_);
				my $unpacked = $_;
				next if ($unpacked eq $item) ;
				# check the unpacked file
				&ProcessFile($_,"$basedir/$rand");
			}
			## all done, remove the rand directory
			if (-d "$basedir/$rand" && "$basedir/$rand" ne "/" && "$basedir/$rand" ne "//") {
				$dump = `rm -Rf "$basedir/$rand"`;
			}
		}
		elsif ($mime eq 'application/x-directory') {
			## get files in subdirectory
			my @files = `cd $basedir/$item/ && ls `;
			foreach (@files) {
				chomp($_);
				# check the file
				&ProcessFile($_,"$basedir/$item");
			}
		}
		else {
			print "  $item => Filetype not allowed and will be discarded.\n";
			`rm -f "$item"`;
		}
	}
	
}

3. How it works

The script should launch on every 'put/store' command. The mod_exec command ExecOnCommand will launch it after the stor command has successfully ended.

The script will first check if the uploaded file is of an accepted file type, specified in the '%accepted' hash. If true, then the script will exit. 

If the mime-type exists in the 'unpack' hash, the corresponding command will be used to unpack the archive (in a temporary subdirectory), and relaunch the check for the unpacked items, traversing subdirectories on its way. 

If an unpacked item is of an accepted mime-type, it will be moved to the basedir where the original archive was located, double checking filenames to prevent overwriting. 

After unpacking, the temporary directory will be deleted, together with the archive. Any non-accepted file will be deleted as well. 

4. Known Issues

For me, the 'ExecTimeout ' option in proftpd.conf does not accept a '0' value as infinite. So make sure you set it high enough if you expect large files. 

You have to use the non-default vroot module, as the standard DefaultRoot causes issues with mod_exec. 

ExecOnCommand, FTP, Perl