New OS!

29 12 2009

So after a good long while on Ubuntu Server, and after a very long and healthy learning experience, we have decided it is time to give our system an UPGRADE! Some of you probably thought you would never see another post on this blog again, but the truth is, I just haven’t done much with the server since I wrote that Cron Job. I had it set up in a way which was usable, but not very functional. It was handy for uploading/downloading all sorts of media and I was even running a forum for the family on it, but it was lacking. Basic features allowing any computer to automatically back up to the server and useful web applications such as document storage systems were missing.

This week I am undertaking the process of switching to Novell Open Workgroup Suite Small Business Edition, or NOWSSBE for short. You can read more about it here, but it basically adds a business element to the server which we were missing with Ubuntu. Now, I know that I could have this same set up with Ubuntu, including the Samba NFS and everything, but I felt like learning a new system and starting completely anew. NOWSSBE seems like a useful system to learn, and I’m pretty excited to get into it. I have all the files downloaded now and will be installing either tonight or tomorrow. More posts to come!

Note: Opinions on NOWSSBE would be appreciated.





My first Cron Job

5 11 2008

So it’s been a little over a month since I’ve posted anything, and that’s cause I’ve been kind of busy with school, and therefore haven’t done much with the server. I did install phpBB3, but I didn’t think that was really worth a mention.

What I just got done doing is writing my first shell script and assigning it as a daily cron job. Basically what the job does is run through my Pictures directory–which I linked to at /var/www/Pictures/–and places an instance of the file from the “Image Thumbnails” post in each of the subdirectories. I did this because I found it annoying to have to constantly check the Pictures directory to see if anyone uploaded any new pictures, and then having to place the file in there myself. So, what the script does is run through all the directory’s contents and if it finds a directory it changes into that directory and copies a file called noindex.php from the maindirectory into a file called index.php in the directory it’s in. It then changes back out and continues the process until the entire contents of the folder is processed. Here’s the script, and a sample output.

#!/bin/sh
#Script Name: index.sh
#Author Name: Willem Ellis
#Date: Tues Nov 4 17:22:00 CST 2008
#Description: This is a file used to insert
# index.php into newly created
# sub-directories within a main
# directory.
cd /var/www/Pictures/
FILES=*
for f in $FILES
do
if [ -d "$f" ]
then
echo "Processing $f..."
chmod 777 "$f"
cd "$f"
pwd
cat "../noindex.php" > index.php
cd ..
elif [ ! -d "$f" ]
then
echo "Not a directory..."
fi
done

Processing Folder1…
/var/www/Pictures/Folder1/
Not a directory…
Processing Folder2…
/var/www/Pictures/Folder2/

This site is where I got a small part of this script, it explains what the for loop does. Basically it stores the list of files within the directory as the variable FILES. The for loop scrolls through the loop and saves each individual item in the list as f and works with each item one at a time until all the items have been processed. It’s saying for the variable f in the list FILES do the following… Then my if else if loop just says if the file you are on (f) is a directory change into it and copy noindex.php to index.php inside the directory. Otherwise if (elif) the file is not a directory, print out “Not a directory…” Each time the for loop completes, it moves to the next item and stores that as f until there’s nothing left in the list, at which point the script is done.

After I wrote the script, all I did was save the script as index.sh in /etc/cron.daily/ and edited /etc/crontab to run the job every day at 6:00PM.

00 17 * * * root /etc/cron.daily/index.sh

Hope someone else finds this handy.





Image Thumbnails

30 09 2008

So I have a directory on my server with several (more than 50) sub-directories filled with pictures. Obviously this is a lot of pictures, and since I did not name the images as I uploaded them, I have no idea what’s what. I just know where and when they were taken from the folder titles. I thought something that would generate thumbnails would be great for browsing through the pictures, especially if you were looking for a certain image. So I found this script, and added a two lines that would automatically create a thumbs directory in the Image directory, and that would redirect to the newly generated html page.

Basically what the script does is resize all the .jpg files in the directory, and saves them in the newly created thumbs directory. It then creates a file called gallery.html which shows all the new thumbnails with links to their originals. It does pretty much exactly what I was looking for. Originally the script was called create_thumbs.php, but I renamed it to index.php so it would load each time you opened the folder. In case images had been added. Then I added a redirect line so that it would automatically redirect to gallery.html. This is what I ended up with.
<?php
/*
This is the PHP code for the How to Create Thumbnail Images using PHP Tutorial
This script creates all of the thumbnail images and the gallery.html page.
Note: Make sure that PHP has permission to read and write
to the directory in which .jpg files are stored and the directory
in which you're trying to create thumbnails.
You may use this code in your own projects as long as this
copyright is left in place. All code is provided AS-IS.
This code is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Copyright 2007 WebCheatSheet.com
*/

header( 'Location: ./gallery.html' ) ;
mkdir("./thumbs", 0777);
function createThumbs( $pathToImages, $pathToThumbs, $thumbWidth )
{

// open the directory
$dir = opendir( $pathToImages );

// loop through it, looking for any/all JPG files:
while (false !== ($fname = readdir( $dir ))) {
// parse path for the extension
$info = pathinfo($pathToImages . $fname);
// continue only if this is a JPEG image
if ( strtolower($info['extension']) == 'jpg' )
{
echo "Creating thumbnail for {$fname} <br />";

// load image and get image size
$img = imagecreatefromjpeg( "{$pathToImages}{$fname}" );
$width = imagesx( $img );
$height = imagesy( $img );

// calculate thumbnail size
$new_width = $thumbWidth;
$new_height = floor( $height * ( $thumbWidth / $width ) );

// create a new tempopary image
$tmp_img = imagecreatetruecolor( $new_width, $new_height );

// copy and resize old image into new image
imagecopyresized( $tmp_img, $img, 0, 0, 0, 0, $new_width, $new_height, $width, $height );

// save thumbnail into a file
imagejpeg( $tmp_img, "{$pathToThumbs}{$fname}" );
}
}
// close the directory
closedir( $dir );
}

function createGallery( $pathToImages, $pathToThumbs )
{
echo "Creating gallery.html <br />";

$output = ";
$output .= "<head><title>Thumbnails</title></head>";
$output .= "<body>";
$output .= "<table cellspacing=\"0\" cellpadding=\"2\" width=\"500\">";
$output .= "<tr>";

// open the directory
$dir = opendir( $pathToThumbs );

$counter = 0;
// loop through the directory
while (false !== ($fname = readdir($dir)))
{
// strip the . and .. entries out
if ($fname != '.' && $fname != '..')
{
$output .= "<td valign=\"middle\" align=\"center\"><a href=\"{$pathToImages}{$fname}\">";
$output .= "<img src=\"{$pathToThumbs}{$fname}\" border=\"0\" />";
$output .= "</a></td>";

$counter += 1;
if ( $counter % 4 == 0 ) { $output .= "</tr><tr>"; }
}
}
// close the directory
closedir( $dir );

$output .= "</tr>";
$output .= "</table>";
$output .= "<br />";
$output .= "<a href = 'http://thesupportdepartment.com/Pictures/Willem Sr Pictures'>Return To Pictures</a>";
$output .= "</body>";
$output .= "</html>";

// open the file
$fhandle = fopen( "gallery.html", "w" );
// write the contents of the $output variable to the file
fwrite( $fhandle, $output );
// close the file
fclose( $fhandle );
}

// call createThumb function and pass to it as parameters the path
// to the directory that contains images, the path to the directory
// in which thumbnails will be placed and the thumbnail's width.
// We are assuming that the path will be a relative path working
// both in the filesystem, and through the web for links
createThumbs("./","thumbs/",100);
// call createGallery function and pass to it as parameters the path
// to the directory that contains images and the path to the directory
// in which thumbnails will be placed. We are assuming that
// the path will be a relative path working
// both in the filesystem, and through the web for links
createGallery("./","thumbs/");
?>

You place it in the folder where the images are located, name it index.php, make sure the folder is chmodded to 777 and when you navigate to that folder, it will create a thumbs directory and the gallery.html file. Then it will redirect to your gallery.

Finally, since I had to place this file in a lot of directories, I placed a copy of the file in the parent directory. Changed to the parent directory, and issued the following Unix command. Then I deleted the copy located in the parent directory.

sudo for dir in *; do [ -d “$dir” ] && cp index.php “$dir” ; done

I got the script from here.





SFTP and SCP

23 09 2008

So after uploading the majority of what I wanted to the server, I decided it was time to close the ftp port. I read a lot of stuff about it being the most vulnerable point, and one of the first ports hackers try. So I read up on SFTP or Secure FTP, and SCP and decided to learn how to use that. Since I had already installed SSH on the server, SFTP and SCP were ready to be used. So I closed the ftp port and have been using these ever since.

SFTP is alright for downloading single files and the way I use it, you can upload music folders pretty easily, since the filenames in a music folder should end in the same extension i.e .mp3. SCP is for downloading directories recursively, so you don’t have to download each individual file, although I suppose you can use SFTP to download like I use it to upload. Since I haven’t found something that allows me to recursively upload with either of these (SCP doesn’t support the put command), I have to improvise a little. Anyway, you can use this source to learn SFTP commands, although if you already know unix, the commands are basically the same. This site on the other hand, explains how to use SCP.

Right now, when I wanna upload a directory with music and it’s structured:

Artist

—-|_    Album

————–|_   Song1.mp3

—————— Song2.mp3

—————— Song3.mp3

I do the following. First I create the Artist and Album folders on the remote server. Then I run the following after connecting via SFTP.

sftp> put *.mp3

The * is a wildcard, which basically says you don’t care about any characters before that, and then the .mp3 would be what I assume (or know) all the songs in that folder end in. This has proven to work well so far, I still don’t have a solution for directories with various types of files. I’m open to some suggestions.





A better way!

20 09 2008

Well, I was told by a reader that .htaccess is not a very secure way to block off parts of your website. Since I am busy learning everything for the first time, I was willing to take his advice and was able to use httpd.conf to restrict access. Also, he showed me a way that sends passwords encrypted, rather than in plain text. Anyway, I’d like to thank Bob for this advice, and here is a much better way to block off parts of your site…
(Please note the following is directly quoted from Bob’s comments as to not take credit for this myself.)

You should never use .htaccess files unless you don’t own the server and don’t have access to the main server configuration file (httpd.conf).

See the Apache documentation article When (not) to use .htaccess files: http://httpd.apache.org/docs/2.2/howto/htaccess.html#when

Basically, httpd.conf would, in your case, look something like:

<Directory>

AuthType Basic
AuthName “SECRET SQUIRREL RESTRICTED ACCESS AREA”
AuthUserFile /etc/apache2/passwd/passwords
Require valid-user

</Directory>

The /var/www/music is the directory you want to password.
AuthType Basic is basic HTTP authentication, as opposed to AuthType Digest which uses MD5 authentication.
AuthName is simply the text that will be displayed in the password request box.
AuthUserFile is where the passwords are stored, which you create using the htpasswd command.
And finally, Require valid-user means that anyone in the password file can get in. Or, you can specify a particular name, such as Require Willem.

See Apache access control for more info: http://httpd.apache.org/docs/2.2/howto/auth.html#gettingitworking

Bob

But wait! There’s an even better way!

AuthType Basic sends the password in clear text, so it’s fairly unsecure. AuthType Digest uses a little bit of encryption with an MD5 checksum and a once-used number and does not send the password in clear text, so it’s more secure.

And, since AuthType Digest is really no harder to use than AuthType Basic, you might as well get the enhanced security. The only real drawback that I see with AuthType Digest is that really old browsers don’t support it. But unless you’re using a browser from 2004, that’s not much of a problem )

Anyway, AuthType Digest is really similar to AuthType Basic except that the way you create a password is slightly different and the httpd.conf entry refers to ‘Digest’, rather than ‘Basic’.

For example, to use AuthType Digest to password-protect the ‘private’ realm and directory /var/www/music and only let in user Joe. First, create a password for joe:

htdigest -c /etc/apache2/passwordfile private joe
(note: location of password file is just an example and doesn’t have to be at that location or use that file name)

Then edit httpd.conf file to require password to access /var/www/music:

<Directory /var/www/music>
AuthType Digest
AuthName “private”
AuthDigestDomain /music/
AuthUserFile /etc/apache2/passwordfile
Require user joe
</Directory>
(Note: replace ( and ) with in the real httpd.conf file. I just used parentheses here so the HTML wouldn’t be stripped out by the blog software).

See here for more techie details: http://httpd.apache.org/docs/2.2/mod/mod_auth_digest.html

Good luck, and enjoy your Ubuntu server! I sure enjoy mine! )

Bob

One more thing on Digest. It is an Apache mod-file, which Ubuntu does not have enabled by default.

If you look in /etc/apache2/mods-available, you will see it: auth_digest.load

But, if you look in /etc/apache2/mods-enabled, it isn’t there by default in Ubuntu server.

But enabling it is easy. If you notice that in /etc/apache2/mods-enabled there aren’t actually any files, just symbolic links to the /etc/apache2/mods-available directory (do an ls -l). So, to enable the auth_digest.load, all you have to do is create a symbolic link like so:

cd /etc/apache2/mods-enabled
sudo ln -s ../mods-available/auth_digest.load auth_digest.load

Then restart Apache:
sudo /etc/init.d/apache2 restart

Have fun! )

Bob

I can verify that this did indeed work, and once again, thanks to Bob for showing me a better way.





.htaccess

14 09 2008

So after I made my music available in my web root, I felt it necessary to password protect it. I don’t want anyone to download my music or stream it or anything like that. No one except me, that is. No illegal activity should be coming from my server I figured. So I thought of the most basic way to do it which was htaccess.

Basically, you create a file called .htaccess with certain parameters. Here’s mine:

AuthType Basic
AuthName "Music"
AuthUserFile /var/.htpasswd
require valid-user

I’m not exactly sure what

AuthType

is, but

AuthName

is whatever you wanna call the folder you’re protecting. It doesn’t change the folder name, this is just what shows up in the little login window that pops up when you navigate to this folder.

AuthUserFile

is where the .htpasswd file is stored. The .htpasswd file specifies which user and password combinations are allowed to access this folder.

require valid-user

just lets it know that any valid user specified within the .htpasswd file is required for access.

In .htpasswd, each individual user should have his own line, and username and password should be seperated by a colon. i.e.

username:password
username:password

Finally, in your apache settings, specifically if you’re using Ubuntu the file will be located at /etc/apache2/sites-available/default. Find the line

AllowOverride None

and change it to

AllowOverride All





Real Quick

10 09 2008

Sorry for the two short posts after each other, but this is definitely worth a mention.

With the external HDD working, and everything being saved on there, I wasn’t sure how to make it accessible without having to connect via FTP every time. More importantly, everyone doesn’t necessarily wanna download every song they wanna listen to, so streaming is nice.

Since all the music and pictures are saved outside the webroot (/var/www/) I wasn’t sure how to make them accessible within the webroot. Well, after searching around, I found this line. It creates what’s called a Symbolic Link. For those familiar with windows, it’s basically like a shortcut, except it’s a little different because it treats it as though the files are saved in the location where the link is. I don’t know if I did a good job of explaining that, but you’ll see what I’m talking about if you have to do this for yourself. So the line is

sudo ln -s source_file my_file

The sudo gives you the correct permissions to do this, which is necessary in Ubuntu. source_file gets replaced with the path to the file (or folder) you wanna make the symbolic link for. my_file gets replaced with the path where you want to place this link.

So now when I open up my page and click on the music link, it takes me to all my music and I can stream it straight through my browser. Pretty sweet!