I had a chat with Getty, another Padre developer, some time ago. He said that writing Tests takes much time and he never did one before. Getty gave testing a try and reported success not long ago.
Perl test? What’s this?
Many developers run tests by starting their program or webbrowser and clicking somewhere within the application. They even try to actually use it the way it should be used – sometimes.
Perl tests are much more powerful. They throw a huge number of things against your application and check if everything goes right and they check that errors are handled correctly. Better: They should do – if you write them.
The story begins…
I wrote Padre::File::FTP just before the last release. It was a less-than-one-hour hack and it worked fine, but there are some things left which need to be improved. I decided to face one of them, the exists – method, this morning.
Here is that method:
sub exists {
my $self = shift;
return if !defined( $self->{_ftp} );
return $self->size ? 1 : 0;
}
If there is a Net::FTP – object, check the size. Whatever has a size, must exist. But I didn’t like that zero-sized files were not detected properly and this was going to change today.
FTP and Net::FTP don’t have “exist” functions. Besides the size-request, FTP provides a “ls” command like it exists on Linux and (called “dir”) on Windows, but I didn’t know if it was useful at all. FTP-Servers like to return their own file listing format and I expected a lot of problems.
Know about your enemy
Net::FTP says about “ls”…
ls ( [ DIR ] )Get a directory listing of DIR, or the current directory.
In an array context, returns a list of lines returned from the server. In a scalar context, returns a reference to a list.
Nothing about the format, what information is returned or a sample. To get a sample, I added the following line to the exists method:
print STDERR join("\n",$self->{_ftp})."\n";
But… how to run this method? This is one of the situations where a test script would safe your day.
A new test script was born
Padre got a lovely function to create a new test from scratch within the file – new – menu. But I decided to start using a copy of the existing t/92-padre-file.t today.
I deleted most of the lines and did some minor changes and got the following core few minutes later:
#!/usr/bin/perl
use strict;
use warnings;
use Test::More;
use Padre::File;
if ( ! $ENV{PADRE_NETWORK_T}) {
plan(tests => 1);
SKIP: {
skip 'This test file requires permission to connect to the internet.',1;}
exit;
}
my $file; # Define for later usage
done_testing();
This is already a complete test script, but it doesn’t run any tests. It has the Perl basics (Perl interpreter, pragmas), pulls in Test::More for testing and Padre::File for being tested. As it should run internet tests, it shouldn’t run without the users explicit permission (given by setting an environment variable). It only shows a message and exits without internet permission
done_testing(); ist a Test::More function indicating that all tests have been completed successful (= without test script crash).
Adding the tests
Remember: I want to run the ->exists method of Padre::File::FTP and here we go:
# Plain file from CPAN
$file = Padre::File->new('ftp://ftp.cpan.org/pub/CPAN/README');
ok( defined($file), 'FTP: Create Padre::File object' );
ok( ref($file) eq 'Padre::File::FTP', 'FTP: Check module' );
ok( $file->{protocol} eq 'ftp', 'FTP: Check protocol' );
ok( $file->size > 0, 'FTP: file size' ); ok( $file->exists, 'FTP: Exists' );
You could see one of the biggest advantages of test scripts in this short sample: I want to check the exists function. All I need is a Padre::File – object created using a FTP url. But adding three simple lines gives me a complete check of the the Padre::File::FTP->new method and the Padre::File->new method. Did I get a object? Is it a FTP object?
Running these simple tests creates a known testing environment: I’ld expect to have a good object in $file but I didn’t know it for sure. ->new may fail or return a Padre::File::HTTP object and every following test won’t give any useful result.
Running the test using “prove -l t/94-padre-file-remote.t” returned a successful test result (all tests work using the old ->exists method) and the required information about Net::FTP->ls.
Using the results
Here is the new ->exists method:
sub exists {
my $self = shift;
return if !defined( $self->{_ftp} );
# Cache basename value
my $basename = $self->basename;
for ($self->{_ftp}->ls($self->{_file})) {
return 1 if $_ eq $self->{_file};
return 1 if $_ eq $basename;
}
# Fallback if ->ls didn't help. A file heaving a size should exist.
return 1 if $self->size;
return 0;
}
It loops through the ->ls – results looking for a full-path or filename-only match. If this fails returning true on match. It also tries the old ->size hack if ->ls failed (due to unexpected syntax or other bad things).
More tests
Another advantage of test scripts: They could test more cases than you could or would do yourself.
Cloning some lines (Padre has Ctrl-D for this) resulted in the following additional tests:
# Test some FTP servers
for my $url (
'ftp://ftp.ubuntu.com/ubuntu/project/ubuntu-archive-keyring.gpg',
'ftp://ftp.proftpd.org/README.MIRRORS', # Proftpd
'ftp://ftp.redhat.com/pub/redhat/linux/README', # vsftpd
'ftp://ftp.cisco.com/test.html', # Apache FTP
'ftp://ftp.gwdg.de/pub/mozilla.org/_please_use_ftp5.gwdg.de_', # Empty file
# TODO: Find a public FTP server using Microsoft FTP service and add it
) {
$url =~ /ftp\:\/\/(.+?)\// and my $server = $1;
$file = Padre::File->new($url);
ok( defined($file), 'FTP '.$server.': Create Padre::File object' );
ok( $file->exists, 'FTP '.$server.': Exists' );
my $size = $file->size;
ok( defined($size), 'FTP '.$server.': '.$size.' bytes' );
}
These few lines run the ->exists and ->size methods on a 5 different URLs. Believe me, getting those URLs took much longer than writing the test.
Benefits
Writing this test used up less time than running all these tests manually once (using Padre). One simple prove command could run 26 tests using the Padre::File::FTP module and if anybody reports a FTP server which can’t be used, one additional line (in the for-loop – preparation part) is enough for a sufficient test run.
Every Padre developer and user could replay this test script and we may detect some bugs before they show up in Padre itself. Every failed test is reported including the test script line number – no need to reverse follow the results of the results of the results of a bug.
Writing tests is time-consuming but you’ll get a much bigger payback on time compared to manual testing and bug-hunting.
The final test script is shown here.
