• Why aren't my CGI scripts working?
• Short overview of the CGI calling process
• Server Errors - The server fails to execute the CGI
  1. Misconfiguration
  2. Syntax Errors
• Logical Errors - Errors in the program flow
  1. Tips for Perl Scripts
  2. Tips for C Scripts
• Where to go from here

Why aren't my CGI scripts working?

Here are some common problems encountered with CGI scripts:

1. Your script was uploaded in BINARY mode instead of ASCII mode. Re-upload the file in ASCII transfer mode in your FTP program. This means the script must actually upload as a 'text' file. If your FTP program automatically puts a ".txt" notation at the end of the script, then simply delete the ".txt" from the file name after you upload it to the server.

2. The directory your CGI or Perl script lives in is not set to the proper permission. You do not have the directory permissions set to 755. ( chmod 755 cgi )

3. CGI's must have permissions of "755" or they will not execute. ( chmod 755 *.cgi )

Make sure your scripts are in the cgi-bin directory, the path to PERL and sendmail are correct, and that the permissions are correctly set.

Location of Perl interpreter: /usr/bin/Perl So you have to put the following line as the first line in your cgi script if you want to use Perl:

#!/usr/bin/Perl -w

Absolute path to your Perl/cgi script:
/usr/home/your_account_name/html/cgi-bin/your_script

URL to your Perl/cgi script:
http://www.your_domain_name.com/cgi-bin/your_script

The path to PERL: /usr/bin/Perl
The path to sendmail: /usr/lib/sendmail

You can set permissions with the WS_FTP program. Click once with the left mouse button to select the file you want to check/change, then right click on the file and choose "chmod (UNIX)" (change mode) from the menu. The standard permissions for a CGI script are:

Owner: Read Write Execute Group: Read Execute Other: Read Execute

Make sure that the extension is either .pl or .cgi

Short overview of the CGI calling process

The process is simple: The webserver receives a request for a document in form of an URL and a method type. For simplicity's sake we will only talk about the GET and POST methods but you should be aware that those are not the only methods (At the time of this writing three other methods are in use, PUT , DELETE and HEAD ). The URL is mapped internally to a filelocation. The type of the requested file determines what the webserver does next. To identify the type of the file the filename suffix is mapped to a (customizable) table in the server configuration. If it doesn't find the suffix the server tries to spit out the requested file on STDOUT and the trouble to identify the filetype is left to the browser. If the suffix is found but not registered as a CGI type, the server adds the MIME type that this suffix is mapped to to its output Header and outputs the file. In this case the method POST is not valid; the server will complain if the request is a POST, usually with a message like "Method not implemented". The browser tries to map the MIME type to its own table of recognized filetypes and determines how to hand the file to the user, i.e. displaying the content in the browser window or lauching an external application or plugin.
If the server encounters a file type that it considers to be a handler ( Apache Jargon) for a CGI program (usually that type would be application/x-httpd-cgi ) it will execute the file. But before it executes the script, the server usually does other security checks first, i.e. whether a certain directory is allowed to contain CGI scripts is entirely up to the person configuring the server. If everything is fine the server finally tries to execute the script, and along the way it passes a variety of Environment Variables to the CGI program, like the method, the content- length of the request, and other useful information about the server and the client. From this point on any output needs to be done by the just called CGI application.

At this point there are already a lot of things that could have gone wrong: The extension must be mapped to a file-type and this type needs to be associated with CGI. Many servers also have the possibility to recognize CGI programs by their location, so you can specify a directory in which every file is considered to be a CGI script. Administrators often make use of this and keep all CGI files in a cgi-bin directory. Also, since the server is the one executing the script it needs to have rights to do that. Finally, the script must actually execute and do some output. Errors at this stage usually result in that famous 500 Server Error every CGI programmer has encountered often enough. The output needed consists of a valid Header (the very least you need to output is a MIME type as the first line of the output like this: Content type: <MIME type> ), followed by a blank line and finally some data (without any data you will get an error).

Server Errors

Misconfiguration

403 Forbidden errors should be rather easy to debug. Somewhere along the way to the requested file the server was instructed not to serve the file. This can be either in form of a htaccess directive or a directory along the way was not readable by the server. Check the permissions of the directories. Keep in mind that the server runs under its own userid and that is most likely NOT root but a user with minimal rights. There are ways to run scripts suid meaning with the ID of the owner of the file but until the server gets to the file this doesn't matter - all directories up to that point must be readable by the userid under which the server runs. If you verified the permissions and can still not figure out why you get this error, chances are the problem lies in the server configuration in form of an access restriction to certain paths.

We already covered an example when the 501 Method not implemented error occurs. This error can also occur when the server configuration says not to execute CGI scripts in this directory. In this case the file is treated as a normal file but was called with the POST method - for the server this is a No-No, only cgi-scripts can be called with POST.

Now to the hardest to debug server error, the all-too-generic 500 Internal Server Error : This is basically the servers way of saying "I tried to access that file and it exists but I either can't execute it or the output wasn't at all what I expected." There are many reasons for the first possibility (can't execute the file):

• The userID the server runs under doesn't have permissions to execute the file, check if you have set the file to be executable.
  
  
• The server configuration disallows execution of CGI scripts in this path, find the configuration files and look if that is the case, best ask the administrator who set up the server.
  
  
• An I/O error occured while reading the file (unlikely), try invoking the script again.

If the server executes the file but the output isn't what the server expects (a valid Header followed by a blank line followed by some data) we have another problem which we will examine next.

Syntax Errors

Let's consider the process of executing a CGI script written in Perl (on a unix machine), the first line in the script points to the local Perl interpreter, i.e. #!/usr/bin/perl . What really happens is the server finds the script, sees it is executable and trusts that from now on everything will work flawless, it's out of the server's responsibility. But this is just a textfile, not a compiled program. The so-called she-bang notation on the first line is part of any shellscript that needs to invoke an interpreter, in this case it happens to be Perl. What it says is "Start up this program to run the remaining lines." Make sure that the path in the first line is correct, a common error is to have the first line point to something like /usr/local/bin while the program actually resides in /usr/bin, or the other way around.

If the script can't be interpreted because of syntax errors the interpreter will output some error message and quit. The webserver gets this output and looks for a correct header, but of course something like "Syntax error on line 5" isn't a valid header so the server has no choice but bailing out with a 500 Server error.

So before you ever test a script via the webserver with your browser, check it for syntax errors first! In Perl you can do that by invoking the script with the -c switch from the commandline, i.e. perl -c myscript.cgi . Be aware that the path to your Perl interpreter has to be right as well, so better use the same path for testing that you have defined in your script (in our example /usr/bin/perl -c myscript.cgi ). Whenever possible test on the same machine your script will be running on, so you'll notice newbie-mistakes like uploading the script source in binary mode instead of ascii right away.

Perl scripts on Windows platforms are a different issue, here we don't have a unix shell that will let us use the she-bang notation, but still the problem with syntax errors is the same. On Windows it depends on the webserver how it implements Standard CGI, Microsoft's IIS ® for example needs to have the info which interpreter to call (and how) in the registry, O'Reilly's WebSite ® on the other hand just needs the extension to be registered with the Windows Explorer. (Correction: version 2.0 of WebSite Pro uses its own internal file mapping which can be set in the Server Properties) So if this information isn't given or is wrong the interpreter will never be executed or not executed correctly, resulting in unexpected output and most of the time a standard 500 Server Error. If you have trouble with CGI on Windows you should definetely check out the Perl for Win32 FAQ at http://velocity.activestate.com/docs/ActivePerll . Among other things it contains detailed configuration instructions for all common Windows webservers.

Logical Errors

Of course apart from syntax errors it is hard to verify that the script does what you want; running on the commandline is not always an option since the program flow is often dependant on Environment Variables that are not available when you run the script from the commandline. In this section we will discuss some general and some language-specific techniques to help you validate the program-flow.

One option is to add a routine to your program that gets called when the script runs from the commandline, an easy test for that would be checking for the absence of the Environment variable REQUEST_METHOD, which is only set when called by the webserver. In this routine you can initialize your variables to some expected input, set all Environment variables you need to some basic test data and run the script on the shell. Of course this is a rather troublesome and not always suitable method. If you are using Perl there is a better way to accomplish the same thing, we will get to this later.

What if the script stops execution due to a runtime error? If it does this before you output a valid header you will get a 500 Server error (optionally with the information premature end of script headers ), if it bails out after printing the headers you probably get a Document contains no data error. The problem is that errors are usually reported to STDERR instead of STDOUT , the CGI interface doesn't catch that and you won't see the error message in your browser. A simple, obvious trick helps: redirect STDERR to STDOUT! Make sure that the first thing your program does is outputting the header and flushing the output, so any output after that will be valid and you can see the error messages. I'll show an example in Perl and C later.

Tips for Perl Scripts

• Use the -w switch to enable warnings. add the switch to your first line, i.e. #!/usr/bin/perl -w
  If you are confused about the meaning of some warnings you can always use diagnostics; - This module extends the terse diagnostics normally emitted by both the perl compiler and the perl interpeter, augmenting them with the more explicative and endearing descriptions found in the perldiag manpage.
  
  
• Output the header right away and enable autoflushing of the output buffer:
  $| = 1; print "Content-type: text/html\n\n";
  
  
• Redirect STDERR to STDOUT: open (STDERR, ">&STDOUT");
  I recommend to use this only for debugging, don't leave this in your public script. Not only are other people likely to get confused when being confronted with Perl's error messages or warnings but also you won't be able to track down those errors in the server's errorlog - that can really be fatal, imagine your perfectly fine script breaks one day due to a changed environment or changed permissions of some files and you don't notice it until weeks later after finally some visitor complained. Instead use the CGI::Carp module (see next tip)
  
  
• use CGI::Carp; - This neat module takes care that the standard warn(), die (), croak(), confess() and carp() calls will automagically be replaced with functions that write out nicely time-stamped messages to the HTTP server error log. If you want to send fatal (die, confess) errors to the browser, ask to import the special "fatalsToBrowser" subroutine:
  use CGI::Carp qw(fatalsToBrowser);
  die "Bad error here";
  Fatal errors will now be echoed to the browser as well as to the log. CGI::Carp arranges to send a minimal HTTP header to the browser so that even errors that occur in the early compile phase will be seen. Nonfatal errors will still be directed to the log file only (unless redirected with carpout). See the CGI::Carp documentation for more detail.
  
  
• Always check if I/O or other critical operations were successful, always include a die statement or write your own error reporting routine with some HTML formatting (much better for scripts other people will execute). i.e:
  open(FILE,"$filename") or die ("Can't open $filename: $!");
  
  
• use strict; !!! Perl is very forgiving, you don't need to declare your variables, you can use weird bare words and other little tricks which are certainly nice if you write quick and dirty little 5-liners. You can force Perl to do a stricter syntax checking by including the standard module strict . This requires you to declare every variable before you use it, and kind of forces you to think about the scope of the variables too. You would be surprised how many silly mistakes or otherwise hard-to-track typos can be avoided that way.
  
  
• use CGI; Even if you don't use the CGI.pm module in your script this is a very useful debugging tool. Running scripts on the commandline that use CGI.pm are put in an interactive mode which allows you to enter input to the script by hand. You can set Environment variables, pass POST or GET data and your script will behave as if called via the CGI interface. The module is part of the standard distribution in the current Perl version, earlier versions can pick up a copy at http://www.genome.wi.mit.edu/ftp/pub/software/WWW/cgi_docs.html or your nearest CPAN mirror .

Tips for C Scripts

• Plan debugging ahead of time, define a bunch of macros that will make it easy to print out Environment variables and other critical variables, or to open a textfile to print out input and output.
  
  
• Turn off buffering on the STDOUT stream. If you mix methods of writing to STDOUT while buffering is turned on, you may end up with jumbled output. Use
  setvbuf(stdout,NULL,_IONBF,0);
  to turn off buffering before you output anything else.
  
  
• Output the header right away:
  printf("Content-type: text/html\n\n");
  Flushing the output buffer shouldn't be neccessary if you use the stdio library, it will flush the output automatically after every newline. If you suspect that something is going wrong there or you are reading from a file or socket you can force flushing of the output with fflush(stdout);
  
  
• Redirect stderr to a file or stdout:
  char *errorfile = "error.txt";
  if (!freopen(errorfile,"a+",stderr)) {
     printf("Error redirecting STDERR.\n"); exit(1);
  }

Where to go from here

• Read the CGI Programming FAQ
• Read your server documentation and FAQ.
• Post your question on the newsgroup comp.infosystems.www.authoring.cgi
  (WARNING: Don't use the comp.lang.perl* groups for CGI related questions!)