Ishida Tech Solutions, Ltd.
The personal and professional home of Brad Ishida

Blueprint for a Perl-based HTML File Upload Progress Bar - September 19, 2008

If you need to create a tool to view the progress of a file upload from an HTML form to a Perl CGI script, there are a few things you’ll need to be aware of. I will explain the basic blueprint for what needs to be done to accomplish this task. One of the first challenges you might notice is that when you try to detect the progress from your CGI script, the file has already been uploaded by the time you can try to do anything. This is due to the way that the form data is fed to the CGI script. The key to this problem is to “capture” all the data that is being sent to the form. By the time you’ve declared your variable to hold your CGI data, the opportunity to view the progress of the file upload has already been lost. You need to capture the data being posted to the script before declaring your CGI variable.

######################################
#you need to detect the progress of the file upload before you declare your
#CGI variable
use CGI;
my $q = new CGI;
#at this point in your script, the data has already been transferred to the server
#and you cannot show the progress in real time
######################################
#!/usr/bin/perl

#keep track of how much data transferred so far
my $transferred_so_far = 0;
#set an interval for how often the upload progress will be updated
my $noticelimit = 10240; #this is 10KB, make larger to decrease reporting overhead
#capture the form data and write it to a unique file. This script has already safely determined
#a unique $username based on a login. Your script may have another way of handling this.
open TMP, "> ./$username.post" or die "cannot open file";
while (read (STDIN ,$LINE, 4096)) {
    #this keeps track of how much has been uploaded so far
    $transferred_so_far += length($LINE);
    #we print the data to a unique user file, so we can re-read from a separate progress script
    #while we are busy in this loop, another progress reporting script will read this file and
    #report the data transfer in a separate pop-up window
    print TMP "$LINE";
    if ($currentnotice > $noticelimit) {
        #at every interval of $noticelimit, we will write the upload status to a unique file
        open NOTICE, "> ./$username.notice" or die "cannot open file";
        print NOTICE "$tmpsofar Bytes transferred so far..";
        close(NOTICE);
        $currentnotice = 0;
    }
}
close(TMP);

#now that the data transfer is complete, go ahead and declare your CGI variable and read in
#the captured data from your temp file to standard in
open(STDIN,"$username.post") or die "can't open temp file";
my $q = new CGI;

Now you are in a position where you can create a separate script that will simply read the contents of the $username.notice file and report what it finds. You can set this script to refresh itself every few seconds and that will serve to show the progress of the data transfer. Placing this script in a small popup window that you can launch when the submit button is clicked for the file upload will handle this nicely.

Filed under: Learn by Doing — admin @ 12:26 pm

Run-time error 3041: Cannot open database with a previous version of your application - August 7, 2008

Recently while helping a client who was doing some software testing of CD-ROM software, a problem was discovered where the application would run on some Windows boxes but not on others. The program gave the error “Run-time error 3041: Cannot open database with a previous version of your application”. This error is usually seen with Access databases, so we started there. The program did in fact have an “MDB” file, so the problem was that the MDB database file was created with some version of Access that was not being handled for some reason.

The solution to this problem was found in the Windows registry. Like most up-to-date Windows XP boxes, this one had the Jet 4 database engine. However, there were missing entries that tell Jet 4 how to handle older databases that would have previously been handled by Jet 3 or Jet 2. To fix this, I did the following:

1) Open the registry: Start->run->regedit

2) Create a key in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Jet\4.0\Engines and name it “Jet 2.x”.

3) Under the “Jet 2.x” key, create a string value called “win32″ and modify it to contain “C:\WINDOWS\System32\Msrd2x40.dll” (assuming your WINDOWS directory is C:\WINDOWS)

4) Create a key in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Jet\4.0\ISAM Formats and name it “Jet 2.x”.

5) Under this “Jet 2.x” key, create a string value called “Engine” and modify it to contain “Jet 2.x”.

6) Register the DLL file above and you’re all set. Start->run->Regsvr32.exe C:\WINDOWS\System32\Msrd2x40.dll


These steps tell Jet 4 how to handle Jet 2 databases. You can repeat the steps for Jet 3 databases as well by just using “3″ instead of “2″ in the appropriate places. Also note the file name of the DLL would be changed to Msrd3x40.dll. This fix was applied to machines running Windows XP Pro, Windows XP Home and Windows Vista Ultimate.

Filed under: Learn by Doing — admin @ 9:59 am

Windows XP Login Screen Won’t Let You Type Your Password - July 29, 2008

I was installing a new KVM switch with some Windows XP boxes and this annoying problem with the Windows XP login screen kept occurring. Sometimes you can’t get the password field on the login screen to take the keyboard focus–the cursor just ceases to exist. I noticed I could get around this by pressing CTRL+ALT+DEL twice. This brings up the alternate login dialog box and the keyboard focus problem goes away. After logging in, there are no more keyboard problems. Guess it’s just Windows being Windows. :)

Filed under: Learn by Doing — admin @ 10:08 pm

Pages Created Outside of WordPress Throw 404 Not Found Error - June 22, 2008

This site is fairly new and I’m still getting acquainted with WordPress, but so far I like it pretty well. It’s pretty easy to to keep everything valid XHTML with minimal effort. Now, there’s probably another way to handle this, but… I wanted to create a page outside of WordPress, but still keep it “part of the site” with the same style sheets, layout, etc. I created my new contact page and then I borrowed from my WordPress theme by using “require(‘../wp-blog-header.php’);” This allowed me to use the get_header() and get_footer() functions to give my new page an identical look to my blog pages even though it was not created from within WordPress. This also made it easier to incorporate a captcha into the page to help keep my email address from getting crawled. Everything appeared to be working fine, but when I used the W3 Validator, it informed me that my server gave it a 404 not found error. This was strange because the page loads just fine. I grepped through the WordPress files for “404″ and found that WordPress likes to throw server codes a lot using status_header(404). Normally, I would think leaving that job up to the web server would probably be preferable, but maybe WordPress has its reasons and to be fair–it probably didn’t count on me creating my own pages outside of WordPress and trying to use the theme files to do it.



The piece of code doing this can be found in wp-includes/classes.php.

if ((0 == count($wp_query->posts)) && !is_404()
&& !is_search() && ( $this->did_permalink ||
(!empty($_SERVER['QUERY_STRING']) &&
(false === strpos($_SERVER['REQUEST_URI'], '?'))) ) ) {
    $wp_query->set_404();
    status_header( 404 );
    nocache_headers();
} elseif( is_404() != true ) {
    status_header( 200 );
}

I just altered that “if” statement to also check if the REQUEST_URI file or folder exists in the local path. This solved my problem well enough and now the 404 codes for my home-grown pages have disappeared and all is well again. Just replace or set $YOUR_LOCAL_WEB_ROOT with the local path to the root of your web directory. Here is the altered block:

if ( (0 == count($wp_query->posts)) &&
!file_exists($YOUR_LOCAL_WEB_ROOT . $_SERVER['REQUEST_URI']) &&
!is_404() && !is_search() && ( $this->did_permalink ||
(!empty($_SERVER['QUERY_STRING']) &&
(false === strpos($_SERVER['REQUEST_URI'], '?'))) ) ) {
    $wp_query->set_404();
    status_header( 404 );
    nocache_headers();
} elseif( is_404() != true ) {
    status_header( 200 );
}
Filed under: Learn by Doing — admin @ 6:47 pm

My Favorite vi Editor Tips - June 21, 2008

Nothing quite compares to the vi editor when performing fast edits of files on a Unix command line. Tip “#0″ would be to know your modes. Command mode (press ESC key) interprets your keystrokes as commands (for example, “j” moves down). Insert mode (press I key) lets you actually type stuff into your file. Line mode (press “:” key) lets you do cool stuff like search and replace. Here are my favorite tips.



  1. In Command mode, CTRL+g shows you current line information.
  2. In Command mode, forward slash “/search_term” searches down. Question mark “?search_term” searches up.
  3. In Command mode, “Number” followed by capital G takes you directly to the line number (23G takes you to line 23).
  4. Need to delete pesky ^M characters from a Windows text file? In Line mode you can do “:%s/^M//g” (to type the ^M you need to hold down CTRL and then press V then press M)
  5. Need to quickly comment out the next bunch of lines in a Perl script? In Line mode, this will comment out the next 5 lines starting with the current line. “:.,+5 s/^/# /g”
  6. Need to uncomment those same lines? “.,+5 s/^# //g”
  7. In Line mode “:set ic” will make searches case insensitive. “:set noic” will change them back to case sensitive.
  8. In Command mode, capital “G” takes you directly to the end of the file. “$” takes you to the end of the current line.
  9. In Line mode, use “c” with a search and replace to make vi ask you to confirm each replacement (example: “:%s/Breed/Bread/gc”).
  10. In Line mode, you can delete all empty lines with “:g/^$/ d”
  11. To search for characters such as the Microsoft Word curly quotes or any non-standard character that displays something like “\xe2\xe2\xe3″, you can use CTRL+x to type them (example: CTRL+x followed by “e” followed by “2″ will print “\xe2″)
Filed under: Learn by Doing — admin @ 11:50 pm

The Worst Thing About Access 2007 So Far Today - June 19, 2008

Apparently there is some logic built into the Microsoft Access 2007 find/replace tool that will automatically select “search fields as formatted” if it thinks you need to do that. From what I read on Microsoft’s site, it sounds to me like it should only automatically select this option if you have certain input filters defined. However, on a table and form I used today that had no input filters, the “search fields as formatted” option simply would not go away. If I turned it off, Access would turn it right back on with the next search. This can be a real problem because a Find action can take close to 10 seconds with this option on and rarely takes more than 1 second with this option off.


The only “solution” I could figure out is equally stupid as the problem. However, following these steps did make the problem go away for the remainder of the session.


1) Open your form, click on the field you normally search on and press CTRL+F to bring up the Find tool

2) Turn the “search fields as formatted” option OFF and then perform a search for something that won’t be found (some random string of 30 numbers should do the trick).

3) Access will tell you it couldn’t find anything. Close the Find tool. Close your form. Re-open your form and the “search fields as formatted” option will stay off for the rest of the session.


If this doesn’t work for you, the “input filters” thing might be where you want to investigate next. Access will turn this option on if you have input filters. Also, if you search the entire form as opposed to just a single field, Access will turn this option on so it may be unavoidable in that scenario as well.

Filed under: Learn by Doing — admin @ 9:41 pm

Create a Hybrid CD-ROM for Mac or PC with Shared Data - June 18, 2008

When I create cross-platform children’s educational games on CD-ROM, it’s important to be able to fit a Mac version and a PC version of the game on 1 CD-ROM. The best way to do this is to create a hybrid CD-ROM that will work on either Mac or PC. If you can keep much of the “bulk” of the game in external folders (such as folders of audio files), then these folders only need to exist once on the CD-ROM and both sides can access them–saving lots of CD-ROM space in the process.

To create a cross-platform CD-ROM, you will need a Mac and some CD burning software that supports the hybrid format. Roxio’s Toast is your best bet. Toast has supported this feature for many versions. This example will use Toast Titanium 8 on Mac OS 10.5.3.





Step 1

Create a new Mac disk partition. Use the disk utility found in Applications->Utilities. You will put all the Mac files on this partition including any of the folders you intend to share with the PC side (like the “sound files” folder). When you choose a size for this partition, you need to stay under the physical limit of your CD-ROM, but since the Mac side will house the shared data, it can take the lion’s share of the space.

mac disk utility

Step 2

Create a folder to store the PC files you plan on burning to CD-ROM. Determine what you want the PC side to be able to share on the Mac side and then create an alias (Command+L) on the Mac side of each file/folder you want to share. Copy the alias to the PC folder. Make sure the alias has the exact same name as the real file/folder–to do this, you may need to remove the word “alias” from it since Mac OS automatically inserts that on a new alias. Delete the aliases on the Mac side.

mac partition and pc folder

Step 3

Open Toast and click on the “Custom Hybrid” format on the left side. If you do not see this as an option, go to your Toast preferences and “enable legacy formats”. You will see 2 buttons–1 to select Mac files and 1 to select ISO 9660 files. Click the “Select Mac” button and then drag-and-drop the mounted disk image you created in step 1. Click the “Select ISO” button and then drag-and-drop the contents of the folder that contains your prepared PC files and the aliases to the files/folders on the Mac side. On the PC side, make sure the “Resolve Aliases” box is checked on. On the Settings tab, you will probably need to specify “Allow Macintosh Names” unless all of your PC files and the shared files/folders are 8 characters or fewer. This setting is basically just letting you use long file names that are supported by modern versions of Windows anyway.

custom hybrid format in toast



Step 4

Burn your CD-ROM and then insert it in a Mac and insert it in a PC and make sure you only see what you would expect. The significant part is that the PC side that has the shared files/folders–the aliases look like real files and folders and the sharing of that data is completely seamless.

Filed under: Learn by Doing — admin @ 10:19 pm

Recompile sendmail to Handle listserv probe bounces in FreeBSD - June 15, 2008

This fix was performed using sendmail 8.13.8, FreeBSD 6.2 and listserv classic 15.5.

L-Soft makes one of the oldest email list management tools there is–listserv. In fact, the word “listserv” is often improperly used to refer to any software that sends email to lists. Similarly to how one might ask for a “Kleenex” or have something “Xeroxed”–listserv has become a generic term for all email list tools whether they have anything to do with L-Soft or not.

If you want to use sendmail as your MTA (or you need to because the L-Soft MTA is not offered on your platform), you will need to recompile sendmail to handle the listserv probe that handles bounces so they can be caught and logged by listserv. There was a time when somebody named Valdis Kletnieks was distributing an actual patch file, but I haven’t seen an updated version floating around the web for a while. He should certainly be given credit for the hard part of this fix. You could use the information below to make your own patch if you would rather manage it that way.




The patch makes a couple small changes to alias.c and then you’re ready to recompile. You can grab a copy of my alias.c file here.

If you’re using FreeBSD, you’ll likely find your alias.c file here:
/usr/src/contrib/sendmail/src/alias.c

After you get your updated alias.c file in place, you’re almost ready to recompile. You need to pass a LISTSERV flag to the c compiler. One way to do this is to add it to the Makefile in /usr/src/usr.sbin/sendmail/Makefile. Find the CFLAGS variable in the Makefile and then add this line:
CFLAGS+= -DLISTSERV

You can see a copy of my altered Makefile here.

Now recompile sendmail:
1) cd /usr/src/lib/libsmutil
2) make depend && make all
3) cd /usr/src/usr.sbin/sendmail
4) make depend && make all install

Restart sendmail and you’re ready to go:
1) cd /etc/mail
2) make stop
3) make start

Filed under: Learn by Doing — admin @ 7:19 pm

Hello. - June 6, 2008

Welcome to my new site! My name is Brad Ishida and I like tech–a lot. Whether it be business or personal, my life tends to center around technology in one way or another. Creating a site to manage both sides only seems to make sense.

The first order of business for the new site was to create a suitable WordPress theme. My goal is to maintain a high information density for this site, so a minimalist theme seems appropriate. You may notice the lack of graphical components to this design. This is in stark contrast to Programming Art–my other company’s website. I’m sure the theme will continue to evolve, but at this time, I’m really enjoying the fluid layout that should accommodate any browser size and the nearly instant load time.

There are some new words on the About page. Since I went to all the trouble to talk about myself in third person for an entire page, it would be great if you went there now and read it–thanks.

Email? One nice thing about having a new domain name like this one is not having to deal with 1000′s of spam messages every day. Even though SpamAssassin does a fine job at keeping the trash sorted, it’s still a chore to scan for those rare false-positives before deleting 5000 messages at a time. I’m sure the lack of spam is a result of not having any email addresses published on this site to be crawled. So… let’s see how long this holds them off–if you want to contact me, use my last name as the first part of the address and use the domain of this website as the last part.

Filed under: Uncategorized — admin @ 10:37 pm
« Previous Page
© Copyright 2008-2011 Brad Ishida