Kent Oyer

So here’s the scenario: I recently migrated from Small Business Server 2003 to SBS 2008. It was a fairly smooth transition except for one thing: There was a web application written in classic ASP that connected to an Access database.

Problem #1: No 64-bit JET driver
As you may know, SBS 2008 is required to run as a 64-bit operating system. That means IIS is also 64-bit. Unfortunately, Microsoft does not provide a 64-bit Jet driver for accessing Access databases. ASP worked but as soon as I tried to open a connection object, I got this error:

500 – Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed.

Tracking this error down was a bit tricky. I had to enable Failed Request Tracing. When I reviewed the trace log, I found this:

Error 800a0e7a
Provider cannot be found. It may not be properly installed.

The provider string I was trying to use was:

Provider=Microsoft.Jet.OLEDB.4.0

Solution:
Create a new Application Pool and set it to 32-bit mode

Set the Default Web Site to use the new application pool

Problem #2: Application Pool Crashes
When I tried accessing my site again, I received a different error message

503 Service Unavailable

I also noticed that the application pool had changed it’s state to “Stopped”. Not good. A quick look in the Windows Event Viewer revealed event 2280:

The Module DLL C:\Windows\system32\RpcProxy\RpcProxy.dll failed to load.

RpcProxy is a 64-bit DLL required for Exchange to work properly. For some reason, it was trying to load in my 32-bit application pool causing the pool to stop working.

Solution:
Edit the c:\windows\system32\inetsrv\config\applicationhost.config file. Search for the following line and add preCondition=”bitness64″

<add name=”PasswordExpiryModule” image=”C:\Windows\system32\RpcProxy\RpcProxy.dll” preCondition=”bitness64″ />

Problem #3
Compression module does not load because there is no 32-bit driver. This will cause you to receive the following error:

HTTP Error 500.19 – Internal Server Error
The requested page cannot be accessed because the related configuration data for the page is invalid.

If you look in the trace log, you will see something like this:

ModuleName StaticCompressionModule
Notification 16
HttpStatus 500
HttpReason Internal Server Error
HttpSubStatus 19
ErrorCode 2147942526
ConfigExceptionInfo
Notification MAP_REQUEST_HANDLER
ErrorCode The specified module could not be found. (0×8007007e)

Solution:
The solution is to disable HTTP compression. Unfortunately you can’t disable compression on a site-by-site basis so you will have to disable it server-wide. Run this command on the server to disable HTTP compression:

%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/httpCompression /-[name='xpress']

Resources:
ASP & Jet Provider
http://forums.iis.net/t/1066385.aspx

500.19 Error When Enabling 32-bit Application Pool
http://forums.iis.net/t/1149768.aspx

The Module DLL C:\Windows\system32\RpcProxy\RpcProxy.dll failed to load
http://forums.iis.net/t/1154189.aspx

Using Classic ASP with Microsoft Access Databases on IIS 7.0 and IIS 7.5
http://learn.iis.net/page.aspx/563/using-classic-asp-with-microsoft-access-databases-on-iis-70-and-iis-75

Yesterday I noticed some odd behavior on one of the Linux servers that I maintain. For one thing, every time I would run ‘top’ or ‘ps’, I would see the following message

Unknown Hz value! (75) Assume 100

I decided to run rkhunter to check for rootkits. Needless to say, I was disappointed to discover that I had the SHV5 Rootkit on my machine. Let me tell you, it’s not a good feeling to know you’ve been hacked. Now I know common wisdom dictates that rootkit-infected machines be formatted and reloaded from scratch but that is not something you do without a lot of planning and preparation. The first thing I wanted to do was get this bad guy off my machine and try to minimize the damage.

How it works

I did some quick googling and I discovered this page which was very helpful. However, I wanted to add a few more details that might help somebody out in the future. Basically what I discovered is that the rootkit installs an SSH server as a backdoor for the hacker to obtain access to your machine. In my case it was running on port 6522 but this is configurable by the hacker so your mileage my vary.

The hacker does not need to know any passwords to login to your system because the rootkit installs a public key file. The hacker simply needs to have the corresponding private key to authenticate to your server. Unfortunately all of this was invisible to me because the rootkit modifies several core utilities such as ‘ls’ and ‘netstat’ to hide it’s presence. The modified versions of these utilities work just like the real ones, except they don’t display anything that might be incriminating such as certain files and folders, or tell-tale processes.

Cleaning it up

The first thing you have to do before you can start cleaning up a rootkit infection is to get your original system files back. Luckily, the rootkit makes this rather easy by setting the immutable bit on the modified files. The purpose is to prevent anyone from deleting or overwriting the hacked files but it also gives us an easy way to tell what’s been modified. After you run rkhunter, there should be a log file in /var/log called rkhunter.log. Search through this log file to see what files have the immutable bit set. These are the ones we need to replace. I’ll use /bin/ls in my examples.

First, unset the immutable flag by using this command:

chattr -sia /bin/ls

Then, you can optionally make a copy of the hacked file for future use. This is helpful to compare the output from the hacked version to the output of the real version to see what’s different.

mv /bin/ls /bin/ls.hacked

Next, figure out which RPM your file belongs to by running this:

rpm -qif /bin/ls

This will tell you the source RPM. To get the binary RPM just replace ‘.src.rpm’ with ‘.i386.rpm’. For example, on my machine, the source RPM for /bin/ls is:

coreutils-5.97-19.el5.src.rpm

So the file I needed to download was:

coreutils-5.97-19.el5.i386.rpm

You can Google the name of this file or look on popular RPM sites such as rpmfind.net or rpm.pbone.net. Once you are sure you have the correct file, install it using the –force command.

rpm -i –force coreutils-5.97-19.el5.i386.rpm

This tells the installer to try to reinstall the package even though it’s already installed and to replace files as necessary. If you get any errors about files that it can’t  replace, it’s probably because those files have the immutable bit set as well. Clear the immutable bit on those files and install the package again until there are no errors.

Repeat this whole process for every file in rkhunter.log that has the immutable bit set. You will probably have to download several different packages.

Now that you have reliable system commands you can begin ripping out the rootkit. Use netstat to see what ports are open and look for anything suspicious.

netstat -ln –programs

This is where it’s handy to have the output from the hacked version of netstat so you can compare. In my case, I found a program called ttyload that was listening on port 6522. I killed the process and deleted the file. After that I confirmed that my server was no longer accessible on that port. I also used the information in rkhunter.log to locate and delete a number of other files and folders as well. Here’s a list of some of the more interesting ones.

Make sure you check your /etc/inittab and all the files in /etc/rc.d to see if the rootkit is loading anything at startup. In my case I had to remove the line that calls /usr/sbin/ttyload from /etc/inittab.

Final thoughts

Here are some final thoughts on what to do next.

1. Change the root password and the passwords for any other privileged accounts you may have.
2. Reboot the server and make sure things are still normal after the system comes back up.
3. Use iptables to block incoming connections on all ports other than the ones you need.
4. Strongly consider a full format and reload of your system.
5. Run rkhunter on a regular basis

I wanted to dump all my MySQL databases using the mysqldump utility, but I wanted each database to be in a separate file. I couldn’t find a solution online, so I wrote my own script. Here it is:

#!/bin/sh
USERNAME=admin
PASSWORD=topsecret
BACKUPDIR=/var/local/mysql-backups

for i in $(mysql -u $USERNAME -p$PASSWORD -e "SHOW DATABASES;" --skip-column-names --batch)
do
   echo "Backing up database $i"
   mysqldump -u $USERNAME -p$PASSWORD --opt $i | gzip > $BACKUPDIR/$i.sql.gz
done

Of course, you will need to edit the first two lines to reflect your own username and password. When you run this script, it will create a bunch of files in the /var/local/mysql-backups directory, one for each database. The files are compressed to save space.  Of course, you’ll need to make sure the destination directory exists.

What is Personal Antivirus?

Personal Antivirus sounds like a wonderful program that might be good to have, right? Actually, it’s just one of an increasingly common type of malware called a rogue anti-virus or rogue anti-spyware program. It appears to be harmless or even beneficial on the surface, but it’s actual goal is to scare you into thinking you have some nasty virus that can only be removed by purchasing their removal software. It’s sort of like digital blackmail. Similar “products” include XP Antivirus, Antivirus 2008/2009 , Antivirus 360, etc…

How do I get rid of it?

Unlike most legitimate programs, there is no automatic un-installation method. That’s because the people behind these scams don’t want you to remove their software. They usually don’t show up in the list of programs in the Add/Remove Programs control panel applet. However, in many cases, manual removal is simple. Here are the steps I use. Keep in mind there are many different variants of this program so your mileage may vary.

Step 1: Terminate the application

Open Task Manager and look for a process called “pav.exe”. Highlight this process and click End Task. Once the process has been ended, the tray icon and application window should disappear.

Step 2: Prevent the program from starting up

Download a program called HijackThis. After you’ve installed it, run HijackThis and do a system scan. Look for any reference to the file pav.exe. Usually there will be at least one reference on a line that starts with “Run”. Put a check beside any that you find and click “Fix”

Step 3: Delete the program files

Delete the Personal Antivirus folder located in C:\Program Files.  You should also delete the shortcut on the desktop and the Personal Antivirus folder in the start menu.

That’s it. Reboot your computer just to make sure it doesn’t come back at startup. One optional step is to run a registry cleaner like CCleaner after you’re done. That should remove any leftover registry keys that reference the missing program file folder.

So do I really have a virus?

Just because Personal Antivirus was on your system and it said you have a virus, that does not mean you’re actually infected. However, I strongly recommend doing a full system scan with a legitimate anti-virus program and an anti-spyware program just to be safe. If you don’t already have one, here’s a list of some free alternatives.

Free Anti-Virus software (install only one at a time)
AVG
Avast
Avira AntiVir

Free Anti-Spyware software
Spybot Search & Destroy
AdAware

Blackberries. You gotta love em. They’re great when they work but sometimes it’s a challenge to get them setup properly. I must have taken at least three phone calls on Friday from clients who needed help getting their Blackberries setup with their corporate email accounts. Two of them were using BES (Blackberry Enterprise Server) and one was using BIS (Blackberry Internet Service).

Blackberry Enterprise Server
If you’re lucky enough to have BES, setting up a new Blackberry is usually a piece of cake. All you have to do is add the user to the Blackberry Enterprise Manager, set an activation password and activate. I usually choose to do a wireless activation because it’s easier to do over the phone. Once you’ve created the password, tell the user to go to Options…Advanced Options…Enterprise Activation. Give them the password that you’ve setup for them and hit Activate. That’s it. Here’s an overview of how the process works.

Here’s a couple of things to watch out for:

1. Make sure they have a good signal. If their signal is spotty, it might take an excessively long time or it might not work at all. The amount of time it takes depends on how good their signal is and how much data they’re trying to sync. I’ve seen it take anywhere from 5-20 minutes.

2. Make sure they have the correct data plan.Phone carriers often charge more for a BES account than they do for a BIS account. If the user doesn’t have the higher-priced BES data plan, the carrier may not allow you to do an enterprise activation. Don’t make the same mistake I did. Call the carrier to confirm the data plan before you spend 2 hours trying to troubleshoot it on your own.

3. Not-so-helpful error messages. If the activation fails with “An error has occurred. Please contact your system administrator” there might be a problem with the password. Reset the password and have them try again. It takes awhile to get used to the Blackberry keyboard so choose a simple password that is easy to type on a small keyboard (e.g. all lower case, no numbers or symbols). This breaks all the normal rules of a good password, but it’s only a one-time activation password. After that, it’s never used again. You might also want to try wiping the handheld and starting fresh.

Blackberry Internet Service
So your company wouldn’t spring for the Blackberry Enterprise Software? Not to worry. Your carrier provides you with something called Blackberry Internet Service (BIS). It’s a web-based application that allows you to setup an email account and control how messages are delivered. It’s actually a service provided by RIM, but each carrier has a different URL for accesing it (for branding of course). Here’s a couple of examples

Verizon: https://bis.na.blackberry.com/html?brand=vzw
Sprint: https://bis.na.blackberry.com/html?brand=sprint
AT&T: http://bis.na.blackberry.com/html?brand=mycingular
Centennial: http://centennial.blackberry.com/

If you can’t find your carrier, go to Google and type “BIS” and your carrier’s name in the search box. It will probably be one of the first hits that comes up.

If you’ve never signed up, click “Create New Account” You will be prompted for your phone’s PIN number and ESN number. Both can be found on the Options…Status screen. Once you’ve created an account on the BIS site, you will be able to create a Blackberry email address for your device. This will be something like joe123@vzw.blackberry.net. It doesn’t really matter what the address is because once it’s set up, you simply forward your corporate email account to the Blackberry account. So it doesn’t matter if you’re using Exchange, Lotus Notes, Groupwise or some other random mail server. As long as you can forward mail, you’re in business. While you’re still logged in to the BIS website, make sure you edit the properties of your mail account and set the “reply-to” address to be your corporate email address. That way when you send a message from your Blackberry, the recipient will see a recognizable address rather than your generic Blackberry address.

That’s it. Note that BIS is also capable of downloading your mail directly from your corporate mail server using POP3 or IMAP. However, that can be tricky to get working and your mileage may vary. There is a wizard on the BIS website that attempts to auto-detect your mail server settings, but it doesn’t always work and you may have to specify the settings manually. I find it’s much simpler and easier to just forward messages to the Blackberry account. Have fun!

Here’s a tip. Occasionally you may find that the Windows Control Panel will not open, or it may open and then close immediatly. Usually, the solution is simple. Windows stores the control panel icons as individual files with a .cpl file extension. These files are located in the C:\Windows\System32 folder. Do a quick search for *.cpl files and you will see names such as the following:

appwiz.cpl
bthprops.cpl
collab.cpl
desk.cpl
Firewall.cpl
hdwwiz.cpl
inetcpl.cpl
infocardcpl.cpl
intl.cpl
irprops.cpl
joy.cpl
main.cpl
mmsys.cpl
ncpa.cpl
powercfg.cpl
sysdm.cpl
TabletPC.cpl
telephon.cpl
timedate.cpl
wscui.cpl

This is the standard set of control panel icons for Windows Vista. However, many software applications will also install a control panel icon so you may see files that aren’t listed here. If so, those are the ones to focus on first. Try renaming them one by one, until you figure out which one is causing the problem. For example, Firebird is known to cause this problem, so if you see a file called Firebird.cpl, try renaming it to Firebird.cpl.old and then try opening Control Panel. If it works, then you’ve found the culprit. If not, rename it back and try renaming a different file. Once you’ve pinned it down, either reinstall the program it’s associated with, or just leave the file renamed and you’re all set!

Many things in your computer depend on having the correct date & time. Fortunately, there are network time servers all over the Internet that you can use to keep your computer’s clock in perfect sync. The trick is knowing which time server to use and how to configure the setting. The Network Time Protocol, or NTP, is the protocol your computer uses to communicate with other time servers across the network. You may have an NTP server on your local network but if not, you can use one of the many NTP servers on the Internet.

Step 1
First, open your date & time properties and make sure you are on the correct time zone. The NTP protocol will adjust your computer’s time, but not the time zone. You have to enter that manually. Also, if your location observes daylight savings time, make sure you check the box as appropriate.

Step 2
If you have Windows XP and are NOT part of a domain you will see an “Internet Time” tab. This is where you set the NTP server you want to sync with. The default time server, time.windows.com, is notoriously unreliable. Perhaps because every Windows PC built in the last 7 or 8 years is set to synchronize with that server. Instead, the time server I use is  north-america.pool.ntp.org. This is not really a time server at all. In fact it is a whole pool of time servers in North America that have agreed to let people use them. If you are outside North America, you can find a pool in your region by visiting http://support.ntp.org/bin/view/Servers/NTPPoolServers

Step 3
Now all you have to do is click the “Update Now” button to test it. You should see a message that the time has been successfully synchronized. If not, check that you entered the correct server name and try again. Also, make sure your clock is set to the correct date. If your time if off by more than 15 hours, it won’t work. Once you have it configured correctly, your clock will be synchronized automatically with a time server once a week.

Synchronizing Time on a Domain
If you don’t see an Internet Time tab, your computer might be part of an Active Directory domain. In that case, your computer will receive it’s time from a domain controller. In that case, all you need to do is make sure your domain controller has the correct time. To do this, log on to your domain controller as an administrative user and execute the following commands:

For Windows 2003 Server

w32tm /config /manualpeerlist:north-america.pool.ntp.org,0×8 /syncfromflags:MANUAL
w32tm /config /update
net stop w32time
net start w32time
w32tm /resync

For Windows 2000 Server

net time /setsntp:north-america.pool.ntp.org
net time /querysntp
net stop w32time
net start w32time
w32tm -s

Once your server has obtained the correct time, the rest of your client PC’s will follow suit.

Today I went out to a job to fix a problem with installing the Citrix XenApp Web Plugin. The user had downloaded the msi file from Citrix but shortly after starting the installation an following error appeared:

Installation Error – Invalid Drive H:\

Of course, there was no H: drive on the system. I tried downloading the .exe version of the installer, but no luck. Then I did a quick search of the registry and discovered the following registry key:

HKCU\Software\Microsoft\Windows\Current Version\Explorer\User Shell Folders\

The “My Pictures” key was set to “H:\My Pictures”. I changed it to the default for Windows XP which is:

%USERPROFILE%\My Documents\My Pictures

I also changed the type from REG_SZ to REG_EXPAND_SZ. I tried running the Citrix Installer again and voila, it worked! I have no idea why the Citrix installer was tring to access the My Pictures folder but if it works, so be it!

If you have more than one computer that you use on a regular basis, it seems like the file you need is always on the other computer. Sure, you can use a flash drive but if that’s still too much work, you need Dropbox! I use it all the time to sync files between my work computer, laptop and home computer. Check it out, it’s free!

http://www.getdropbox.com/

One of the questions I get asked the most is “How do I backup my PC?” Of couse, there’s a million ways to backup data but few are easy, reliable, or cheap. This might be the answer to all three of those problems. All you need is a high-speed Internet connection, and a friend.

http://www.cucku.com/