It was Friday around 3PM and most people were already gone for the day. One of the few employees left came by my desk and asked if something was up with the Exchange server. I checked Outlook – it said it was connected. Shift + F9 to force a reconnection and it failed out. Error 0x80004005. Last email received was 30 minutes ago. Great.

So error 0x80004005 says this:

The client operation failed. Microsoft Exchange Information Store.

Log onto the Exchange server and verify the services are all running. They all were running.
Check the event viewer log files – way too many errors to go through all of them while a production server was down, but this one stood out:

Event viewer showed this error:

A transient failure has occurred. The problem may resolve itself in a while. The service will retry in 56 seconds. Diagnostic information:
Cannot open mailbox /o=blah/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=blah/cn=Microsoft System Attendant.
Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException: Cannot open mailbox /o=blah/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=blah/cn=Microsoft System Attendant. —> Microsoft.Mapi.MapiExceptionLogonFailed: MapiExceptionLogonFailed: Unable to open message store. (hr=0x80040111)

Restart the Information Store service.
Now I get this error in the event viewer:

MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=1276)

Crap. I verify that we’re running Exchange enterprise and not standard (database store limits). I check the management console and see that the database is offline and refuses to mount. Since the database was already offline, I decided to run a repair on the DB.

Open a command prompt:
eseutil /p "C:\program files\microsoft\exchange server\mailbox\first storage group\mailbox database.edb"
This took roughly 30 minutes on a 44GB database.

Try to mount the database again. It fails but gives me a different error this time:

Exchange is unable to mount the database that you specified. Specified database: Exchange07\Storage Group 1\Mailstore 1; Error code: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=-515)

This means that I’m missing a required transaction log and that the DB won’t start without it. Great.

So I navigate to the first storage group log files (C:\program files\microsoft\exchange server\mailbox\first storage group)
I moved all of the files (excluding Mailbox Databse.edb and CatalogData-* folder) to another directory. Then I attempted to mount the DB again. Success!

Total downtime: 75 minutes
Active working time: 45 minutes

So what happened?
McAfee Antivirus grabbed a file from the transaction log before it could be written to the database. Since the transaction log was altered while it was trying to write to the database, the mailstore became slightly corrupt and required a repair. And because the transaction logs and the database were at different parts, even after the repair, the log files needed to be “destroyed” in order for the DB to be mounted again. Oh, the logs were recreated after successfully mounting the database.

I enabled Outlook Anywhere on the Primary external-facing Exchange server. But then some of the employees were complaining that whenever they would connect to exchange using Outlook that it would ask for their credentials. I chalked it up to being a permissions issue, but apparently the old Ex03 server was somehow still in the mix.

Open the registry editor on your Outlook Anywhere machine and checking the following key:
HKLM\SOFTWARE\Microsoft\Rpc\RpcProxy\ValidPorts
Make sure that you only see the name of your exchange server and the ports 6001, 6002, 6004 for the servername and servername.domain.tld.
I noticed that the old server was still listed in there. I deleted it, and then restarted the Microsoft Exchange Service Host to complete the task. Unfortunately the old server was put back in.

So then I checked ASDIedit and saw no traces of the old exchange server in the mix. Odd. So why was the old server still showing up in the RPC registry keys?

I had to load up IIS/Adminpak.msi on an XP machine to get Exchange 2003 System Manager to install (ESM). Loaded that up and under Admin Groups I could clearly see the old Exchange server. Booo.
Right-click, delete. *poof*.

Redo the registry edit steps and restart the service host service and bam, working again. Now no more complaints.

Exchange 2007 is pretty nice. They made Outlook Anywhere (RPC over HTTPS) pretty darn easy to setup – assuming you have an SSL certificate. Unfortunately (or fortunately depending on how you look at things) a lot of the commands must be performed using the command line PowerShell application. Good thing the PowerShell and Exchange 2007 allows for more granular permissions with Outlook Anywhere.

One such command is to allow/disallow Outlook Anywhere per user. By default Exchange allows all authenticated users to connect via Outlook Anywhere. There’s no nice way using the GUI to disable access – like there is for POP3/IMAP/MAPI/etc – so you’ll have to fire up the PowerShell.

Want to check the current settings of your user?
get-mailbox USERNAME | get-casmailbox | fl
That will fully list the CAS settings for that mailbox. Look under MAPIBLOCKOUTLOOKRPCHTTP. It’s probably set to “false”. If you want to block that user from accessing Outlook Anywhere:
get-mailbox USERNAME | set-casmailbox -mapiblockoutlookrpchttp $true

That’s it.

I’ve done plenty of new self-signed certificates for Exchange. Most places don’t mind if the certificate displays an error when users visit the webmail site (OWA), but they do mind if the users receive an error saying the certificate name is invalid when using Outlook.

Had the self-signed certificate installed on a standard Exchange 2007 server. CRM 4 requires an SSL/TLS connection. While we could have created another internal certificate with the export = $true key, the customer also wanted to rid themselves from the invalid certificate when browsing to the Outlook Web Access site.

Obviously replace “domain.tld” with your actual information.

Create the certificate request:

Open PowerShell on Exchange
New-ExchangeCertificate -DomainName webmail.domain.tld,other.domain.tld,autodiscover.domain.tld -FriendlyName "Site Webmail Certificate" -GenerateRequest:$True -Keysize 2048 -path c:\Webmailcertificate.txt -privatekeyExportable:$true -subjectName "c=US, o=CompanyName Inc., OU=IT, L=City, S=State, CN=webmail.domain.tld"

Purchase the site certificate:

Go to your favorite SSL supplier (Verisign, Thawte, etc.) and purchase an SSL Certificate. Standard is fine for this mostly internal-only site.
Paste the code from c:\Webmailcertificate.txt when applicable
After the certificate has been authorized, download the .crt certificate and the intermediary Certificate Authority files

Install your certificate:

Back on PowerShell for Exchange
Import-ExchangeCertificate -path c:\webmailcertificate.txt
Get-ExchangeCertificate
Copy the Thumbprint from the NEW certificate (probably the one with “…..” listed under Services
Enable-ExchangeCertificate -Services IMAP, POP, UM, IIS, SMTP -Thumbprint 896B74B2YourExchangeThumbprintFC6A7
Click Y for Yes if prompted to replace from an old(er) certificate

Now your webmail access (OWA) should no longer have a certificate issue. However, if the issued name on the certificate is DIFFERENT from your NETBIOS name of your email server, you will have issues INTERNALLY. Namely, all of your outlook clients will report a certificate is invalid error – that the names do not match. This is because the Exchange Server now has the certificate that points to webmail.domain.tld and your outlook clients are pointing to exchange07.domain.local.

To fix this issue:

Once again, use PowerShell for Exchange
Get-ClientAccessServer
Copy the servername
Set-ClientAccessServer -Identity SERVERNAMEHERE -AutodiscoverServiceInternalUri https://webmail.domain.tld/autodiscover/autodiscover.xml
Set-WebServicesVirtualDirectory -Identity "SERVERNAMEHERE\EWS (Default Web Site)" -InternalUrl https://webmail.domain.tld/ews/exchange.asmx
Set-OABVirtualDirectory -Identity "SERVERNAMEHERE\oab (Default Web Site)" -InternalUrl https://webmail.domain.tld/oab
There is one final step required – recycle the MSExchangeAutodiscoverAppPool:
On Exchange 2007, Open IIS Manager
Navigate to Local Computer > Application Pools
Right-Click on MSExchangeAutodiscoverAppPool and select Recycle

That should be it. Everything works here after recycling.

Or you could always just put in the required domains on your certificate request:
NetBIOS name
FQDN external (if different)
autodiscover.domain.tld
autodiscover.domain.local (if applicable)
webmail.domain.tld (obviously change accordingly)