I had a problem in my lab environment yesterday when trying to fix a corrupted search catalog on one of my Exchange 2010 mailbox servers. I knew the search catalog was corrupted because I was seeing the following events:
Log Name: Application
Source: ExchangeStoreDB
Date: 21/06/2010 12:02:03 p.m.
Event ID: 123
Task Category: Database recovery
Level: Error
Keywords: Classic
User: N/A
Computer: MXB02.contoso.com
Description:
At ’21/06/2010 12:02:01 p.m.’ the Microsoft Exchange Information Store Database ‘DB01’ copy on this server experienced a corrupted search catalog. Consult the event log on the server for other “ExchangeStoreDb” and “MSExchange Search Indexer” events for more specific information about the failure. Reseeding the catalog is recommended via the ‘Update-MailboxDatabaseCopy’ task.
I knew I had a good copy of the database on my other server (MBX01), so I went ahead and issued the following Powershell command:
Update-MailboxDatabaseCopy -Identity “DB01\MBX02” -CatalogOnly
This generated a nasty looking erorr, which corresponded to the following Application event log entry:
Log Name: Application
Source: MSExchange Configuration Cmdlet – Remote Management
Date: 21/06/2010 4:54:08 p.m.
Event ID: 4
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: MBX02.contoso.com
Description:
(PID 9940, Thread 21) Task Update-MailboxDatabaseCopy writing error when processing record of index 0. Error: Microsoft.Exchange.Cluster.Replay.CiSeederGenericException: A server-side seed operation has failed. Error: An error occurred while performing the seed operation, which may indicate a problem with the source disk. Error: An error occurred while updating the search catalog files from server ‘MBX01’ to ‘MBX02’. Error: Can’t dismount the search catalog. Error: Microsoft.Exchange.Search.Common.FteCatalogNotFoundException: SearchCatalog.Dismount failed, error 0x80043629 —> System.ComponentModel.Win32Exception: Unknown error (0x80043629)
— End of inner exception stack trace —
at Microsoft.Exchange.Cluster.Replay.CiFilesSeederInstance.<>c__DisplayClass5.<SeedThreadProcInternal>b__2(Object , EventArgs )
at Microsoft.Exchange.Cluster.Replay.CiFilesSeederInstance.RetryCiOperation(EventHandler evt) —> Microsoft.Exchange.Search.Common.FteCatalogNotFoundException: SearchCatalog.Dismount failed, error 0x80043629 —> System.ComponentModel.Win32Exception: Unknown error (0x80043629)
— End of inner exception stack trace —
at Microsoft.Exchange.Cluster.Replay.CiFilesSeederInstance.<>c__DisplayClass5.<SeedThreadProcInternal>b__2(Object , EventArgs )
at Microsoft.Exchange.Cluster.Replay.CiFilesSeederInstance.RetryCiOperation(EventHandler evt)
— End of inner exception stack trace (Microsoft.Exchange.Search.Common.FteCatalogNotFoundException) —
at Microsoft.Exchange.Cluster.Replay.CiFilesSeederInstance.SeedThreadProcInternal()
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
— End of stack trace on server (MBX02.contoso.com) —
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
at Microsoft.Exchange.Cluster.Replay.SeedProgressReporter.GetException()
Event Xml:
After some fruitless retries and even more fruitless Googling, I finally found a Technet article that indicates I should have suspended the database copy before running the Update-MailboxDatabaseCopy command. Sure enough, once I suspended the database copy I was able to run the command successfully.
Interestingly (and somewhat worryingly) the corrupted search index appeared immediately after an attempt to perform a server switchover. If I didn’t know better I would think that the switchover generated the corruption. 😉
Hopefully this will help others who run into the same problem.