Today I had an issue escalated to me where the following Exchange 2003 services would not start due to problems speaking with Active Directory:
- Microsoft Exchange Information Store
- Microsoft Exchange MTA Stacks
- Microsoft Exchange System Attendant
While diagnosing this issue I had the following diagnostic logging turned to medium for all sub components of the following:
- MSExchangeDSAccess
- MSExchangeSRS
- MSExchangeSA
Diagnostic logging is turned on in the Exchange 2003 server properties under the diagnostic logging tab.
There were two problems causing this issue - I will go through how I fixed both of them.
Problem 1 - The Local DC was not listening on TCP 3268
After looking at the issue initially I found out that the domain controller in the same active directory site as Exchange 2003 server was not listening on TCP 3268 (Global Catalog). The server was marked as being a global catalog server. During this time the following errors were being generated.
Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7024
Date: 01/11/2010
Time: 4:03:31 PM
User: N/A
Computer: MELB-EXCH-31
Description:
The Microsoft Exchange Information Store service terminated with service-specific error 2147500037 (0x80004005).
Event Type: Error
Event Source: MSExchangeMTA
Event Category: Operating System
Event ID: 2248
Date: 01/11/2010
Time: 5:18:22 PM
User: N/A
Computer: MELB-EXCH-31
Description:
A fatal error occurred. Directory operation (ds_initialize) failed with problem RD Server. [MAIN BASE 1 1 %5] (16)
Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 1121
Date: 01/11/2010
Time: 6:14:38 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Error 0xfaf connecting to the Microsoft Active Directory.
This last error lead me down the right track, "0xfaf". I found a handy forum post that lets you understand all the different codes in relation to the information store connecting to Active Directory:
- Error: 0x96e, Service: Microsoft Active Directory - This problem occurs because the domain controller and other Exchange-Server-dependent services do not start completely when Exchange 2007 tries to start. See 940845 for details.
- Error: 0x8004010f, Service: Microsoft Active Directory - This behavior may occur if the organization name that you select during setup contains the forward slash mark (/) character. See 329599 for more information on this problem.
- Error: 4015, Service: Microsoft Exchange Information Server Directory - The Directory service will not start if the system date is later than January 17, 2038. See 154595 for details about this problem.
- Error: 0x80004005, Service: Microsoft Exchange Server Directory - This behavior may occur because the information store database is not initializing properly. See 322315 to solve this problem.
- Error: 0x80004005, Service: Microsoft Active Directory - You may not be able to mount Exchange 2000 information store databases and this event is logged. See 314294 and 822579 for details.
- Error: 0xfb5, Service: Microsoft Exchange Server Directory - This issue can occur if the Information Store service has been configured to log on as a system account or to use the Local System account. See 288952 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
As per the post above - I went to Microsoft Knowledgebase article 823163:
http://support.microsoft.com/kb/823163
This article stated:
This issue may occur if one or both of the following conditions are true:
- There is no global catalog in the forest.
- There is a problem with the connection to the global catalog.
There was only a single domain controller in the same Active Directory site as the Exchange 2003 server. This domain controller was marked as being a global catalog server however performing a "netstat -ant" revealed that it was not listening on the global catalog port TCP 3268.
Initially I tried removing the global catalog role of the server rebooting, re-adding the global catalog role then rebooting again. This did not resolve the problem.
To resolve this problem I demoted the server from being a domain controller and re-promoted it. After re-promoting the DC it then came up as a global catalog server again. There were no errors in the event logs about not being a global catalog server.
Problem 2 - The Exchange 2003 Server could not find the Active Directory Site
After re-promoting the domain controller I ran into another problem, the Exchange 2003 server could not find which Active Directory Site it was in.
Exchange was producing the following error:
c007077f no site available
The following errors were experienced in the event logs:
Event Type: Error
Event Source: MSExchangeDSAccess
Event Category: Topology
Event ID: 2114
Date: 01/11/2010
Time: 9:55:46 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Process MAD.EXE (PID=1936). Topology Discovery failed, error 0x80040952.
I ran NLTEST /DSGETSITE to see if windows server itself could find its active directory site. This command threw out the following error:
1919 0x77f ERROR_NO_SITENAME
The exchange server was speaking to random domain controllers in other Active Directory sites. I determined this by following this article:
http://clintboessen.blogspot.com/2010/05/how-to-find-out-which-domain-controller.html
All other workstations and servers in the same Active Directory site could determine its site location. The problem was isolated to the Exchange 2003 server. To resolve this problem I found a workaround with a registry key where I hard coded the Exchange 2003 server's Active Directory site.
HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
"SiteName" as a String Value
Enter the active directory site name as it appears in AD Sites and Services.
Then restart the Netlogon service. After this the Exchange Server 2003 was behaving properly and the services were able to be started.
- Microsoft Exchange Information Store
- Microsoft Exchange MTA Stacks
- Microsoft Exchange System Attendant
While diagnosing this issue I had the following diagnostic logging turned to medium for all sub components of the following:
- MSExchangeDSAccess
- MSExchangeSRS
- MSExchangeSA
Diagnostic logging is turned on in the Exchange 2003 server properties under the diagnostic logging tab.
There were two problems causing this issue - I will go through how I fixed both of them.
Problem 1 - The Local DC was not listening on TCP 3268
After looking at the issue initially I found out that the domain controller in the same active directory site as Exchange 2003 server was not listening on TCP 3268 (Global Catalog). The server was marked as being a global catalog server. During this time the following errors were being generated.
Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7024
Date: 01/11/2010
Time: 4:03:31 PM
User: N/A
Computer: MELB-EXCH-31
Description:
The Microsoft Exchange Information Store service terminated with service-specific error 2147500037 (0x80004005).
Event Type: Error
Event Source: MSExchangeMTA
Event Category: Operating System
Event ID: 2248
Date: 01/11/2010
Time: 5:18:22 PM
User: N/A
Computer: MELB-EXCH-31
Description:
A fatal error occurred. Directory operation (ds_initialize) failed with problem RD Server. [MAIN BASE 1 1 %5] (16)
Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 1121
Date: 01/11/2010
Time: 6:14:38 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Error 0xfaf connecting to the Microsoft Active Directory.
This last error lead me down the right track, "0xfaf". I found a handy forum post that lets you understand all the different codes in relation to the information store connecting to Active Directory:
- Error: 0x96e, Service: Microsoft Active Directory - This problem occurs because the domain controller and other Exchange-Server-dependent services do not start completely when Exchange 2007 tries to start. See 940845 for details.
- Error: 0x8004010f, Service: Microsoft Active Directory - This behavior may occur if the organization name that you select during setup contains the forward slash mark (/) character. See 329599 for more information on this problem.
- Error: 4015, Service: Microsoft Exchange Information Server Directory - The Directory service will not start if the system date is later than January 17, 2038. See 154595 for details about this problem.
- Error: 0x80004005, Service: Microsoft Exchange Server Directory - This behavior may occur because the information store database is not initializing properly. See 322315 to solve this problem.
- Error: 0x80004005, Service: Microsoft Active Directory - You may not be able to mount Exchange 2000 information store databases and this event is logged. See 314294 and 822579 for details.
- Error: 0xfb5, Service: Microsoft Exchange Server Directory - This issue can occur if the Information Store service has been configured to log on as a system account or to use the Local System account. See 288952 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
As per the post above - I went to Microsoft Knowledgebase article 823163:
http://support.microsoft.com/kb/823163
This article stated:
This issue may occur if one or both of the following conditions are true:
- There is no global catalog in the forest.
- There is a problem with the connection to the global catalog.
There was only a single domain controller in the same Active Directory site as the Exchange 2003 server. This domain controller was marked as being a global catalog server however performing a "netstat -ant" revealed that it was not listening on the global catalog port TCP 3268.
Initially I tried removing the global catalog role of the server rebooting, re-adding the global catalog role then rebooting again. This did not resolve the problem.
To resolve this problem I demoted the server from being a domain controller and re-promoted it. After re-promoting the DC it then came up as a global catalog server again. There were no errors in the event logs about not being a global catalog server.
Problem 2 - The Exchange 2003 Server could not find the Active Directory Site
After re-promoting the domain controller I ran into another problem, the Exchange 2003 server could not find which Active Directory Site it was in.
Exchange was producing the following error:
c007077f no site available
The following errors were experienced in the event logs:
Event Type: Error
Event Source: MSExchangeDSAccess
Event Category: Topology
Event ID: 2114
Date: 01/11/2010
Time: 9:55:46 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Process MAD.EXE (PID=1936). Topology Discovery failed, error 0x80040952.
I ran NLTEST /DSGETSITE to see if windows server itself could find its active directory site. This command threw out the following error:
1919 0x77f ERROR_NO_SITENAME
The exchange server was speaking to random domain controllers in other Active Directory sites. I determined this by following this article:
http://clintboessen.blogspot.com/2010/05/how-to-find-out-which-domain-controller.html
All other workstations and servers in the same Active Directory site could determine its site location. The problem was isolated to the Exchange 2003 server. To resolve this problem I found a workaround with a registry key where I hard coded the Exchange 2003 server's Active Directory site.
HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
"SiteName" as a String Value
Enter the active directory site name as it appears in AD Sites and Services.
Then restart the Netlogon service. After this the Exchange Server 2003 was behaving properly and the services were able to be started.
No comments:
Post a Comment