This is pertaining to an issue I have been seeing in my setup which has 4 virtualized Exchange 2013 Nodes (Multi-role) on Win2012 Servers.
After DAG creation, as I attempt to join additional hosts to the DAG (after the first server has joined the DAG),the operation fails.
Even though the EAC gives me the message that operation completed successfully but actually it has failed (see the attached logs: first attachment details the addition of first Exchange server in the DAG; second attachment describes the failed operation when adding second node) .
No matter which server I start with (all 4 Exchange servers have full permissions on the AD’s CNO (pre-staged, disabled) for DAG, still the DAG is unable to join any additional hosts after the first host. DAG Cluster core resources (Cluster name, Cluster networks etc.) are online on the first server (which joins the DAG cluster) & ping-able by name so DNS resolution is not the issue. Everyone can ping everyone else. But still, I suspect it has to do with networking.
I have created DAG in virtualized Exchange 2013 setup several times earlier but it always with 2 vNIC’s only.
But, in my current Exchnage servers, we have3 vNIC’s on the Exchange Servers ( a separate management network is mandatory in my architecture) Here’s is the setting I am using on my 3 vNIC’s. There is only 1 default gateway (with the MAPI network). The Mgmt. has no default gateway. Network bindings, priorities etc. are all correct.
Network | Client for MS Networks | QoS Packet Scheduler | File & Print Sharing | IPv6 | IPv4 | Link-Layer Mapper I/O Driver | Link Layer Responder |
MAPI | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled |
Replication | Disabled | Enabled | Disabled | Enabled | Enabled | Enabled | Enabled |
Management | Disabled | Enabled | Disabled | Enabled | Enabled | Enabled | Enabled |
DAG adds the “Management network” as a DagReplicationNetwork02; to get the DAG to ignore the Management Network, I do this: Set-DatabaseAvailabilityGroupNetwork -Identity DAG2\ DagReplicationNetwork02 -ReplicationEnabled:$false -IgnoreNetwork:$trueOperation succeeds but DAG still sees 3 networks.I try deleting the DagReplicationNetwork02 (Mgmt. Network) from EAC console after deleting its subnet but does not works even after rebooting. This I think is crux of issue. I believe, there should be only 2 DAG networks for a single DAG in its networking. While google-ing, I found this :http://social.technet.microsoft.com/Forums/en-US/exchangesvrdeploy/thread/0734bd9e-3c42-4199-9171-312c8684b603
Where someone has posted: There is a bug in MS network teaming software. The solution is toeither break Teaming or removing the "offloading" on the network adapter and the Team.
&, On similar lines:http://buenoflex.com/archives/322. It says that:
teaming of windows server 2012 is not really compatible with the Failover Cluster service. The teaming is (by default) in “Switch independent” and the algorithm used is“Hash Address”. This algorithm creates a hash based on components of the packet, and then it assigns packets that have that hash value to one of the available network adapters.
In Thor 2.0, we are using Hyper-V Port switch independent teaming.
Below are 2 extracts from the DAG creation logs where the issue could be traced. First error is :fully qualified domain name for node 'DAG2' could not be found.
But, in the AD, the “Attribute Editor” property has been used for the CNO to set its DNS hostname to its FQDN (final error is: Operation timed out). See below:
Second, error is operation timed out. Could you suggest any pointers as to where the issue might be? Thanks in advance.
DAG DAG2 has 1 servers:
[2013-05-20T10:44:58] DAG DAG2 contains server EXCHANGE-NODE1.
[2013-05-20T10:44:58] Updated Progress 'Checking if Mailbox server 'EXCHANGE-NODE2' is in a database availability group.' 4%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:44:58] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(EXCHANGE-NODE2.MSCollab.lab) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed.. This is OK.
[2013-05-20T10:44:58] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-05-20T10:44:58] DumpClusterTopology: Opening remote cluster DAG2.
[2013-05-20T10:44:58] DumpClusterTopology: Failed opening withMicrosoft.Exchange.Cluster.Replay.AmGetFqdnFailedNotFoundException: An Active Manager operation failed with a transient error. Please retry the operation. Error: The fully qualified domain name for node 'DAG2' could not be found.
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.<>c__DisplayClass1.<ResolveFqdn>b__0(Object param0, EventArgs param1)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmHelper.RunADOperationEx(EventHandler ev)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.ResolveFqdn(String shortNodeName, Boolean throwException)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.GetFqdn(String shortNodeName, Boolean throwException)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerName..ctor(String serverName, Boolean throwOnFqdnError)
at Microsoft.Exchange.Management.SystemConfigurationTasks.DumpClusterTopology..ctor(String nameCluster, ITaskOutputHelper output)
[2013-05-20T10:44:58] Dumping the cluster by connecting to: DAG2.
[2013-05-20T10:44:58] DumpClusterTopology: Not dumping, since the cluster could not be contacted.
[2013-05-20T10:47:59] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = EXCHANGE-NODE2, dwStatus = 0x5b4 )
[2013-05-20T10:47:59] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:51:17] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:51:17] The preceding log entry comes from a different process running on computer 'Exchange-Node1.MSCollab.lab'. END
[2013-05-20T10:51:17] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-05-20_10-44-58.519_add-databaseavailabiltygroupserver.log".
[2013-05-20T10:51:17]WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed.. ---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
--- End of inner exception stack trace ---
=========================================================================
Please do let me know if any other info is needed.
Full logs are here:
add-databaseavailabiltygroupserver started on machine EXCHANGE-NODE1.[2013-05-20T10:44:58] add-dagserver started
[2013-05-20T10:44:58] commandline: $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-05-20T10:44:58] Option 'Identity' = 'DAG2'.
[2013-05-20T10:44:58] Option 'MailboxServer' = 'EXCHANGE-NODE2'.
[2013-05-20T10:44:58] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-05-20T10:44:58] Option 'WhatIf' = ''.
[2013-05-20T10:44:58] Process: w3wp w3wp.exe:9756.
[2013-05-20T10:44:58] User context = 'NT AUTHORITY\SYSTEM'.
[2013-05-20T10:44:58] Member of group 'Everyone'.
[2013-05-20T10:44:58] Member of group 'BUILTIN\Users'.
[2013-05-20T10:44:58] Member of group 'NT AUTHORITY\SERVICE'.
[2013-05-20T10:44:58] Member of group 'CONSOLE LOGON'.
[2013-05-20T10:44:58] Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-05-20T10:44:58] Member of group 'NT AUTHORITY\This Organization'.
[2013-05-20T10:44:58] Member of group 'BUILTIN\IIS_IUSRS'.
[2013-05-20T10:44:58] Member of group 'LOCAL'.
[2013-05-20T10:44:58] Member of group 'IIS APPPOOL\MSExchangeECPAppPool'.
[2013-05-20T10:44:58] Member of group 'BUILTIN\Administrators'.
[2013-05-20T10:44:58] Updated Progress 'Validating the parameters.' 2%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:44:58] Mailbox server: value passed in = EXCHANGE-NODE2, mailboxServer.Name = EXCHANGE-NODE2, mailboxServer.Fqdn = EXCHANGE-NODE2.MSCollab.lab
[2013-05-20T10:44:58] LogClussvcState: clussvc is Stopped on EXCHANGE-NODE2.MSCollab.lab. Exception (if any) = none
[2013-05-20T10:44:58] The IP addresses for the DAG are (blank means DHCP): 100.10.10.50
[2013-05-20T10:44:58] Looking up IP addresses for DAG2.
[2013-05-20T10:44:58] DAG2 = [ 100.10.10.50 ].
[2013-05-20T10:44:58] Looking up IP addresses for exchange-node2.
[2013-05-20T10:44:58] exchange-node2 = [ 100.10.10.221 ].
[2013-05-20T10:44:58] Looking up IP addresses for EXCHANGE-NODE2.MSCollab.lab.
[2013-05-20T10:44:58] EXCHANGE-NODE2.MSCollab.lab = [ 100.10.10.221 ].
[2013-05-20T10:44:58] DAG DAG2 has 1 servers:
[2013-05-20T10:44:58] DAG DAG2 contains server EXCHANGE-NODE1.
[2013-05-20T10:44:58] Updated Progress 'Checking if Mailbox server 'EXCHANGE-NODE2' is in a database availability group.' 4%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:44:58] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(EXCHANGE-NODE2.MSCollab.lab) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed.. This is OK.
[2013-05-20T10:44:58] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-05-20T10:44:58] DumpClusterTopology: Opening remote cluster DAG2.
[2013-05-20T10:44:58] DumpClusterTopology: Failed opening with Microsoft.Exchange.Cluster.Replay.AmGetFqdnFailedNotFoundException: An Active Manager operation failed with a transient error. Please retry the operation. Error: The fully qualified domain name for node 'DAG2' could not be found.
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.<>c__DisplayClass1.<ResolveFqdn>b__0(Object param0, EventArgs param1)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmHelper.RunADOperationEx(EventHandler ev)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.ResolveFqdn(String shortNodeName, Boolean throwException)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerNameCache.GetFqdn(String shortNodeName, Boolean throwException)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmServerName..ctor(String serverName, Boolean throwOnFqdnError)
at Microsoft.Exchange.Management.SystemConfigurationTasks.DumpClusterTopology..ctor(String nameCluster, ITaskOutputHelper output)
[2013-05-20T10:44:58] Dumping the cluster by connecting to: DAG2.
[2013-05-20T10:44:58] DumpClusterTopology: Not dumping, since the cluster could not be contacted.
[2013-05-20T10:44:58] DumpClusterTopology: Opening local cluster.
[2013-05-20T10:44:58] Dumping the cluster by connecting to: .
[2013-05-20T10:44:58] The cluster's name is: DAG2.
[2013-05-20T10:44:58] Groups
[2013-05-20T10:44:58] group: Available Storage [not a CMS]
[2013-05-20T10:44:58] OwnerNode: Exchange-Node1.MSCollab.lab
[2013-05-20T10:44:58] State: Offline
[2013-05-20T10:44:58] group: Cluster Group [Cluster Main Group]
[2013-05-20T10:44:58] OwnerNode: Exchange-Node1.MSCollab.lab
[2013-05-20T10:44:58] State: Online
[2013-05-20T10:44:58] Resource: Cluster Name [Online, type = Network Name, PossibleOwners = EXCHANGE-NODE1 ]
[2013-05-20T10:44:58] NetName = [DAG2]
[2013-05-20T10:44:58] Resource: Cluster IP Address [Online, type = IP Address, PossibleOwners = EXCHANGE-NODE1 ]
[2013-05-20T10:44:58] Address = [100.10.10.50]
[2013-05-20T10:44:58] EnableDhcp = [0]
[2013-05-20T10:44:58] Network = [Cluster Network 1]
[2013-05-20T10:44:58] Nodes
[2013-05-20T10:44:58] node: Exchange-Node1.MSCollab.lab [ state = Up ]
[2013-05-20T10:44:58] Subnets
[2013-05-20T10:44:58] Name(Cluster Network 1), Mask(100.0.0.0/8), Role(ClusterNetworkRoleInternalAndClient)
[2013-05-20T10:44:58] NIC 100.10.10.220 on Node Exchange-Node1 in State=Up
[2013-05-20T10:44:58] Name(Cluster Network 2), Mask(110.0.0.0/8), Role(ClusterNetworkRoleInternalUse)
[2013-05-20T10:44:58] NIC 110.10.10.220 on Node Exchange-Node1 in State=Up
[2013-05-20T10:44:58] Name(Cluster Network 3), Mask(10.0.0.0/8), Role(ClusterNetworkRoleInternalUse)
[2013-05-20T10:44:58] NIC 10.10.10.220 on Node Exchange-Node1 in State=Up
[2013-05-20T10:44:58] Opening the cluster on nodes [exchange-node1].
[2013-05-20T10:44:58] Other mailbox servers in the DAG are already members of cluster 'DAG2'
[2013-05-20T10:44:58] The server EXCHANGE-NODE2 does not belong to a cluster, and the other servers belong to DAG2.
[2013-05-20T10:44:58] Successfully resolved the servers based on the stopped servers list.
[2013-05-20T10:44:58] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-05-20T10:44:58] The following servers are in the StoppedServers list:
[2013-05-20T10:44:58] Verifiying that the members of database availability group 'DAG2' are also members of the cluster.
[2013-05-20T10:44:58] Verifying that the members of cluster 'DAG2' are also members of the database availability group.
[2013-05-20T10:44:58] According to GetNodeClusterState(), the server EXCHANGE-NODE2 is NotConfigured.
[2013-05-20T10:44:58] The CNO is currently Online.
[2013-05-20T10:44:58] InternalValidate() done.
[2013-05-20T10:44:58] Updated Progress 'Adding server 'EXCHANGE-NODE2' to database availability group 'DAG2'.' 6%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:44:58] Updated Progress 'Adding server 'EXCHANGE-NODE2' to the cluster.' 8%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:51:17] The following log entry comes from a different process that's running on machine 'Exchange-Node1.MSCollab.lab'. BEGIN
[2013-05-20T10:51:17] [2013-05-20T10:44:58] Opening a local AmCluster handle.
[2013-05-20T10:44:58] Updated Progress 'Adding server 'exchange-node2' to database availability group 'DAG2'.' 2%.
[2013-05-20T10:44:58] Working
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = DAG2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = EXCHANGE-NODE2.MSCollab.lab, dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:44:58] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:47:59] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = EXCHANGE-NODE2, dwStatus = 0x5b4 )
[2013-05-20T10:47:59] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = EXCHANGE-NODE2, dwStatus = 0x0 )
[2013-05-20T10:51:17] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )
[2013-05-20T10:51:17] The preceding log entry comes from a different process running on computer 'Exchange-Node1.MSCollab.lab'. END
[2013-05-20T10:51:17] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-05-20_10-44-58.519_add-databaseavailabiltygroupserver.log".
[2013-05-20T10:51:17] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed.. ---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
--- End of inner exception stack trace ---
at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
--- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.AmClusterApiException) ---
at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
--- End of stack trace on server (Exchange-Node1.MSCollab.lab) ---
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2013-05-20T10:51:17] Updated Progress 'Done!' 100%.
[2013-05-20T10:51:17] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().
==
Regards, Rahul