Server Upgrade – NETLOGON – Service Issues

October 6, 2017 Leave a comment

Problem
Performing an upgrade to Server 2016 from 2012 R2 (perhaps 2012 or 2008 R2 as well).
A number of services do not work correctly (e.g. RD Gateway, Direct Access).
Found that the errors relate to “could not contact domain” or you see that the AD directories do not show up in various GUI’s. Cluster Services don’t process actions as expected and fail with an error.
nltest does not indicate issues.

Explanation
During the upgrade process the NETLOGON service is set to manual is not correctly set to Automatic afterward, as it should be.

Resolution
Set NETLOGON service to Automatic and reboot server.

Advertisements

SSL VPN access to TFS Server – Critical Error – SharePoint Foundation – 8321

October 6, 2017 Leave a comment

Problem/Error

User was SSL VPN connecting to our office and then attempting to access TFS Server website (WatchGuard OpenVPN on iPad via Home wireless/Telco, irrelevant for end result however). The page was timing out 90% of the time.

Didn’t make sense really so looked at Event Viewer on TFS server and the only error was the following;

Critical Error – SharePoint Foundation – 8321

A certificate validation operation took 14994.8386 milliseconds and has exceeded the execution time threshold. If this continues to occur, it may represent a configuration issue. Please see http://go.microsoft.com/fwlink/?LinkId=246987 for more details.

 

Solution

Install the SharePoint Root Authority certificate in the Trusted Root Certification Authorities store.

After the root certificate is added to the local certificate store, the certificate validation is no longer performed over the Internet.

The below steps will cause the BuildChain to succeed by finding the certificate in the local store, therefore eliminating the need for the retrieval of an object from the network.

The following steps have to be completed on each SharePoint server in the farm to add the root certificate to the local certificate store:

Export the SharePoint Root Authority certificate as a physical (.cer) file.

Start the SharePoint 2010 Management Shell as an Administrator,and then run the following Windows PowerShell commands:
$rootCert = (Get-SPCertificateAuthority).RootCertificate
$rootCert.Export(“Cert”) | Set-Content C:\SharePointRootAuthority.cer -Encoding byte
Note: This will export the internal root certificate (.cer file) for SharePoint to Drive C. You can copy and use this file on all servers in the farm for importing without having to run the PowerShell commands again.

Import the SharePoint Root Authority certificate to the Trusted Root Certification Authorities store.

  • To add the SharePoint Root Authority certificate to the Trusted Root Certification Authorities store, follow these steps:
    Note “Administrators” is the minimum required group membership to complete these steps.
  • Tap or click Start, type mmc in Start search, and then press Enter.
  • On the File menu, click Add/Remove Snap-in.
  • Under Available snap-ins, click Certificates, and then click Add.
  • Under This snap-in will always manage certificates for, select Computer account, and then click Next.
  • Select Local computer, and then click Finish.
  • If you have no more snap-ins to add to the console, click OK.
  • In the console tree, double-click Certificates.
  • Right-click the Trusted Root Certification Authorities store.
  • Click All Tasks, click Import to import the certificate, and then follow the steps in the Certificate Import Wizard.

Disable revocation check thru local group policy (gpedit.msc) – Computer Configuration -> Administrative Templates -> All Settings -> Check for serer certificate revocation.

The operation failed because: The attempt at remote directory server to remove directory was unsuccessful. “Access is denied.”

October 6, 2017 Leave a comment

Problem/Error

Attempting to demote a Domain controller with DCPROMO, receive the following error;

Network Credentials
The operation failed because: The attempt at remote directory server <Servername> to remove directory <servername to remove> was unsuccessful. “Access is denied.”

 

Solution

I came across this scenario in a lab environment. After using a local Administrator, i then tried the domain\administrator account. The problem still persists. I checked the dcpromoui.log and found the same error as above.

I then did the following checks;
Checked Active Directory Users and Computers for the Computer-Object (DC) if “Protect object from accidental deletion” is is set to the object – It wasn’t.
Checked Active Directory Sites and Services for the Computer-Object (DC) if “Protect object from accidental deletion” is set to the object and all sub-objects – It wasn’t.

So I checked the security-tab on the objects.
<domain.com>/Configuration/Sites/<SiteName>/Servers
<domain.com>/Configuration/Sites/<SiteName>/Servers/<DC>
<domain.com>/Configuration/Sites/<SiteName>/Servers/<DC>/NTDS Settings
On all those objects i found a strange security setting… Everyone had “Delete all child objects” on Deny. So i removed all the everyone permissions for the object servers, DC and NTDS settings.

After i did that, it worked.

It’s so strange, because nobody set this permission and it was just in one site on 2 of 3 DC’s. So as i see, the setting can be pretty random and removing the security setting was enough for getting the error away.

Error when attempting to run a published app via RDS website

March 29, 2017 Leave a comment

Problem
User reported an error when attempting to run a published app via our RDS Server website.
The error was;
Your computer can’t connect to the remote computer because an error occurred on the remote computer that you want to connect to. Contact your network administrator for assistance.

Explanation
Looking into the event viewer, at the Applications and Services Logs > Microsoft > Windows >TerminalServices-Gateway node found the following error.
The user “**DOMAIN\User**”, on client computer “**External_IP**”, was not authorized to connect to the RD Gateway server because a tunnel could not be created. The authentication method attempted: “NTLM” and connection protocol “HTTP”. The following error occurred: “2147965421”.

Resolution
I updated my SSL certificate in IIS but forgot to bind it / update the certificate in Server Manager under Remote Desktop Services.

WSUS – Error during Cleanup wizard -SQL Timeout

December 23, 2016 Leave a comment

Problem
When attempting to perform a WSUS Server Cleanup on the WSUS Server always getting a error and have to ‘Reset Server Node’.
Checking the Event Viewer showed the following error;
Log Name:      Application
Source:        Windows Server Update Services
Date:          23/12/2016 9:42:55 AM
Event ID:      7042
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      Server.domain.local
Description:
The WSUS administration console was unable to connect to the WSUS Server Database.
Verify that SQL server is running on the WSUS Server. If the problem persists, try restarting SQL.
System.Data.SqlClient.SqlException — Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated.
Source
.Net SqlClient Data Provider
Stack Trace:
at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
at System.Windows.Forms.Control.Invoke(Delegate method, Object[] args)
at Microsoft.UpdateServices.UI.SnapIn.Wizards.ServerCleanup.ServerCleanupWizard.OnCleanupComplete(Object sender, PerformCleanupCompletedEventArgs e)
Event Xml:

7042
3
0
0x80000000000000

122805
Application
Server.domain.local

The WSUS administration console was unable to connect to the WSUS Server Database.
Verify that SQL server is running on the WSUS Server. If the problem persists, try restarting SQL.
System.Data.SqlClient.SqlException — Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated.
Source
.Net SqlClient Data Provider
Stack Trace:
at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
at System.Windows.Forms.Control.Invoke(Delegate method, Object[] args)
at Microsoft.UpdateServices.UI.SnapIn.Wizards.ServerCleanup.ServerCleanupWizard.OnCleanupComplete(Object sender, PerformCleanupCompletedEventArgs e)
Resolution
I have found there are two ways to resolve this issue;
Firstly…
For those less experienced in SQL, here is the process:
1. Enable “named pipes” access for the Microsoft##WID database. Access wasn’t enabled on mine initially.
2. Open SSMS (SQL Server Management Studio and enter “\\.\pipe\MICROSOFT##WID\tsql\query” under server name.
3. Click “options” and select “Named Pipes” under “Network protocol” under the “Connection Properties” tab
4. Click “connect”
5. Execute a query for:
USE SUSDB
GO
exec spGetObsoleteUpdatesToCleanup
You will see a list of obsolete updates pop up. Take note of these UpdateID’s.
6. Execute “exec spDeleteUpdate @localUpdateID=000000” where 000000=UpdateID

Another option is to increase the timeout on the sql server.
Extend Timeout (f.e. for Cleanup-Wizard)
Connect to SQL via: Extend Timeout (f.e. for Cleanup-Wizard)
Connect to SQL via: \\\.\pipe\MICROSOFT##WID\tsql\query
Right click on \\.\pipe\MICROSOFT##WID\tsql\query (first in the row on the left)
Select Properties
Select Connections
Set timeout from default 600 to 1200 (or greater, avoid 0 as this is unlimited and not best practise).

Content Index Failed – Microsoft Exchange Search Service Constantly Crashing

December 21, 2016 Leave a comment

Problem

Single Microsoft Server 2012, Exchange 2013 server.
There are 4 Database Files (e.g. DB01, DB02, DB03, PFDB).

There was one Database showing the ContentIndexState as ‘FailedandSuspended’.
Administrator performed the following in an attempt to resolve the issue;
[PS] C:\>stop-service MSExchangeFastSearch
[PS] C:\>stop-service HostControllerService
Deleted the catalog index location folders.
[PS] C:\>start-service MSExchangeFastSearch
[PS] C:\>start-service HostControllerService
The catalog index location folders are recreated and the crawling begins.

Very shortly after the Microsoft Exchange Search service restarts and the following error is logged to the System Event Viewer log.
Event ID 7031, System, Service Control Manager
The Microsoft Exchange Search service terminated unexpectedly. It has done this 1 time(s).
The following corrective action will be taken in 5000 milliseconds: Restart the service.

In the Application Event Viewer log the following warning and error are logged also. Unsure of the relevance of these two but thought I better put here for completeness.
Event ID 10015, Application, MSExchange Mid-Tier Storage
Active Manager Client already doing query for object ‘SERVER’ on another thread, however this thread didn’t complete in 100 msec.

Event ID 4999, Application, MSExchangeCommon
Watson report about to be sent for process id: 2284, with parameters: E12IIS, c-RTL-AMD64, 15.00.1236.003, M.E.Search.Service, unknown, M.E.D.D.A.SessionSettingsFactory.FromOrganizationIdWithoutRbacScopesServiceOnly, System.ArgumentNullException, a135, unknown.
ErrorReportingEnabled: True

This situation loops and loops, I had to disable the Microsoft Exchange Search service to stop the process until I could resolve the issue.

Resolution

I have resolved this issue by following these steps;

  1. Double Checked there was a Group in AD named ContentSubmitters.
    Ensured that ‘Domain Admins’ and ‘NETWORK SERVICE’ was added to Security tab with Full Rights.
  2. Download CU15 and extract to C:\Temp
  3. Run CMD as Administrator and complete the following commands
    C:\Temp\setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms
    C:\Temp\setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms
    C:\Temp\setup.exe /PrepareDomain /IAcceptExchangeServerLicenseTerms
    C:\Temp\setup.exe /IAcceptExchangeServerLicenseTerms
  4. Restart Server
  5. Enabled service ‘Microsoft Exchange Search’ (which was previously disabled due to main issue’.
  6. Run PowerShell as Administrator and complete the following commands
    C:\>stop-service MSExchangeFastSearch
    C:\>stop-service HostControllerService
  7. Navigate to the location of the content index folders for the database/s, this can be found out via the following PowerShell command. If you have multiple databases (as I do) may need to run for each database.
    C:\>Get-MailboxDatabase DB01 | select EdbFilePath
  8. In this path there will be a folder named after the GUID of each database. Delete the folder/s
  9. Run PowerShell as Administrator and complete the following commands
    C:\>start-service MSExchangeFastSearch
    C:\>start-service HostControllerService
  10. The content indexes will be rebuilt, which can take some time. Eventually the content indexes were healthy again. Can use the following PS commands to check status;
    C:\>Get-MailboxDatabaseCopyStatus * | ft -auto
    C:\>Get-MailboxDatabaseCopyStatus | FL Name,*Index*
  11. I then double checked Event Viewer for any further errors.

ReadyNAS – Lost Permissions / No Access

December 12, 2016 Leave a comment

Problem
ReadyNAS 2120 was exhibiting strange behaviour with permissions. Via network access the shared folders were visible but permission was denied when attempting to browse. When accessing the ReadyNAS via the web portal the shares would not show up at all. Sometimes when rebooting the web page would just refresh the front page and couldn’t access any of webpage to make changes or investigate. The odd time that configuration/share pages would appear, it would take ages for any changes to take affect.

Investigation/Explanation
I searched for a decent amount of hours as usually someone has experienced similar on the web however not much came up. There were a few similar scenarios, enough to suggest there was corruption and the only way was to factory reset to default. The problem with that is that it would also lose data. This was not really an option! I was unable to get into the ReadyNAS via web to allow any protocols (e.g. SSH, FTP etc.), however knowing that the share was viewable on the network I sort of assumed that SMB was working and perhaps just the AD permissions were not working.

Resolution
To resolve the issue I had to get the data off the NAS, which I did using the following steps, and using XXCOPY (which is my preferred copy tool, this would work with robocopy, xcopy etc.,) however XXCOPY has switches to do incremental per se;
Run CMD as Administrator…
c:\>cd Windows\System32
c:\Windows\System32>net use * /DELETE
c:\Windows\System32>net use \\172.16.0.100\BigFolder /user:Domain\Administrator
c:\Windows\System32>xxcopy \\172.16.0.26\BigFolder j:\Backups\BigFolder /H/K0/K/KS/E/BB/R/Q/Y/PB/C

Once data was backed up to external drive (plugged into my local lappy) I then performed a factory reset and started again, this time with a working backup 🙂