Archive | System Center 2012/2012R2 RSS for this section

Service Manager PowerShell Workflow: The operation has timed out

Recently I was working on a PowerShell Workflow for Auto Group Assignment for Service Requests and Incidents for a customer and ran into a snag… Effectively, although the script appeared to be running as it should and the Support Group would auto-assign I received the following error:

System.TimeoutException: The operation has timed out.
at Microsoft.EnterpriseManagement.TaskRuntimeManagement.ExecuteTaskInternal[T](IEnumerable`1 targets, Guid taskId, TaskConfiguration configuration)
at Microsoft.EnterpriseManagement.TaskRuntimeManagement.ExecuteTask[T](IEnumerable`1 targets, ManagementPackTask task, TaskConfiguration configuration)
at Microsoft.ServiceManager.WorkflowAuthoring.ActivityLibrary.TaskExecutor.RunTask(String sdkServerName, Guid taskId, IList`1 taskTargetIds, Dictionary`2 taskArguments, Int32 taskTimeout)
at Microsoft.ServiceManager.WorkflowAuthoring.ActivityLibrary.RunTaskActivity.Execute(ActivityExecutionContext executionContext)
at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)
at System.Workflow.Runtime.Scheduler.Run()

Multiple re-creations of my workflow using the authoring tool and manual script execution showed the script was working as intended.  It turns out the issue was that the SQL Broker wasn’t enabled for the ServiceManager Database, thankfully an easy fix.

  • Stopped the SQL Server Agent
  • Stopped all Service Manager related services
  • Checked the Broker Enabled Status via SQL Query: SELECT is_broker_enabled from sys.databases WHERE name = ‘ServiceManager’.  This returned 0 (it wasn’t enabled)
  • Ran the Query ALTER DATABASE ServiceManager SET ENABLE_BROKER.  Note:  If it times out due to a lingering connection, “alter database ServiceManager set enable_broker with rollback immediate;” does the trick too.
  • Restarted the SQL Server Agent and all Service Manager services and Workflows now run as expected!

HP 3PAR with SCOM “Unable to connect to the remote server”

I was recently configuring the HP 3PAR monitoring for System Center Operations Manager 2012R2 for a customer.  After installing the HP SCOM Storage Monitoring Application and adding my local management server, I proceeded to add the HP 3PAR Storage System via the HP Storage Management Pack User Configuration Tool and received the error “Unable to connect to the remote server” whenever I attempted to connect.

2016-09-26_17-28-13

I confirmed my username/password was correct – even setup a new account, all with the same result.  It turns out that the CIM service was not running on the 3PAR, which is a requirement for the application/management pack to function.  I connected to the storage system via SSH, typed “showcim” and confirmed that it was not functional.  Running the command “startcim” started the CIM service and after a few seconds, returned to the HP Storage System dialog and re-connect with success!

2016-09-26_17-41-08

2016-09-26_17-28-50

Finally, after enabling discovery in SCOM for the Management Server and waiting a couple of minutes the storage system appeared in SCOM and reported as it should.

2016-09-27_9-26-23

 

Hyper-V 2012R2/VMM 2012R2 VM “Missing” status/Stuck “Starting”

I recently had an issue with a customer with a missing VM after it was shutdown for maintenance.  As soon as it was started in Virtual Machine Manager, it immediately went to a Failed state and was missing from all Hyper-V hosts and stuck in a Failed state in Failover Cluster Manager.

2016-09-08_15-39-132016-09-08_15-38-27

The unusual thing was VMM and Failover Cluster manager thought they were on different hosts(follow steps below for the host that Cluster Manager says it should be on). The issue was due to the missing symbolic link to the VM on the Host it was missing from.

The fix was to re-create the symbolic link on the Host Failover Cluster Manager thinks it’s missing from.  First, find the GUID for your VM from the source location for the specific VM config:

2016-09-08_16-01-23

Then browse to C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines on the host in question.  Run CMD prompt as Administrator from the folder and create the link with the following command: Mklink FA060046-2C40-4D8E-A6EA-2A4587A84B3B.xml “C:\ClusterStorage\Volume4\YOURVMNAMEHERE\Virtual Machines\FA060046-2C40-4D8E-A6EA-2A4587A84B3B.xml”

Then grant permissions:

icacls “%SYSTEMDRIVE%\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines\FA060046-2C40-4D8E-A6EA-2A4587A84B3B.xml” /grant “NT VIRTUAL MACHINE\FA060046-2C40-4D8E-A6EA-2A4587A84B3B”:F /l

2016-09-08_16-09-08

Finally restart the Hyper-V Virtual Machine Service and all should be back online!

2016-09-08_15-44-13

2016-09-08_15-44-43

SCCM 2012R2 Distribution Point Transfer/Setup Error: 0x80040154

While troubleshooting package transfer failures for a Distribution Point in a customer’s SCCM 2012R2 environment, I came across the following errors:

ExecStaticMethod failed (80040154) SMS_DistributionPoint, AddFile SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)
CSendFileAction::AddFile failed; 0x80040154 SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)
Failed to add the file ccmsetup.cab in content SKI00003.4. Error 0x80040154 SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)
CSendFileAction::AddFileMetaData failed; 0x80040154 SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)
CSendFileAction::SendFiles failed; 0x80040154 SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)
CSendFileAction::SendContent failed; 0x80040154 SMS_PACKAGE_TRANSFER_MANAGER 10/06/2016 2:18:13 a.m. 12632 (0x3158)

2016-06-09_10-56-42

After researching the issues and checking installed components on the Distribution Point, I noticed that IIS had been manually configured and the RDC (Remote Differential Compression) feature wasn’t added to the server.

2016-06-09_13-34-19

After installing this and triggering a distribution to this DP everything kicked into life.

2016-06-09_13-35-05

2016-06-09_13-45-42

MS Operations Management: Error 2013 Unable to register to Advisor Service

I was recently working with a customer to integrate Microsoft Operations Management Suite into their SCOM environment and noted they received the following error when connecting SCOM to MOMS:

Error 2013: Unable to register to Advisor Service.  Please contact the system administrator.

2016-05-18_10-38-00

Thankfully this is a relatively simple fix.  While they had upgraded their SCOM environment to the latest patch (UR9 at the time of this post), they had not imported the updated Management Packs (C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Management Packs for Update Rollups) as part of the upgrade process.  These Management Packs include the update to the old Advisor Server – now known as “Operations Management Suite.”  Simply adding the new MP’s resolves the issue:

2016-05-18_10-40-47

Prepopulate Office 365 servers for Lync/Skype for Business Client

I’m currently working through a Lync cut-over for a customer to Office 365.  As they are moving from a single on-premise Lync 2010 server and didn’t wish to go the hybrid route, we are running through a cut-over process for users.  Ideally we would like to do this with no interruption to existing users (ie, updating DNS settings for all).  As such, would like to manually configure the test users to point directly to Office 365 Lync Online/Skype for Business – preferably with as little hands on as possible.  Thankfully, these settings can be configured in the registry.  These registry keys to set can be found:
under HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Lync.

2-10-2015 11-49-58 a-m- 2-10-2015 11-53-12 a-m- 2-10-2015 12-00-59 p-m- 2-10-2015 12-03-07 p-m-

I am packaging these settings into the installer for SCCM deployment but this could easily be adjust using a GPO, script, etc.

Unable to start “System Center Data Access Service” on Service Manager Data Warehouse

I was recently faced with an issue for a customer where their Service Manager Data Warehouse Service would start but not stay running in their lab environment.  The following three events were logged in the Operations Manager event log:
14-04-2015 12-33-51 p-m-
Event ID: 26325
An authorization store exception was thrown in the System Center Data Access service. Exception message: Unable to perform the operation because of authorization store errors.

14-04-2015 12-36-04 p-m-
Event ID: 26339
An exception was thrown while initializing the service container.
Exception message: Initialize
Full exception: Feature of type ‘Microsoft.EnterpriseManagement.ServiceDataLayer.IAuthorizationFeature, Microsoft.EnterpriseManagement.DataAccessService.Core, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ cannot be added to the container.
14-04-2015 12-36-15 p-m-
Event ID: 26380
The System Center Data Access service failed due to an unhandled exception.
The service will attempt to restart.

Exception:
Microsoft.EnterpriseManagement.ConfigurationReaderException: Feature of type ‘Microsoft.EnterpriseManagement.ServiceDataLayer.IAuthorizationFeature, Microsoft.EnterpriseManagement.DataAccessService.Core, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ cannot be added to the container. —> System.ServiceModel.FaultException`1[Microsoft.EnterpriseManagement.Common.UnknownAuthorizationStoreException]: The creator of this fault did not specify a Reason. (Fault Detail is equal to Microsoft.EnterpriseManagement.Common.UnknownAuthorizationStoreException: Unable to perform the operation because of authorization store errors. —> System.Runtime.InteropServices.COMException (0x80070539): The security ID structure is invalid. (Exception from HRESULT: 0x80070539)
at Microsoft.Interop.Security.AzRoles.AzAuthorizationStoreClass.Initialize(Int32 lFlags, String bstrPolicyURL, Object varReserved)
at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AzManHelper.Initialize(String pathToStore, String appName, AzManHelperModes helperMode, String storeDesc, String appDesc)
— End of inner exception stack trace —
).
— End of inner exception stack trace —
at Microsoft.EnterpriseManagement.ConfigurationReaderHelper.ReadFeatures(XPathNavigator navi, IContainer container)
at Microsoft.EnterpriseManagement.ConfigurationReaderHelper.Process()
at Microsoft.EnterpriseManagement.ServiceDataLayer.DispatcherService.Initialize(InProcEnterpriseManagementConnectionSettings configuration)
at Microsoft.EnterpriseManagement.ServiceDataLayer.DispatcherService.InitializeRunner(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart(Object obj)
Thankfully I came across the following blog post which pointed me in the right direction: http://blogs.technet.com/b/servicemanager/archive/2011/10/04/system-center-data-access-service-start-up-failure-due-to-sql-configuration-change.aspx

I checked the SQL security method and discovered it was set to mix Windows/Sql Auth.  I then found a sqltest user that was set as db_owner for all three of the Service Manager Data Warehouse Databases.  Removed the DB_Owner role from each DB for the sqltest user and restarted the System Center Data Access service with no further issues.

14-04-2015 12-36-31 p-m- 14-04-2015 12-37-33 p-m-

14-04-2015 12-43-17 p-m-

SCSM: “An error was encountered while running the task”

I recently came across an unusual issue with Service Manager were all of the connectors stopped functioning.  Any time I went into the properties or attempted to create a new connector (in particular when starting to create and selecting test authentication), the connector would crash with the following error:

Application: System Center Service Manager
Application Version: 7.5.3079.0
Severity: Error
Message: An error was encountered while running the task.

scsm-error

Really helpful message…

I had seen a similar issues previously with the Exchange Connector due to a corrupted Default SR or IR Template.  As this was in the sandbox environment, I started removing custom Management Packs as a test – all that had been created since the connector had stopped working.  None of this resolved the issue.

The issue it turned out was with the System Center Operations Manager Agent installed on the SCSM server.

The Microsoft SCOM Agent is bundled in with Service Manager by default.  I have ran into several random issues before with System Center Operations Manager, where I needed to rebuild the Health Service State folder by stopping the “Microsoft Monitoring Agent” and renaming this folder.  On the SCSM server, the service is still the same but the Health State is in a different location.

First, stop the service:

scom-agent-service
Rename the SCOM Health Service State Folder (Located: C:\Program Files\Microsoft System Center 2012 R2\Service Manager\Health Service State)
health-service-state
Restart the Microsoft Monitoring Agent (this will recreate the Health Service State folder).

You should now be able to create connectors!

Remove assigned user from Service Manager Incident or Service Request with Orchestrator

I was recently given a customer request to remove the assigned user on an Incident or Service Request (in order to place it back into the unassigned job queue) in SCSM 2012 R2 when their external support returned their Service Manager job back to them via e-mail.  As the external vendor was utilizing their own ticketing system, all of this transfer was occurring via email communication and the Exchange Connector for SCSM.  Standard Exchange connector keywords were being utilized (ie. [Resolved], [Completed]), etc. but to remove via any non-standard keyword we needed to use Orchestrator and the SC 2012 Service Manager Integration Pack.
scorch-scsm
First step (Monitor Analyst Comments) was to create a monitor task for new Trouble Ticket Analyst Comments, with a filter for specific Keyword – in this case: “Assignee Group: Internal IT Department”
scorch-scsm1

Next step (Get Parent IR) was to create an SCSM Relationship between the Trouble Ticket Analyst Comments and the Parent Incident
scorch-scsm3

Next we Get the Relationship between the parent incident and the Active Directory User (Get Assigned User from IR)
scorch-scsm4

Finally (Remove Assigned User Relationship from IR) we remove the relationship for the AD User from the Incident, which places the job back as unassigned
scorch-scsm5
Simply repeat the steps above for Service Requests, changing the class where appropriate from Incident to Service Request.

Now whenever the Analyst Comments are added that contain the words “Assignee Group: Internal IT Department” to an Incident or Service Request, either via comment or e-mail the assigned to user is removed. Done!

Remove “Unknown” computer from System Center Configuration 2012 R2 Manager PowerShell

Recently while testing out some task sequences with non-domain systems, I ended up with a bunch of “Unknown” computers in the Devices list in SCCM.  While I could filter this and remove it easily from the console, I wanted to set a task sequence to continually run through process while testing.

First, open PowerShell from Configuration Manager (to easily load the SCCM PowerShell Modules)
removeunknowndevice1

#Define your 3 character site code with this variable, in my example “ABC.”  I have defined the Site Code as I didn’t want the inherent x64 & x86 Unknown Computer Devices to be filtered from my results.  You could replace Unknown with another computer name, import a list via CSV, etc.

$sitecode = “ABC”
$devicename = “Unknown”
Get-CMDevice | Where-Object { ($_.SiteCode -ne $sitecode) -and ($_.Name -eq $devicename) } | Remove-CMDevice -force -confirm:$false

removeunknowndevice2

Note:  Please do this at your own risk – I’d suggest lab testing this first. 😉