Home » Orchestrator
Category Archives: Orchestrator
Contoso.se
Everything you read here is my own personal opinion and any code is provided "AS-IS" with no warranties.
MVP awarded 2007,2008,2009,2010
Service Manager Unleashed
Orchestrator 2012 Unleashed
Inside the Microsoft Operations Management Suite
How many instances of this activity do we have? And where are they?
From time to time there is a need to list all instances of a specific activity type in Orchestrator. For example it is not recommended to have any send platform event activities in production, instead we can use a SQL database for logging. There are two ways to investigate if there are any send platform events in the environment, review each runbook or ask the database. In this blog post I will show you how to list all instances of a specific activity type.
The first thing we need to do is to find the uniqueID for the type of activity in the OBJECTTYPES table. The following question will give us all info about the Send Platform Event, but you can query for any kind of activity.
select * from OBJECTTYPES where name = ‘Send Platform Event’
The OBJECTTYPE table will give us the uniqueID for the activity type, in this example 87E28B20-3E83-45E0-985A-FEEA6CE09084. The table will also tell us where to look for configuration data for these activities, in the PrimaryDataTables and SecondaryDataTables. If we want to list all Send Platform Events activities we have we can run the following query. The “AND (Deleted = ‘0’) part will make sure we only list activities that are not marked as deleted.
select * from OBJECTS where (ObjectType = ’87E28B20-3E83-45E0-985A-FEEA6CE09084′) AND (Deleted = ‘0’)
If we now want to add on in which runbook we are using them we can join the ParentID from the OBJECTS table with the uniqueID in the POLICIES table, like the following query. The Objects table will give us general information about each activity.
SELECT OBJECTS.Name AS [Activity Name], POLICIES.Name AS [Runbook Name] FROM OBJECTS INNER JOIN POLICIES
ON
OBJECTS.ParentID = POLICIES.UniqueID WHERE (OBJECTS.ObjectType = ’87E28B20-3E83-45E0-985A-FEEA6CE09084′) AND (OBJECTS.Deleted = ‘0’)
We now see activity names and runbook names. But what if we also want to include configuration of the Send Platform Event activity? Then we need to join the table seen in the first query, the PrimaryDataTable for the Send Platform Event activity type, example query
SELECT OBJECTS.Name AS [Activity Name], POLICIES.Name AS [Runbook Name], TASK_SENDOPALISEVENT.EventType, TASK_SENDOPALISEVENT.EventSummary, TASK_SENDOPALISEVENT.EventDetails FROM OBJECTS
INNER JOIN POLICIES ON OBJECTS.ParentID = POLICIES.UniqueID INNER JOIN TASK_SENDOPALISEVENT ON OBJECTS.UniqueID = TASK_SENDOPALISEVENT.UniqueID WHERE (OBJECTS.ObjectType = ’87E28B20-3E83-45E0-985A-FEEA6CE09084′) AND (OBJECTS.Deleted = ‘0’)
AS you can see some EventDetails and EventSummary includes a GUID. When we configure a Send Platform Event activity to publish data from the data bus we see the uniqueID of the activity in the text. As we can see in the figure many of my Send Platform Events will use data from the data bus.
Add a runboook activity from a runbook
Earlier this week I needed to add a runbook activity to a Service Request depending on the result from a review activity. I have written about adding activities to a service request from a runbook before, here. The difference is that this time I needed to add a runbook activity and that is a bit more complicated than adding for example a review activity or manual activity.
To add a runbook activity you need two activities, first one that gets the runbook (System Center Orchestrator Runbook Item) you want to use, the second activity creates a new related runbook activity (Runbook Automation Activity) to the service request. In this example I will add a Runbook Automation Activity that runs a runbook named “12.4 Build new VM”.
For the Create Related RB activity there are more to configure compared with the “Get RB Item” activity.
- Source Object Guid. This is the SC Object GUID from a Get Object activity that gets the Service Request that we want to add a Runbook Automation Activity to.
- Sequence ID. This field configure where, compared with already existing activities in the Service Request, this new activity will be added. The first activity in a Service Request has sequence ID 0, the second 1 and so on. In my service request I have one review activity and then a runbook activity. This new runbook activity will be added after both of them, sequence ID 2 in other words.
- ID. This is the id of the new runbook activity. {0} will give it next free serial number from Service Manager, “RB” is added to make sure it is named not just a number but also RB in front of the number.
- Is Ready For Automation is set to TRUE. This will allow the Runbook Activity to be triggered when the Service Request is created and the Runbook Activity becomes In Progress.
- Title. Title of the runbook Runbook Automation Activity, for example “Build new VM”
- Runbook Identifier. This is the ID of the runbook you want to run, you get the ID from the previous Get RB Item activity.
- Template Identifier. This is a bit more complicated to find, this is the Name of the template you have built for the Runbook Automation Activity in Service Manager. If you start Service Manager Shell (Service Manager PowerShell) you can run “Get-SCSMObjectTemplate | ft Name, DisplayName” and you will find the value you need. Most likely your template name starts with Template.
- Status. Status of the new activity, configure it to Pending
-
Property Mapping. This is parameters that we pass to the new Runbook Automation Activity.
- <NAME> is the name of the input parameter in the runbook, the parameter set in in the Initialize Data activity
- <ID> is the ID of the parameter in Orchestrator. Each parameter has a unique ID. You can list get the connection between parameter, activity and runbook with the following SQL query. In my example query I ask for all input parameters named “SR”.
SELECT OBJECTS.Name AS [Activity Name], POLICIES.Name AS [Runbook Name], CUSTOM_START_PARAMETERS.UniqueID AS [Parameter ID], CUSTOM_START_PARAMETERS.Value AS [Parameter Name] FROM CUSTOM_START_PARAMETERS INNER JOIN OBJECTS ON CUSTOM_START_PARAMETERS.ParentID = OBJECTS.UniqueID INNER JOIN POLICIES ON OBJECTS.ParentID = POLICIES.UniqueID WHERE (CUSTOM_START_PARAMETERS.Value = ‘SR’)
- <VALUE> the value I pass to the new runbook is the Runbook Automation Activity ID. Based on that ID you can find the for example related service request
- <RunbookID> is the ID of the Runbook Item. You can get it from the Get Runbook activity or you can use Service Manager Shell to get it. To get it with Service Manager Shell run “Get-scsmclass –name Microsoft.SystemCenter.Orchestrator.RunbookItem | get-scsmclassinstance | where-object {$_.Name –like “*12.4*”} | ft Name, Id
In this example I ask for a runbook activity that I know is named something with 12.4.
Those two activities is the two activities you need to add a Runbook Automation Activity. I notice that it works better if you also mark the current runbook as completed, from the runbook itself instead of from Service Manager. When adding activities in general I have notice that mark the current runbook as completed makes the service request process work better.
In the runbook you invoke you will get the Runbook Automation Activity SC GUID as input parameter. From there you can use a Get Relationship activity to find the related Service Request.
Once you have the Service Request you can read for example all use input data, in my example settings for a new virtual machine. User Input, everything the user has input in for example the self-service portal when submitting the service request, is stored in XML format. For example
To get only “Stockholm” you can use a Query XML activity
Summary: In this blog post we have looked at how we can use a Runbook Automation Activity in a Service Request to add another Runbook Automation Activity. We have also looked at how to pass data to the new runbook. This scenario is useful if you for example want to add activities based on the answer from a review activity.
Runbook PID
Yesterday I was investigating runbooks consuming much memory. We had a number of runbooks running Powershell scripts and we could see that the Runbook server was almost out of memory. When looking in Task Manager we saw a lot of PolicyModule.exe processes, some using a lot of memory, but we had no mapping between running runbooks and running PolicyModule.exe process. We needed to figure out which script/runbook is consuming all the memory.
This can be done in a couple of different ways. One way is to publish Runbook Process ID for any activity in a runbook. To use Runbook Process ID you need to enable “Show common Published Data” in the Published Data dialog box. The published data will show you the PID for the PolicyModule.exe process.
Another way to get the PID is to look in the Orchestrator database. The following SQL query will show you runbook name, start and stop time and process ID.
SELECT POLICYINSTANCES.TimeStarted, POLICYINSTANCES.TimeEnded, POLICYINSTANCES.ProcessID, POLICYINSTANCES.SeqNumber, POLICIES.Name
FROM POLICYINSTANCES INNER JOIN POLICIES ON POLICYINSTANCES.PolicyID = POLICIES.UniqueID
ORDER BY POLICYINSTANCES.TimeStarted DESC
How-to handle monitoring activities in SMA
In Orchestrator we often use a monitoring activity in the beginning of a runbook, for example monitor Service Manager for new incidents. In SMA we do this a bit different which I will explain in this blog post. Remember that you can connect Orchestrator and SMA together as shown in this blog post http://contoso.se/blog/?p=3857 , and get the best of both of them.
In general we need to write a runbook (Powershell workflow script) that runs every X minute and checks “a source” and then performs some “action”, as shown in figure 1. We could do this in one runbook but it might be a better idea to split it up, so we can re-use and share code in other scenarios.
Figure 1
In the following example, shown in figure 2, we monitor Service Manager for new incident that don’t have a support group configured, we do this every 10 minute and if we find a incident that don’t have support group set, we update it. The “SCSM_Search_IR” runbook pass the found incident ID and new support group to the “SCSM_Update_IR” runbook.
Figure 2
We build the “Monitor” runbook with input parameters for “Search” runbook and interval, so the same “Monitor” runbook can be used in multiple scenarios. The “Action” runbook have input parameters for the incident to update and support group to set, so we can reuse the runbook in other scenarios were we need to set support group for an incident. We can start the “Monitor” runbook multiple times, for example if we need to monitor Service Manager for incident with specific state every five minutes we can start a new instance of the “Monitor” runbook. If the action is to update the incident with new support group we can reuse the current “Action” runbook, just need to write a new “Search” runbook. This example is shown in figure 3.
Figure 3
Let’s look at runbook examples.
The “Monitor” runbook
This is the “Monitor” runbook that runs based on an interval and starts a “Search” runbook.
workflow blog_monitor {
param (
# Mandatory parameter of type INT, number of seconds between loops
[parameter(Mandatory=$true)]
[int]$Interval,
# Mandatory parameter of type String, name of search runbook
[parameter(Mandatory=$true)]
[string]$SearchRunbook
)
$WAP = Get-AutomationVariable -Name ‘SMA-WebServiceEndpoint’
while($true)
{
start-sleep -s $Interval
Start-SmaRunbook -Name $SearchRunbook -WebServiceEndpoint $WAP
}
}
The “Search” Runbook
This is the “Search” runbook. It search for incidents created since last run with blank support group setting. The runbook use a variable (blog_search_SCSM) to keep track of processed incidents. If it finds an incident it starts the “SCSM_SetTier” runbook and pass the incident ID and new support group (tier).
workflow blog_search_SCSM
{
$SCSMServer = Get-AutomationVariable -Name 'Service Manager server'
$fromdate = Get-AutomationVariable -Name 'blog_search_SCSM'
$wap = Get-AutomationVariable -Name 'SMA-WebServiceEndpoint'
inlinescript {
$class = Get-SCSMClass -computername $using:SCSMServer -Name System.Workitem.Incident$
$incidents = Get-SCSMObject –Class $class –filter "CreatedDate > $using:fromdate" -computername $using:SCSMServer | where{$_.tierqueue -eq $null}
ForEach ($incident in $incidents) {
Start-SmaRunbook -Name scsm_set_tier -Parameters @{"IRid"=$incident.ID; "Tier"="Tier 1"} -WebServiceEndpoint $using:wap
}
}
$fromdate = get-date -format G
Set-AutomationVariable -Name 'blog_search_SCSM' -Value $fromdate
}
The “Action” runbook
This is the runbook that sets support group for an incident
workflow scsm_set_tier
{
param (
# Mandatory parameter of type String, id of Incident
[parameter(Mandatory=$true)]
[string]$IRid,
# Mandatory parameter of type String, name of new support group
[parameter(Mandatory=$true)]
[string]$Tier
)
$SCSMServer = Get-AutomationVariable -Name 'Service Manager server'
inlinescript {
$class = Get-SCSMClass -computername $using:SCSMServer -Name System.Workitem.Incident$
$incident = Get-SCSMObject –Class $class –filter "ID -eq $using:IRid" -computername $using:SCSMServer
$incident | Set-SCSMObject -ComputerName $using:SCSMServer -Property tierqueue -Value "$using:Tier"
}
}
Test it
To start these runbooks start the “Monitor” runbook and fill in the parameters. After a while we can see the different runbooks are running or has ran
These examples are written in a very “raw” format, to show clean examples. It is a good idea to add for example error handling, logging and parameter validation. In some scenarios it can also be an idea to combine some of the runbooks. When you use a variable for “checkpoint” it is important to understand that only one instance of the runbook can be running. In this scenario it is not an issue as we will only have one runbook running looking for new incidents without support group.
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
Building a SMA activity for Service Manager
Service Manager and Orchestrator are well connected with a connector. It is easy to use Orchestrator runbooks as activities in for example change requests and service requests. When SMA was released as part of Orchestrator 2012 R2 a number of customers asked how to use SMA runbooks as activities in Service Manager. Some runbooks running in (classical) Orchestrator would fit better in SMA, for example long running jobs that can benefit from check pointing and also jobs that need to scale large.
In this blog post I will show you a way to quickly create an activity for Service Manager to start a specific runbook. The example will trigger a SMA runbook from Service Manager through the SMA web service. SMA will then execute the runbook and update the Service Manager SMA activity, mark it as completed. In this example we will invoke a SMA runbook that build new VMs in Windows Azure. We could build a new VM with the Windows Azure integration pack in Orchestrator runbook designer too, but it is handy to use Powershell in this scenario and if we want to build multiple VMs at the same time SMA can use parallel execute to save some time.
These are the steps we need to take
- Install SMA powershell module on the Service Manger management server running workflows
- Install SMLets on the Service Manger management server running workflows
- Create a new activity that starts a SMA runbook in a new management pack
- Import the management
- Use the new activity in any work item
SMLets is a PowerShell module built by the community. It can be used to automate common tasks in Service Manager. You can download SMLets here. The SMA PowerShell module can be found on the Orchestrator 2012 R2 installation media.
Building the SMA activity
First step is to build the SMA activity. We can use the Service Manager Authoring Tool for that.
- Start the Service Manager Authoring Tool
- Service Manager Authoring Tool, click File and select New
- New Management Pack, input the name for the new management pack, for example Contoso.SMA. Click Save
- In the Management Pack Explorer, right-click Classes and select Create Other Class
- Base Class, select Activity and click OK
- Create Class, input Contoso.SMA.DeployVM, click Create
-
In the class properties list, delete the default property named Proerty_XX
We will create two new properties on the new class, on for virtual machine name (VMName) and one for virtual machine size (VMSize). We will configure the VMSize property with data type List. In Service Manager we will configure a list of values used to select virtual machine size.
- Click Create property and input VMName as internal name, click Create
- Click Create property and input VMSize as internal name, click Create
- In the Details pane for the VMSize property, change Data Type to List
- In the Select a list dialog box, click Create List
- Create List, input VMSizeList as internal name and VM Size as Display name. Click Create
- In the Select a list dialog box, select the new VM Size list, click Ok
- In the Management Pack Explorer, right-click Workflows and select Create
- Create Workflow Wizard, General, input ContosoSMAInvokeRunbook as name. Click Next
- Create Workflow Wizard, Trigger Condition, select Run only when… click Next
- Create Workflow Wizard, Trigger Criteria, click Browse and select the Contoso.SMA.DeployVM class. Change Change event to When an object of the selected class is updated, click Additonal Criteria
- Pick additional criteria, click the Change To tab, add Criteria as
[Activity] Status equals In Progress
Click Ok
- Create Workflow Wizard, Trigger Criteria, click Next
- Create Workflow Wizard, Summary, click Create
- Create Workflow Wizard, Completion, click Close
- Once you click Close the workflow designer will be displayed. Add a Windows PowerShell Script to the workflow
- We will configure the WindowsPowerShell activity to run a Powershell script to start the SMA runbook. Select the WindowsPowerShell activity, in the Details pane click Script Body
- Configure a Script Activity, paste the following script to the Script Body text fieldimport-module SMLets
$size1 = “$InstanceSize”
$size2 = $size1 -replace “{“, “”
$size3 = $size2 -replace “}”, “”
$size4 = get-scsmenumeration -id $size3
$SizeName = $size4.DisplayName
Start-SmaRunbook -WebServiceEndpoint https://wap01 -Name “SCSM__Deploy_New_VM” -Parameters @{“InstanceSize”=”$SizeName”; “VMName”=”$VMName”;”ActivityID”=”$ActivityID”}The script first imports the SMLets PowerShell module. The SMLets module is used to convert the GUID for the list value (enum value) into displayname. For example from Service Manager the workflow gets {300dfe04-a0a0-e5fc-8892-963e7146d86c} instead of Medium, as VM Size. Before we can convert the GUID into a displaystring we must remove “{” and “}”, which the two replace lines do. We then start the runbook and pass on three parameters, VM name, VM Size and the activity ID. This example script starts a SMA runbook named SCSM__Deploy_New_VM and the SMA web service has adress https://wap01 - Configure a Script Activity, click Script Properties. Create three parameters. Use class properties from the Contoso.SMA.DeployVM class for all three parameters. Use the ID property from the class for the ActivityID property. Click OK
- The activity and management pack is now ready to import. Great work! Save the management pack in the Service Manager Authoring Tool
- Copy the workflow dll, ContosoSMAInvokeRunbook.dll from the management pack folder to Service Manager installation folder (C:\Program Files\Microsoft System Center 2012 R2\Service Manager) and then restart the Microsoft Monitoring Agent on the Service Manager management server running workflow
- You can now open Service Manager console and import the management pack
- In the Service Manager Console, browse to Library and Lists, open the VM Size list
- List Properties, add the following items, that are all Azure VM sizes, click OKExtraSmall
Small
Medium
Large
ExtraLargeFor more information about Windows Azure VM sizes look at http://www.windowsazure.com/en-us/pricing/details/virtual-machines/ - In the Service Manager Console, browse to Library and Templates, click Create Template
- Create Template, input a name, for example Contoso SMA Deploy Azure VM. Select Contoso.SMA.DeployVM as Class and click OK
- Contoso.SMA.DeployVM Properties, input Contoso SMA Deploy Azure VM as Display Name and click OK
- You can now use the new activity anywhere you like, for example service request templates.
But what about the SMA runbook?
SMA Runbook
The SMA runbook is attached to this blog post. The important part of the runbook, or the really interesting part of it, is the last part of the runbook. This part creates a remote PowerShell session to the Service Manager server (SM01) and marks the activity as completed, the work item will now move on to the next activity.
Inlinescript {
$session = new-pssession -ComputerName SM01
Invoke-Command -Session $session {
$workitem = Get-SCSMObject -Class (get-scsmclass -name Contoso.SMA.DeployVM) | Where-Object {$_.ID -eq “$using:ActivityID”}
$workitem | Set-SCSMObject -Property Status -Value Completed
}
In the Windows Azure Pack portal you can see the SMA runbook start and you can also see the input parameters coming from Service Manager
Summary
This post show you an example of how to invoke a SMA runbook from Service Manager with a custom activity.
This example is written in a very simply way, no complicated configuration or requirements. For example you might want to add error handling in the script, handle situation when SMLets can’t be loaded or Service Manager can’t be contacted. In both those cases you want to set the activity status to failure in Service Manager. Also it is a good practices to seal management packs that include class structure.
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
Automate failover in SMA with Operations Manager
In this post I talked about hos failover in SMA works. As you could read in that post there is a manual step to transfer jobs from one worker to another. This process could of course be automated in a couple of different ways, you can use Orchestrator for example. Today I will show how it can be done with Operations Manager.
I have setup a rule in Operations Manager that runs a command every five minutes. The command starts a Powershell script that checks how all workers are doing. First it checks which workers there are in the environment and then it checks if the Runbook Service is running on each of them.
If any of the workers, I have two in my sandbox, is not working the script will exclude them from the SMA configuration. The script will also stop and start the Runbook service (rbsvc) on the working worker server. If the script do any changes it will generate an event in the Application event. That event is picked up by another rule that generate an alert in Operations Manager.
If I then run Get-SmaRunbookWorkerDeployment I can verify that SMA02 is the only worker in my environment. Also I can see in my log table that the runbook is resumed on the SMA02 worker.
A couple of comments around the script, first, in the script I have hardcoded https://wap01 , WAP01 is my SMA web service. The SMA management pack discover this component so the script could find the web service hostname from that discovery, it could also find the workers based on default discoveries in the SMA management pack. Second, if a worker goes offline this script will exclude it. But the script will not include the worker again when it comes back online. That has to be done manual or with a updated version of the script.
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
State tracking and failover in SMA
As you might know a runbook restarts at the first activity when it fails over to secondary runbook server in Orchestrator. The failover mechanism is automatically in Orchestrator, but restarting at first activity is a challenge. Read more about state tracking in Orchestrator here.
SMA do this a bit different A great news for all Orchestrator administrators is that you can create checkpoints in your runbooks. A checkpoint is a snapshot of a running runbook or workflow, including variable values, output and everything done until that point. You can place a checkpoint anywhere you want in a runbook, but each time you do a checkpoint it cost a bit of performance and storage. A best practices is to add a checkpoint after each important part of the runbook that you don’t want the runbook to re-run if it is restarted. For example if you have a runbook building a virtual machine and then configure it you might want to do a checkpoint after the virtual machine is created. If the runbook is interrupted it will continue with the configuration when it is resumed. As checkpoints are stored in the SMA database they are worker independent, meaning that one worker can start the workflow, then if interrupted and another worker can pick up at latest checkpoint. Another nice benefit with checkpoints is that you can suspend a running runbook. For example if you need to do maintenance on the worker or if the runbook is interrupted due to a network error, you can suspend it, fix the network error and resume the runbook.
When a worker (Runbook Service) starts it decide which queue “slots” to process. Let’s say you have two worker servers, then they will pick up half of the “slots” each. If one worker goes offline you need to update runbook worker deployment settings to only include one worker. Else there is a risk that jobs will not be picked up and suspended jobs will not be resumed. This re-configuration is done with Powershell, the New-SmaRunbookWorkerDeployment cmdlet. As the worker service configure queue “slots” when starting it is important to stop all workers before running new-smarunbookworkerdeployment cmdlet, else there is a risk that jobs will be picked up multiple times or jobs become corrupt. One important thing to note about that cmdlet is that New-SmaRunbookWorkerDeployment will replace the existing settings. If you want to add one worker you need to run New-SmaRunbookWorkerDeployment and add both the old and new worker. The following script adds a worker named SMA02 to the existing configuration. WAP01 is the machine running SMA web service.
$youbService = “https://wap01” $workers = (Get-SmaRunbookWorkerDeployment -YoubServiceEndpoint $youbService).ComputerName if($workers -isnot [system.array]) {$workers = @($workers)} $workers += “SMA02” New-SmaRunbookWorkerDeployment -YoubServiceEndpoint $youbService -ComputerName $workers
If you have two workers, SMA01 and SMA02 and SMA01 goes offline you can run the following command to remove SMA01 and configure only SMA02 as worker
New-SmaRunbookWorkerDeployment -WebServiceEndpoint https://wap01 -ComputerName SMA02
Once you have run the New-smarunbookworkerdeployment cmdlet you can start the worker services again.
Quick summary
- Make sure to include checkpoints in your runbooks
- Make sure to have multiple workers
- When one worker goes offline, or you need to do maintenance, use New-SmaRunbookWorkerDeployment to remove/add workers. But first stop all running workers (Runbook Service)
- Start all workers (runbook service)
Common questions
Q: What if I need to run new-smarunbookworkerdeployment when I have running runbooks?
A: It is possible to configure drain time. Drain time can be up to 20 minutes and is started when you stop the runbook service. During drain time the worker will not pick up any new jobs and running jobs will suspend if possible (if they have a checkpoint). Runbooks without a checkpoint will continue to run and might be interrupted when the runbook service stops after drain time.
Q: How many checkpoints can I create in a runbook?
A: You can create as many as you like, but SMA will only use the latest one. You can only resume at the latest one.
Q: What will happened if a runbook have no checkpoints and it is moved to another worker?
A: If there are no checkpoints in the runbook and it is transferred to another worker, then the runbook will restart at the second worker
Q: how do I create a checkpoint?
A: Include “Checkpoint-Workflow” in your runbook
Q: I see “Queued” as job status after I reconfigured workers, seems like no jobs are running?
A: This can happened if no worker is running, make sure at least one of the workers are running
Q: When I tries to start the Runbook Service on a worker I get an error in the System log saying “The Runbook Service service terminated with the following error: Incorrect function”
A: Most likely you are trying to start a worker that is not in the runbook worker deployment settings. Re-configure with new-smarunbookworkerdeployment
Q: Is it possible to automate the “new-smarunbookworkerdeployment” part?
A: Yes! See this blog post for example
Example
In this example I have two runbook workers, SMA02 and WAP01. I have a runbook named test_failover. This runbook runs 20 loops. For each loop it writes a timestamp to a SQL table, creates a checkpoint and waits two minutes.
In the SQL table I can see the test_failover runbook writing rows, one for each loop.
If I now stop the runbook service on SMA02 I can see that the Runbook service is “Stopping” for 10 minutes. Drain time in my environment is 10 minutes. The test_failover runbook stopped at loop number 4.
I stop the Runbook Service on WAP01 and run “New-SmaRunbookWorkerDeployment -WebServiceEndpoint https://wap01 -ComputerName wap01″ to remove SMA02 from my configuration. Then start the Runbook Service on WAP01 again. I can see that the runbook now running on WAP01. As you can see in the figure loop 4 and 3 was written same second. When I stopped the Runbook Service loop 3 was just about to be written by the “WriteLog” runbook, but it was queued until WAP01 resumed the jobs.
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
Copy or rename a SMA runbook
You may have notice that in the Windows Azure portal there is no way to copy or rename a SMA runbook. But it is of course possible with a small Powershell script. The script in this blog post will copy a source SMA runbook and store all settings in a new SMA runbook. The script will also import the new runbook and delete the old one. You can comment (#) the last part if you just want to copy your source runbook and not delete it. If you keep the last part, delete source runbook, then the result of the script will be a rename of the runbook.
$path = "C:\temp"
$targetrunbookname = "newdebugexample"
$sourcerunbookname = "debugexample"
$WebServiceEndpoint = "https://wap01"
##
## Get source runbook
##
$sourcesettings = Get-SmaRunbook -WebServiceEndpoint $WebServiceEndpoint -Name $sourcerunbookname
$source = Get-SmaRunbookDefinition -WebServiceEndpoint $WebServiceEndpoint -name $sourcerunbookname -Type Published
##
## Create the new runbook as a file with source workflow and replace workflow name
##
New-item $path\$targetrunbookname.ps1 -type file
add-content -path $path\$targetrunbookname.ps1 $source.content
$word = "workflow $sourcerunbookname"
$replacement = "workflow $targetrunbookname"
$text = get-content $path\$targetrunbookname.ps1
$newText = $text -replace $word,$replacement
$newText > $path\$targetrunbookname.ps1
##
## Import new runbook to SMA and set runbook configuration
##
Import-SMArunbook -WebServiceEndpoint $WebServiceEndpoint -Path $path\$targetrunbookname.ps1 -Tags $sourcesettings.Tags
Set-SmaRunbookConfiguration -WebServiceEndpoint $WebServiceEndpoint -Name $targetrunbookname -LogDebug $sourcesettings.LogDebug -LogVerbose $sourcesettings.LogVerbose -LogProgress $sourcesettings.LogProgress -Description $sourcesettings.Description
##
## Delete sourcerunbook
##
Remove-SmaRunbook -WebServiceEndpoint $WebServiceEndpoint -name $sourcerunbookname -Confirm:$false
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
Logging in Service Management Automation (SMA)
In Service Management Automation (SMA) there are a couple of different ways to write to a log. In this blog post I will show you how to write to the job history tab and how to write to an external SQL database.
If you have published a runbook you can use Write-Verbose, Write-Progress and Write-Debug as shown in figure 1 to write log message. The result of write actions will be shown in History for each runbook job, example shown in figure 2. Figure 3 show important settings to make this work, you need to enable logging on the runbook, else you will see no log messages in the job history.
Figure 1 Write to log examples
Figure 2 Log examples
Figure 3 Log settings
Another alternative for logging is logging to an external database. We have earlier used the same logging solution for Orchestrator runbooks. Figure 4 shows a SMA runbook that writes to a SQL database named SMALOG. As you can see in figure 4 the “writelog” SMA runbook use three parameters, Runbook, Job and Description. Other runbooks invoking this runbook need to provide current runbook, current job ID and a log message description. To get the current runbook name and job ID you can use the code shown in figure 5. The writelog runbook use the Get-Date to get current time. The log database is shown in figure 6.
Figure 4 SMA runbook writes logs to SQL database
Figure 5 Get current runbook name and job ID
Figure 6 Example of log database
Download examples here. Logging . Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.
Pass information between Orchestrator runbook and SMA runbook
Difficult to come up with a good blog post title, as Service Management Automation (SMA) is part of Orchestrator 2012 R2, so in some way a SMA runbook is also a Orchestrator runbook J This blog post will show you how to start a SMA runbook from Orchestrator and then pass information from the SMA runbook to Orchestrator runbook data bus.
From Orchestrator we can start a SMA runbook with a Run .NET Script activity, shown in figure 1. In this example we start a SMA runbook named “justadelay”. “Justadelay” is a small SMA runbook that waits a minute and then sends back three variables, just an example to show how this integration can be done. Figure 3 show the SMA runbook “justadelay”.
In this example I run PowerShell remote session from Orchestrator runbook to a machine named WAP01. WAP01 is my SMA and Windows Azure Pack (WAP) server with the SMA Powershell module installed. You could of course run the script locally on your Orchestrator runbook server if you have SMA PowerShell module installed. The script includes a “while” loop that checks if the SMA job is completed, once it is completed we get all output from the SMA runbook job and insert it into an array named “output”. The “while” loop will make sure the Run .NET Script activity (in the Orchestrator runbook) waits until the SMA job is completed. The entire command (invoke-command) is within a variable named SMAoutput. SMAoutput is the variable we publish to the data bus in the Orchestrator runbook, figure 2.
The Send Platform event activity in the Orchestrator runbook (figure 1) is used to output the data returned by the SMA runbook, figure 4 shows one of three events.
Figure 1 Start SMA runbook from Orchestrator
Figure 2 Publish output to data bus
Figure 3 SMA runbook
Figure 4 Data from SMA runbook
Note that this is provided “AS-IS” with no warranties at all. This is not a production ready solution, just an idea and an example.